Slashdot Mirror
Lead Developer of SPF Anti-Spam Scheme Interviewed
penciling_in writes "CircleID has a great two-part interview with Meng Wong, lead developer of the anti-spam authentication scheme Sender Policy Framework (SPF). He has responded to various questions (which also touches on issues previously raised by Slashdot folks), including the merger with Microsoft's Caller ID, incompatibility of SPF with email forwarding services, and what he thinks about Yahoo's DomainKeys, as well as where he believes the fight against spam is headed. (He has also confirmed that the name SPF and references to sunblock are intentional!) In response to the first question in the interview on how SPF got started, Meng says: 'In 2002 Paul Vixie, the brains behind BIND, wrote a short paper titled 'Repudiating Mail-From'. That inspired two other proposals, 'Reverse MX' by Hadmut Danisch and 'Designated Mailer Protocol' by Gordon Fecyk. In late 2003 I combined the best of both proposals and called the result SPF.' Vixie replies to this reference in comments following the first article."
No these schemes are designed to work with SMTP.
It would be very difficult to replace something as as popular as SMTP. It would at least have to be backwards compatible with SMTP which means it would expose itself to the same problems as SMTP.
IMO SMTP does a good job apart from its obivous faults.
for every article about new anti-spam schemes, I would now be richer than I would have been had I gotten $0.01 for every spam mail urging me to buy penis-enhancing pills.
My understanding of SPF is this:
the recipient checks that the sender has authoritiy to send out email for the domain, i.e. if you send an email from whatever.com via SMTP server 123.123.123.123, the recipient checks that 123.123.123.123 has the authority to send email for whatever.com by checking it's SPF record (which similar to an ordinary DNS record).
So, we all have to set up SPF records for our domains or our emails will get rejected by some ISPs. Is my understanding right?
IANAL, but the text of this agreement seems to indicate that this implementation license applies to any products that "implement and are compliant with" Sender ID (section 1.2), and that Microsoft may subsequently terminate the license (section 3).
Anybody familiar with this? Is there a RFC for Sender ID?
IMO Windows does a good job apart from its obvious faults too.
"Armed forces abroad are of little value unless there is prudent counsel at home" - Cicero
Paul Vixie's comment:
I'd have to disagree with Paul Vixie here. Most of the spam today comes through compromised home machines on a broadband line. Of course, spammers could include the zombie they're using in their SPF record and use their own domain in the "Purported Return Address", but that would make them so traceable that they might as well spam from their own systems.
So I'd definitely disagree that SPF/SenderID will not discourage spammers. It will certainly discourage the worst of them: the guys who don't want to be found out.
Great, SPF now not only protects from the sun, but from spam as well. I bet we could get XML involved somehow, too.
OK, I'm worried; unless I've completely missed something here, it seems as though the 'little guy' could get hit quite badly by SPF.
Let me explain; my domain is handled by a hosting provider here in the UK. Because I don't have a static IP address (and also because I don't want the hassle of handling a publicly visible SMTP server), I've set up a single mailbox with the hosting provider that acts as a catch-all account.
Locally, behind my firewall, I use fetchmail to retrieve the contents of this account, and I use qmail to distribute the mail into various IMAP folders; naturally I'm also using ClamAV and SpamAssassin as well.
All well and good, but the problem is that my domain hosting provider does not allow SMTP relay *at all*. Therefore, I use the SMTP relay service provided by my ADSL provider.
Obviously, neither my local qmail system nor my ADSL providers' SMTP relay will be listed in any SPF records; how will I be able to carry on locally managing my mail without automatically being rejected by SPF-aware mail servers?
Life is like a sewer; what you get out of it depends on what you put into it...
explain this to me
if you are going to HAVE to use ESMTP why not add the ability to look up public key for domain ?
if you are doing the domain why not query for user ?
finger server or in DNS record ?
is this in the spec ?
in the future then everyone can use weak crypto for emails and not send everything plain text
(speak to the person in internet cafe or bussiness and they dont understand that their msg is transmited plaintext and maybe through other peoples servers who may or may not read the email )
it would be nice to say we thought of providing keys but people dont have to use them...
regards
John Jones
my webmail rightly lets me send with whatever From: field I choose
So I can emailwise be both at work and at play from the same webmail
for spamming from a zombie (with an IP of 111.222.123.124)
From: zombie@111.222.123.124
Subject: Stop spam now!
or it wouldn't be too much trouble to look up the MX record of 111.222.123.124 and set an appropriate From: header accordingly
This scheme is as temporary as any other and it also prevents me from sending mail with my own computer, I will have to route my mail through my ISP's mail server in order to tag on to their SPF
oh well, something's got to give
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
The machine was taken over from Joe Sixpack in the first place.
He won't know why his box is just a bit slower than usual. Soo he thinks DSL is all bull because is hardly faster than his ol' dial-up.
Get a thousand zombies sending a hundred spam a minute and you see that its not so tough to send a hundred ACK signals as well...
Potential end result: 1,440,000,000 spams/day from ONE infected net.
Go after the spammer's clients instead. Spam and you get jail time and a whopping big fine, paid locally, regardless of the jurisdiction you're in.
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
That's interesting! I'd like to plug two bug reports of mine (I wish I had time to hack, but I haven't). Friend-of-a-friend makes great start for a reputation system, at least for whitelisting people you know well.
So, I there's a Spamassassin bug on this, and it has generated some interest.
Now, the problem is to generate FOAF-records easily and reliably, and for that, I suggest for example enabling KAddressbook to export them.
Employee of Inrupt, Project Release Manager and Community Manager for Solid
For a moment, neglect the high costs, complexity, worldwide legality and PKI problems of crypto, which are all solvable at some cost. They aren't the fundamental problem.
Strong authetication answers the question:
That's a very nice to know if you need to place a high level of trust in the message, such as correspondance from your bank. Today email is virually worthless (in the absence of PGP) for trusted messages.
Saddly, the answer to this question is not valuable for filtering out junk. The question that needs to be answered instead is:
PGP does not answer this question. Neither does Yahoo's DomainKeys. There are many circumstances where the signature can not be verified. You can not use the failure to verify a PGP signature as a reliable means to filter out junk.
SPF and MS Caller ID are designed to answer this second question reliably enough to discard forged messages. They answer this question for all domains that publish SPF records (aol.com, gmail.com, etc) regardless of whether SPF is widely adopted by the rest of the world.
When the claimed domain published a list of designated senders, and the sending MTA's IP number isn't among them, you can be certain the message is a forgey and discard it (or close the connection before the data phase, saving bandwidth).
PGP and other crypto signature schemes simply do not deliver this ability to detect forgey. They only detect authenticity.
PJRC: Electronic Projects, 8051 Microcontroller Tools
shemes, the whoreabully infactdead/pateNTdead PostBlock censorship devise, gnu online dating, & a host of other quick buck scams, interview, still forthcoming?
remember, keep it simple. no questions about the monIE, or whatever happened to the won-eyed girl?
the scriptdead 'answers' have already been prepared buy a cubicle full of phonIE ?pr firm? hypenosys talknicians, & will be delivered onto you, when your questions match them acceptabully.
remember, lookout bullow.
consult with/trust in yOUR creators.... self-promoting by natural attraction since/until forever. see you there?
Your envelope FROM is not the same as your From: header. Get that through your skull and then come back.
There's something I don't get: What if the spamming computer sends a mail, but pretends to be relaying it? He could act as if he actually got the mail from someplace else which makes the fake from-address valid in the view of the receiver.
How does that work?
lucas
(x) It is defenseless against brute force attacks
Explain: how would you brute force it? One way would be to search until you find a domain without SPF information and fake addresses from that. That will reduce the pool of domains you can fake, and be an incentive for them to start using it. In a way it's shifting the damage over to those that doesn't try to help solving the problem, they decide to be easy targets they take the consequenses.
(x) It will stop spam for two weeks and then we'll be stuck with it
It will stop spam from beeing sent with faked addresses from a domain, if the owners of that domain wishes it. That means I will never se a bounce for spam using my address if the recieving end uses SPF, and the reciever willl not see spam that fakes my address as its sender
(x) Users of email will not put up with it
Why not, all I have to do is configure my mail program to use the correct mailserver for outgoing mail
(x) The police will not put up with it
The police has never cared about anything to do with spam, why should they care about this?
(x) Requires immediate total cooperation from everybody at once
Bull. Mail (allegedly) from domains that doesn't publish SPF information will get through, and mail to recievers that doesn't check SPF will come through. Domain owners will be encouraged to implement this because they will se fever (misdirected) abuse reports. Users will be encouraged to implement this because they will see less spam
(x) Lack of centrally controlling authority for email
The owner of a domain is the central controlling authority for email from that domain, that's all you need
(x) Ease of searching tiny alphanumeric address space of all email addresses
Eventually all spam will use a sender address from domains without SPF informations (or nonexistent domains), people will start dumping mail from domains without SPF (or give it a high spamassassin score) and those domains will effectively be forced to implement SPF or see their users unable to send email
(x) Asshats
Which is why you have to use additional measures, such as scoring mail without SPF low, blacklisting domains and ISPS that allow spammers and other kinds of filters. There is no tool that will block ALL spam, but the right combination will reduce it drastically
(x) Unpopularity of weird new taxes
No tax involved
(x) Huge existing software investment in SMTP
It's compatible (actually uses) SMTP, no software has to be replaced. (Unless it already sucks)
(x) Willingness of users to install OS patches received by email
(x) Armies of worm riddled broadband-connected Windows boxes
And all the mail sent out to SPF using clients using from addresses with SPF domains will be dropped, eventually this number will rise as more people adopt SPF.
(x) Extreme profitability of spam
Making them work harder and reach less people will decrease the profitability, that will make the situation improve
(x) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
Eventually the spammers will have fewer domains to use for sender addresses, they will have to buy domains (increasing cost) or use domains without SPF. Both can be blacklisted
(x) Outlook
This can be implemented serverside, outlook is not an issue
(x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
None has ever been standardised and tried large scale
(x) We should be able to talk about Viagra without being censored
You can as long as you don't forge the sender domain. But if you try to sell it to someone a complaint may well make you lose access to that domain
(x) Countermeasures must work if phased in grad
- We are the slashdot. Resistance is futile. Prepare to be moderated -
...i own a domain name. i use it for email forwarding to whatever account i happen to be using, so that i can keep the same email address. this is going to break that, isn't it? i'm just a little guy who doesn't run his own smtp server (i'd have a job: my ISP blocks that anyway...)
SPF is not an anti-spam system. SPF is an anti-joe-job system. It happens that the most frequent joe jobs tend to come from spammers, however:
1. Not all joe jobs are spam.
2. Most spam (looking at my bulk mail folder on Yahoo) doesn't involve joe jobs.
A number of people are equating this with anti-spam systems, and that's just wrong. I thought one of the most revealing answers in the interview above was:
What's signficant about this answer? It's that it doesn't answer the question. More specifically, it answers the first part of the question, but (rightfully) ignores the second part, because the second part of the question is a "Why are you still beating your wife?" question (a question based upon a false assumption.)Proper anti-spam systems are based upon dealing with unwanted email, not dealing with non-relevent characteristics. If you create a system that makes it easier to control who gets your email address (such as my solution, Yahoo!'s AddressGuard(tm), and TMDA), then you're implementing a relevent solution, because dealing with spam is about controlling who sends you email. Likewise, if you operate Bayesian, CRM114, Mail.app, etc, AI filters based upon spam, then you're coming up with a relevent (if imperfect) solution, and the solution can work if combined with a whitelist, especially if you can automate the generation of the whitelist through systems like Camram.
SPF isn't such a system, it's designed to deal with a totally different issue. Arguably, given that it breaks mailing lists and forwarding and has many other documentable faults, it really ought to be being used as a measure of last resort, though it's a good idea for mail receipients to implement the logic so that domains that are having these issues can make the choice when they find themselves being targetted.
It'd be nice if every solution to every problem on the net wasn't always promoted as an anti-spam solution...
You are not alone. This is not normal. None of this is normal.
SRS/SPF still have a large number of problems to solve. SPF alone is very good idea, but the special-case of mail-forwarding is not compatible with the current design. As SPF breaks forwarding, SRS is an ugly hack which tries to repair the damage done.
I am not convinced that SRS does not introduce ugly bugs, which enables spammers to circumvent SPF alltogether.
Specifically I think SRS can as easily be forged as mails can be forged today, as noone hinders you to fake a forward which hasnt taken place in the first time. If this forward looks ok, you are in.
Now: Forging Spammer -> Recipient
SRS/SPF: Forging Spammer -> Recipient
^
|
Imaginary Origin
Why should this work at all? Please enlighten me.
Meme of the day: I browse "Disable Sigs: Checked". So should you.
People are already scaning for open SPF records.
.cf file (no it can't in any sane way), and the very 1st spam I got with my new hacked system had a vaild SPF record. If a spamer gets paid $5000 to send out a million messages, a $15 throwaway domain is nothing...
I was looking at seeing if sendmail could parse the records in the
Too bad its still broken:
# Its parsing is too complex
# No sane firewall is going to let TXT records through
# No sane firewall is going to let TCP DNS packets through
# The parsing can loop forever
# It will increase DNS scaning as spamers hunt for broken SPF records
# Its too complex to be implimented inside the MTA where it needs to be done
# It can't be properly parsed in sendmail
# ISO 8839 8859 59-15 utf-8 issues for domain names may kill some dns servers
Its a step in the right direction but its the wrong step.
Score: 5, HAHAHHAHHAHAHAHHAHA.
m l
In return I give you this: http://www.rhyolite.com/anti-spam/you-might-be.ht
People are already scaning for open SPF records.
I can't say that I'm surprised
I can imagine that spam filters like spamassassin could check if the mail came from a domain with an open SPF record, it would simply be a matter of retrieving the record, doing a little parsing and assigning a score.
If a spamer gets paid $5000 to send out a million messages, a $15 throwaway domain is nothing...
The cost is also time to get the domain in place, less time spent on defeating other anti spam measures. If that spammer is using zombies he'll have to use an open record that a spamfilter rule like the one above would detect. He must eiter accept that less of his spam will arrive or add each single sender machine to the SPF record. In addition I expect we'll soon see domain block lists in addition to IP block lists so each spammer domain will only have a lifited "shelf life".
As I said SPF is no silver bullet by itself, but I believe it will improve the situation, just avoiding joe jobs is worth it in itself and if it can make the spam flow a little slower that's good too.
- We are the slashdot. Resistance is futile. Prepare to be moderated -
I have people claiming to be from my domain sending spam, so if this will stop or slow them down I'm all for it. I published my records a while ago. It hasn't seemed to make any difference, but maybe it is making a difference and I just don't know it.
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
I've seen a lot of heavily virus infected and spyware infected PCs but I have never seen a PC that I knew was zombified as a spam sending monster.
Maybe I just didn't know it.
Has anyone seen one? What applications are running on them? It seems like maybe we could mess with the spammers if we knew what they were using.
The man who trades freedom for security does not deserve nor will he ever receive either. - Benjamin Franklin
It does NOT account for Outlook. It's a method for preventing address forgery.
Create a new bug that sends spam with the Outlook and you won't be able to differentiate between spam and real mail from the Outlook user.
I'm sorry if I haven't offended anyone
Boycott Caller-ID for E-mail is still wary of the new, combined scheme. Neither Meng nor Microsoft has clarified whether the draconian patent schemes from Microsoft's earlier attempt still exist!
We need to pressure the people coming up with the standards to avoid patented schemes at all costs, or assign the patents to a neutral third-party with the mandate of using them defensively.
That actually helps...With tighter regulations on, for example, using accurate contact info when getting a domain name, bulk-emailers will be reachable by phone and mail.
tasks(723) drafts(105) languages(484) examples(29106)
I haven't seen any clarification of these patent issues yet, personally. What about the anti-GPL license that CID has? Is that still there?
æeee!
ever heard of key signing ?
so you end up with a web of trust...
SPF and Caller ID are a solution until you end up sending emails from your outlook automatically...
oh wait thats been done before
OpenPGP and email clients that support it, go a long way to solving the problem i.e. set your boundrys
I trust bob
bob trusts jane
jane trusts bart
bart trusts lisa
so I think that I want only 3 degrees and everyone else has to tell me via phone fax and friends they want to email then I trust them and their friends...
works a bit like friendster and gmail invites only way of doing things I can see that will work
regards
John Jones
It does NOT account for Outlook.
A zombie machine (infected through outlook or any other means) would have to send mail with addresses it is allowed to send from. That would be a domain controlled by the spammer (can be blacklisted), a domain with an open / nonexistant SPF record(can be filtered on), or a domain it's already allowed to send for (the ISP, which means chances that the machine would be stopped are far greater, or the ISP risks getting blacklisted).
- We are the slashdot. Resistance is futile. Prepare to be moderated -
With domains being $15, its trivial to get a domain with fake data. Besides the bulk mailer is just going to use an address of one of the idiots paying them to send out the ads. Remeber this is to solve the joe-jobing issue. If they are willing to do it on-line, why not in the real world.
As for why you'd need a license for this, it may the case that MS has a number of pending patents on the concept (orginally termed Caller ID) and the license mentioned prior is meant to assure people that if this makes it out there as a standard, they will have a license to practice with having to pay royalties.
... they certainly should not be given government monopoly entitlements to the concept, nor if our corrupt government does grant them such entitlements, should anyone respect it. Locate the infrastructure off shore instead.
Rather than give tacit support of software licenses to Microsoft (or anyone else), I'd rather just locate my DNS and mail server overseas, in a sane regime that doesn't recognize software patents, and use SPF irrespective of Microsofts intellectual property grab.
Microsoft obviously didn't invent anything here (the SPF folks did)
The Future of Human Evolution: Autonomy
SPF bascially assumes that if you have chosen the solutions that the grandparent has chosen that you are wrong. You now have to do something else. What that something else is is of no concern to SPF. Deal with it.
As a spammer, this is just a cost of doing business.
I'll apply to become a registrar. I'll stock up on cheap $3 domains, and use one for each few million emails.
My bulletproof DNS host will include SPF records that are completely valid - my peer2peer zombies will let me update the SPF records dynamically.
Shame about not leaving any 2-word domain names for you kind folks. You can have 3-dictionary-word domain names until I get there.
Proper anti-spam systems are based upon dealing with unwanted email, not dealing with non-relevent characteristics
It seems that the only proper anti-spam solution is to cut of the testicles of anyone that advertises their products with a spammer. Stop the silly technological arms race, and just assault the spammers' supply chain.
If the zombie is actually using the user's proper (sender) domain and sending through the ISPs mail server, then SPF doesn't help directly, it gives the ISP the power to simply monitor what is sent and shut the zombie down if spam is being sent.
So, SPF contributes nothing. ISPs already have the ability to monitor their clients for spam and shut them down. It also shows that Vixie is not correct about the only benefit he sees for SPF, prevention of domain spoofing. The owner of a zombie machine will send out spam with the targeted domain. Spammers will simply slow the number of messages from each zombie to whatever does not trigger the ISP's monitoring program. The spam will go on and it will come from every big dumb ISP until people quit using crap from M$.
Friends don't help friends install M$ junk.
All your questions are answered in the papers on SRS which were written by a professional cryptographer and security researcher at the University of Bath.
A lot of very smart people just like you have spent a lot of time thinking through the problems. Much of their thinking is captured in essays and FAQs available online.
http://www.libsrs2.org/docs/
If you run your own mailserver, you can setup port 587/tcp, which is the smtp-auth port. Configure your MTA to only accept authorized connections. SASL is very common for this. When your MTA connects it issues a STARTTLS, which encrypts the session, then it authenticates as you, and then your MTA can allow it to relay. The 'traveling executive' problem is now solved.
RFC 2554 for more information.
I'm getting the impression that a lot of the people who post objections haven't bothered to read the FAQ.
I suppose they also write lengthy criticisms of movies based on the teaser trailer and on reading reviews, without actually seeing the movie.
How is SPF going to deal with the Malicious forwarder?
Resent-From: pretendsitslegitforwarder@example.com
From: security@ebay.com
Subject: $I_wanna_steal_your_identity
$Body
Assume that example.com publishes SPF records and that this mail passes that check. The spammer has just passed its authentication check and successfully forged an email.
Lets make this a little more practical. Spammer buys CD of million names.
>grep '@pobox.com'
$id@pobox.com
He then sends mail to $id@pobox.com through his domain (even with valid SPF records). To hide his tracks, he continues forging -- saying that he received the mail from yet another forwarding service (ie, puts in a fake resent-from).
Mail From:
RCPT TO:
Resent-From: $id@pobox.com
Resent-From: spammer
Resent-From: another_forged_legit_domain
From: security@ebay.com
Subject: $I_wanna_steal_your_identity
$Body
Pobox.com receives the mail, validates that the mail is from spammer and forwards it along to the end user. The end user's system verifies pobox's SPF -- bingo, we have gotten a forged email through. SPF delegates the identity check to domains that you do not control. In today's world that is not a good idea.
Today, 60-80% of the mail coming from pobox.com is spam. If the receiver applies pobox's reputation to its mail, then it should reject all its mail. I assume that would not make Meng happy.
More and more people are realizing that authentication is not an anti-spam solution, but that authentication allows reputation and other antispam components to be built on top of it. Unfortunately this is exactly why SPF will fail.
I know that I personally had an exchange with Scott Hazen Mueller (of CAUCE) via email back in 1999 or 2000, proposing this very thing.
Where we use MX records for sending, I proposed an MS record for authorized Mail Senders. That's all it did, but at least you could look to DNS to see if this remote system was actually a designated speaker for that domain.
Of course, he told me it would never work and that it was unreasonable to use DNS to provide such a service.
Oh well...
The general approach taken by SPF and friends does
,
nothing directly to stop SPAM. As numerous other posters
have pointed out, the throw-away domain is still available
to spammers and it is easy for them to generate valid
SPF records for those throw-away domains, even using
zombie networks (with dynamic DNS, you can even limit
the number of IP addresses in the records to mask
the size of the network).
It is useful, though, both immediately and in the medium
term. The immediate usefulness is in bounce processing.
An MTA whose role is to generate or forward bounces
can check the SPF record before action and not send
those for which there is no connection between the
message and the target of the bounce. That's very
useful, since it removes the clutter from the mailboxes
of those who were the targets of joe jobs. (Note that
an MTA forwarding a bounce can do this, as well as
one originating a bounce--that means an enterprise
can do this at the MTAs listed in the MX records).
In the medium term, the SPF record can be used to
limit certain kinds of forgery, making it harder for
social engineering attacks to work. It's not perfect,
by any means, since registration of domains like
"companyname-support.com" will still fool some
users. It also relies to some extent on the first
MTA in a series to perform the checks, as the scheme
gets weaker and weaker with the need to grovel
through more headers to make an association.
In other words, it works well when it is
ORIGINMTA--DESTMTA, but less well when it
is ORGINMTA--IntermediateMTA-IntermediateMTA-DESTMTA
since the RFC 2822 header (From:, Resent-From:, etc.)
processing gets more complicated. Since the latter is a core
part of the design of email (store and forward), there
are limits to SPFs role in this arena--especially when
spammers decide to pass there messages through multiple
MTAs under their own control to generate the appearance
of this complexity.
In the long term, the hope is that it starts folks toward
a more generic mechanism that allows the sending domain
to describe the characteristics of its email (e.g. all mail
is signed as S/MIME and all signatures derive from this CA)
and the receiving domain to check the characteristics
of the messages it's received against those.
Again, though, this is all about forgery, not SPAM.
How is this supposed to stop spam, then? The spammers will simply make sure they use a domain that doesn't have SPF records.
Simply put, it won't stop spam. (There, I've said it.)
It's not supposed to. The goal of SPF (and the other reverse-MX proposals) is to allow a domain owner to put a dent in joe-jobbing by whitelisting all of the mail servers that are authorized to send e-mail on behalf of the domain. As a mail admin, it lets me say: "Don't bother accepting any e-mail purporting to be from my domain unless it comes from machines X, Y, and Z".
It's the natural mirror of the MX record (which specify what machines are authorized to accept incoming mail for my domain).
Wolde you bothe eate your cake, and have your cake?
SPF is not an anti-spam system. SPF is an anti-joe-job system.
Gads yes. That's probably one of the biggest issues with the entire SPF idea. People either mistakenly think that it's an anti-spam solution, or else they're mislead to think so by others.
(In fact, my biggest complaint about SPF and some of the extensions is that they stray too far from the anti-joe-job duty and are trying to do anti-spam stuff.)
Wolde you bothe eate your cake, and have your cake?
This article advocates a
( ) technical ( ) legislative ( ) market-based (x) vigilante
approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
(x) No one will be able to find the guy or collect the money
(x) It is defenseless against brute force attacks
(x) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
(x) The police will not put up with it
(x) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
(x) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
(x) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
(x) Open relays in foreign countries
(x) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
(x) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
(x) Willingness of users to install OS patches received by email
(x) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
(x) Joe jobs and/or identity theft
(x) Technically illiterate politicians
(x) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
(x) Outlook
and the following philosophical objections may also apply:
(x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
(x) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
(x) Countermeasures should not involve sabotage of public networks
(x) Countermeasures must work if phased in gradually
( ) Sending email should be free
(x) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
(x) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(x) Killing them that way is not slow and painful enough
Furthermore, this is what I think about you:
(x) Sorry dude, but I don't think it would work.
(x) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, l0ser! I'm going to find out where you live and burn your house down!
That won't help the Joe much, when the MTA's block a message due to the SPF, and then the email bounces back to the Joe just the same. Unless the MTA's are changed to NOT send non-delivery-bounces, it will increase the amount of bounces the poor guy receives.
AFAICT, the only thing it can decrease is the amount of delivered forced email, at the expense of increasing bounces and making forwarding and vanity domains behind an uncooperative (read: corporate) ISP impossible.
Unless, of course, everybody starts using web mail, through companies like Microsoft and Pobox.com, of course... Hmmm...
--
*Art
Hey, you can't be new here.
You figure that out WAAAY to fast!
-oo-OO-oo-
The "web of trust" idea is not quite right. Unless you're going to require all incoming email to be signed, it doesn't help you to reject forged messages. Such a bold move would result in most people losing all their incoming email (since it wouldn't be signed).
What you'd need would be a way to look up the sender and see if they always sign their messages. If they do and this message is unsigned, it would be known to be a forgery. This method is not as effective as SPF because each user would have to change their email habits individually and publish their intent to always sign their messages individually (somehow). With SPF, each domain can effect the change for all their users, and (as long as the user's email software is setup right), the users will not have to learn a new habit for sending mail -- things just improve as more and more sites publish SPF records (and check it on delivery).
You'll note that all recent viruses and spam-bots spoof the From header, which means that they would NOT be valid email with SPF enabled for your domain. If virus writers and spammers have to go back to sending their emails with the real domain name of the host they're running on that would be very useful in helping to identify who is responsible for the virus or spam.
Remember, SPF doesn't claim to be a 100% solution to all email problems, nor does it need to be. It slams shut one huge open door in the current protocol, and it does so very effectively. I think it is a very good, targetted solution that will be of great benefit to the net.
..wayne..
So we have the Addressing Spam Channel of the leading Intelligence Hub for The Internet's Core Infrastructure & Policies interviewing the lead developer of the lead antispam solution, and half the comments under the interview are from a spamming all-caps shill who repeats his identical message over and over.
This is...
O
U
T
R
A
G
E
O
U
S
!
Forcing people to use a providers mail server stinks big time. Why should I do that? It compromises my privacy and slows things down. The only people suggesting that as a solution are people that do not run their own mail server at the end of their broad band connection. It's a classic "I'm not using it therefore it's ok to ban it for others".
The very principle of clustering mail from a provider is a little bit stupid as the number of potential mail clients grows. In an idea world, all mail would go directly peer to peer.
- Kim
Mercatur hates you.
And me, for that matter.
Explain: how would you brute force it? One way would be to search until you find a domain without SPF information and fake addresses from that. That will reduce the pool of domains you can fake, and be an incentive for them to start using it. In a way it's shifting the damage over to those that doesn't try to help solving the problem, they decide to be easy targets they take the consequenses.
It sounds more than plasubile to me. So, I'm curious. You propose to try, by social pressure, to force *every single* domain to disallow non-SPF email?
Heck, if *that* kind of approach worked (force everyone to use system Foo and use it correctly), that past few years where people *swore* to use that the answer to stopping spam was just eliminating all the open relays ("sure, it'll be easy") should have solved the spam problem. It doesn't work. You must have a system that does not require Internet-wide perfect cooperation.
It will stop spam from beeing sent with faked addresses from a domain, if the owners of that domain wishes it. That means I will never se a bounce for spam using my address if the recieving end uses SPF, and the reciever willl not see spam that fakes my address as its sender
You don't need to do so now. There are existing, workable systems that don't require an iota of effort on the part of the domain mail admin that work fine. One would be to slap a "X-Fromme:" header in all your emails with a secret in it, or hash of a tuple of the to address and a secret, or whatever makes you happy, and have procmail or whatever system floats your boat. It can all be done without leaving the client or screwing with existing infrastructure.
Why not, all I have to do is configure my mail program to use the correct mailserver for outgoing mailBull. Mail (allegedly) from domains that doesn't publish SPF information will get through, and mail to recievers that doesn't check SPF will come through. Domain owners will be encouraged to implement this because they will se fever (misdirected) abuse reports. Users will be encouraged to implement this because they will see less spam
Users will *not* see less spam when they do this unless everyone implements. Why would spamemrs spam with a forged from address from an SPF-enabled domain when there are an infinite pool of non-SPF-enabled domains (or even non-forged addresses).
You must admit one of two things. Either (a) people don't have to deploy SPF records, and SPF is not an antispam system, or (b) SPF requires everyone to completely and fully deploy and it is not workable (no system requiring all nodes to be trusted and not screw up that I know of has ever worked). You can't have both the benefits of requiring SPF records and not have the drawbacks.
Eventually all spam will use a sender address from domains without SPF informations (or nonexistent domains), people will start dumping mail from domains without SPF (or give it a high spamassassin score) and those domains will effectively be forced to implement SPF or see their users unable to send email
As I said above.
Which is why you have to use additional measures, such as scoring mail without SPF low, blacklisting domains and ISPS that allow spammers and other kinds of filters. There is no tool that will block ALL spam, but the right combination will reduce it drastically
The schemes that the SPF people have proposed are vulnerable to DoS attacks. "Asshats" include those people that want to DoS people. Suppose you spoof a little DNS record or two, setting a high TTL on cacheable lifetime, and manage to make AOL's main mailserver think that MSN's SPF record says that the only legitimate mailserver is somewhere in Korea. It'd be funny when millions of emails start getting bounced as spam, eh? It'd be even *funnier* if asshats use the domain-level-reputation-based-tools that the SPF people are advocating to patch over the fact that SPF can't deal with throwaway domains improperly -- say, compromi
May we never see th
SPF is not an anti-spam system. SPF is an anti-joe-job system. It happens that the most frequent joe jobs tend to come from spammers, howev
As I've pointed out below in a number of comments, SPF is not a good anti-joe-job system either. A lot of people have identified the fact that SPF does not actually deal with spam very well, have wondered *why* the designers came up with it, and apparently arrived at the conclusion that it's a brilliant way to avoid joe jobs. It's not.
Proper anti-spam systems are based upon dealing with unwanted email, not dealing with non-relevent characteristics. If you create a system that makes it easier to control who gets your email address (such as my solution, Yahoo!'s AddressGuard(tm), and TMDA)
Your solution would be really nice if more ISPs had good mail servers.
I enjoy the use of Carnegie Mellon's Cyrus mail server, which is set up to allow any address that looks like username[+optionaltext]@andrew.cmu.edu to be delivered to username@andrew.cmu.edu. A number of CMU folks use your approach, and you'll see stuff like tom6+compdiscuss@andrew.cmu.edu on a "compdiscuss" forum. It lets people track where people get their email addresses from, in addition to giving them an effectively infinite number of email boxes -- you can tell students to send current problem solutions to jsmith+15676_hw5@andrew.cmu.edu, for instance.
SPF isn't such a system, it's designed to deal with a totally different issue. Arguably, given that it breaks mailing lists and forwarding and has many other documentable faults, it really ought to be being used as a measure of last resort, though it's a good idea for mail receipients to implement the logic so that domains that are having these issues can make the choice when they find themselves being targetted.
I don't think so -- I think that SPF actually is a misguided effort intended to eliminate spam.
SPF is, when all is said and done, an authentication system, and nothing more. The problem is that it fails a number of basic requirements that most people have on authentication systems -- it can be decieved. It does not provide sufficient granularity of identity (no user-level auth) to implement a number of systems that might be built on top of it that *might* be effective antispam systems. If decieved, a great deal of damage can be done. It natively has an extremely generous "trust" system (once something has been accepted by any SPF-enabled system, it's kosher) or an attackable and DoSable trust system (if domain reputations are used, as vaguely proposed by its authors). It does not allow delegation of authority.
SPF is not a good anti-spam system -- it doesn't stop spam unless everyone uses it perfectly Internet-wide (which doesn't work).
It is not a good anti-joe-job system -- it fails to secure a number of links in the mental association process between an email and a trusted authority (I've detailed this in several comments further down).
It is not even a good authentication system to build poential anti-spam, anti-joe-job, or other distributed Internet systems on -- it can be attacked, DoSed, and provides a very limited set of functionality.
May we never see th
So what's the story on the patent claims? Boycott Email caller id still seems wary, and I see no evidence that Eben Moglen's concerns (such as incompatibility with the GPL) have been addressed.
Any such patent application is likely to be granted, since Microsoft has lots of $$$ to press their case and the patent office has neither the knowledge or time to determine if they're obvious or in any other way counter them.
I remember the joys of dealing with GIF patents. We're better off without this combined approach if the patent applications will make it unworkable.
So, what's the situation?
- David A. Wheeler (see my Secure Programming HOWTO)
You are not alone. This is not normal. None of this is normal.
MUDs use many ports, some say to connect with port 23, others say port 25, while more say 1997, 2040, etc. Sometimes people can log into some muds while they can't others. This is because they're using different ports. I think it is a fair assumption to say I am using port 25, 1997, 2040, whatever the mud is telling me to.
And why would I want something more secure then telnet just to play a game? Sheeesh.
It sounds more than plasubile to me. So, I'm curious. You propose to try, by social pressure, to force *every single* domain to disallow non-SPF email?
Well not really social pressure. When enough people remove themselves as targets for having the from domain faked, the chances that the remaining domains will be targeted is greater. I wouldn't want to remain standing as all others drop to the floor when the bank robbers enter :)
It doesn't work. You must have a system that does not require Internet-wide perfect cooperation.
Again you're looking for that silver bullet. SPF can't be used alone (unless we lived in a perfect world, bet then there would be no spammers) SPF gives us one more thing to filter and score based on, but your filters have to be set up with the assumption that there are domains without SPF or with misconfigured SPF:
Mail from doman with SPF, sender is allowed by SPF record => low probability of spam
Mail from domain without SPF => medium probability of spam
Mail from domain with open SPF record => medium probability of spam
Mail from domain with SPF, sender disallowed by SPF record => high probability of spam
Obviously you have to filter/score on other things too
There are existing, workable systems that don't require an iota of effort on the part of the domain mail admin that work fine. One would be to slap a "X-Fromme:" header in all your emails with a secret in it, or hash of a tuple of the to address and a secret, or whatever makes you happy, and have procmail or whatever system floats your boat. It can all be done without leaving the client or screwing with existing infrastructure
I can't see how that is going to prevent anyone from faking the addresses, it would make it easier to filter out bounces to mail I didn't send if that was what you were thinking about. But how would it help with the bandwith cost of the bounces from a joe-job?
Users will *not* see less spam when they do this unless everyone implements.
You are correct, but only if you assume that the users won't be using additional ways of filtering, and feeds the result of the SPF check as input to that filtering
You must admit one of two things. Either (a) people don't have to deploy SPF records, and SPF is not an antispam system, or (b) SPF requires everyone to completely and fully deploy and it is not workable (no system requiring all nodes to be trusted and not screw up that I know of has ever worked).
(a2) people don't have to deploy SPF records, and SPF is not an antispam system by itself
Suppose you spoof a little DNS record or two, setting a high TTL on cacheable lifetime, and manage to make AOL's main mailserver think that MSN's SPF record says that the only legitimate mailserver is somewhere in Korea.
People that can't trust their DNS servers not to cache bogus responses to DNS queries shouldn't depend on DNS for anything, they certainly shouldn't be dumping incoming mail based on it.
You think it's "fun" getting off the blacklist for RBL because some dipshit on your network screwed up now, wait until everyone' using *domain-level* reputation, and any person at your enterprise-class company can screw your domain's reputation over severely, and kill your domain's mail privileges.
Not fun no, I've heard the stories
Obviously a company should have a policy that does everything to prevent this. But that can happen with or without SPF with todays IP blocklists
Especially nasty if that one person is completely innocent, but happens to be using a Windows laptop that picks up a worm...but *that* wouldn't ever happen, would it?
Not so innocent if that person broke company security policy by disabling virus checks, or installed downloaded software from shady sites. It won't happen again after you fire the first one.
All the people out there have to put in new,
- We are the slashdot. Resistance is futile. Prepare to be moderated -
Why not set up trust networks a la gpg?
Distribute the standard mail apps with gpg compiled in by default and let each user either import or create a key upon account information setup.
Then, when reading mails, there should be two buttons: trust or untrust sender if the message has been signed. When clicking "trust", a message dialog pops up and informs the user how he can do that, eg. not to trust keys just because he had an email conversion with the sender for some time, but rather make a phone call or exchange keys via secure ways.
I wonder if there is any downside -- except that the OSS community should persuade the user somehow to create and trust keys.
Again you're looking for that silver bullet. SPF can't be used alone (unless we lived in a perfect world, bet then there would be no spammers) SPF gives us one more thing to filter and score based on, but your filters have to be set up with the assumption that there are domains without SPF or with misconfigured SPF:
SPF is really nothing more than an authentication system. I've argued elsewhere in this story that SPF is not a very good authentication system. I'm not sure that basing other antispam systems on SPF *is* a good idea.
People that can't trust their DNS servers not to cache bogus responses to DNS queries shouldn't depend on DNS for anything, they certainly shouldn't be dumping incoming mail based on it.
I'm curious -- what folks are you thinking of that *can* trust their DNS server caches not to be poisoned?
And while I agree that it might be nice not to route/blow away mail based on the relatively insecure DNS, it's also done on a regular basis and is a fundamental element in the way email functions. For a close parallel to SPF's DNS usage, consider mail servers with a reverse-MX-required policy.
Obviously a company should have a policy that does everything to prevent this. But that can happen with or without SPF with todays IP blocklists
Right. But with domain reputation, there may be many sources of input and since reputation is probably not binary, it's hard to know exactly "how in the clear" you are. With an RBL, if you fix your open relay, you're in the clear until next time your mail admin screws up. You can be careful to avoid the problem, and easily demonstrate that you are *not* a spam domain by fixing the open relay and sending email to an RBL service asking them to retest your mail server. With domain reputation, you may lose your entire mail service when a few accounts start spamming, and if reports come in over a period of time, even if your domain is cleared and rerated as "not a spam source", you may have difficulty ensuring that you stay above the threshhold as people read their email and send in futher spam reports.
You allow incoming mail from nonexistent domains?
No. But it's easy to register citibank-customerservice.com and then start sending email that *looks* like it comes from Citibank.
Noone should rely on email for anything of importance today.
That's probably true. However, people are desperate for a form of reliable Internet-based communication, and like it or not, email *is* used for crucial communications every day. Social Security numbers and mother's maiden names are both really awful forms of authentication (and SSNs are *specifically* not supposed to be used for authentication), but both are commonly used for exactly that. An ISP can't just get away with saying "well, you shouldn't have been using email for anything important" to their business clients if they go through a software upgrade that suddenly starts dumping 5% of (legitimate) incoming email.
That would depend on the policy of your ISP. It should be possible for a mail server to decide if/how to perform SPF checks depending on the user the mail is for.
Yes, but it "should be possible" today, but ISPs don't generally provide such an option, nor is there any set of extensions to IMAP/POP3 to let you "set" filtering options.
May we never see th