IETF Approves SPF and Sender-ID

NW writes "According to the records in the IETF's database (here and here), both the SPF and Sender-ID anti-spam proposals were tentatively approved by the IESG (the approval board of the IETF) as experimental standards. It remains to be seen whether any of them will actually put a dent into spam." At the same time, the FTC has opened a central site about email authentication.

Comments

I'm not sure about a dent in spam

[Mustang Matt]

But this will help me out tremendously.

Not getting joe jobbed will be a huge step forward. Not to say that everyone's going to instantly adopt these standards but it won't hurt that these are Official now.

Not about spam, it's about joe-jobs.

[khasim]

Before the rush of posts about how this won't do anything about spam, this is not about spam. This is about stopping spammers from using your address which results in your email servers dealing with the mass of bounces and spam reports from clueless admins.

Of course, only the admins with a clue will correctly implement either of these so ...

Re:Not about spam, it's about joe-jobs.

[dsginter]

Joe Job

For those wondering.

Did IETF change their mind?

[ravenspear]

I thought the IETF had already rejected Sender-ID because it was MS proprietary.

Re:Did IETF change their mind?

[pyrrhonist]

I thought the IETF had already rejected Sender-ID because it was MS proprietary.

Yes, they did, and they did not change their mind. They labelled these documents as "experimental". See here for details.

Re:Did IETF change their mind?

[pyrrhonist]

Which still doesn't really explain why they're conducting the experiment.

They aren't, "conducting an experiment". The draft was submitted as "experimental".

The "Experimental" designation typically denotes a specification that is part of some research or development effort. Such a specification is published for the general information of the Internet technical community and as an archival record of the work, subject only to editorial considerations and to verification that there has been adequate coordination with the standards process (see below). An Experimental specification may be the output of an organized Internet research effort (e.g., a Research Group of the IRTF), an IETF Working Group, or it may be an individual contribution.

To ensure that the non-standards track Experimental and Informational designations are not misused to circumvent the Internet Standards Process, the IESG and the RFC Editor have agreed that the RFC Editor will refer to the IESG any document submitted for Experimental or Informational publication which, in the opinion of the RFC Editor, may be related to work being done, or expected to be done, within the IETF community. The IESG shall review such a referred document within a reasonable period of time, and recommend either that it be published as originally submitted or referred to the IETF as a contribution to the Internet Standards Process.

If (a) the IESG recommends that the document be brought within the IETF and progressed within the IETF context, but the author declines to do so, or (b) the IESG considers that the document proposes something that conflicts with, or is actually inimical to, an established IETF effort, the document may still be published as an Experimental or Informational RFC. In these cases, however, the IESG may insert appropriate "disclaimer" text into the RFC either in or immediately following the "Status of this Memo" section in order to make the circumstances of its publication clear to readers.

Unless MS have pressured them into it, so they can get a crack at declaring Sender-ID as the de facto standard and thereby sidestepping the standards process, of course. That might explain it.

They aren't "sidestepping" the standards process. An "experimental" designation is part of The Internet Standards Process, but is not on the standards track.

Zombies anyone?

[dancpsu]

What's to stop the spammers from using a zombie to fake the sender id? It's a good step forward (you would know where the zombie was), but the bigger challenge would be to have some kind of capability to restrict mail servers to authenticated ones rather than some kind of recipient call-back mechanism. Otherwise the future of email will be "sender ID from mail server ZOMBIE294346.earthlink.net"

Re:Zombies anyone?

[Anonymous Coward]

Because the SPF record is a DNS record. It's not exactly trivial to fake this. Plus, in your specific case, it won't work.

Earthlink owns earthlink.net. Therefore, they get to set the policy for who gets to send mail originating from the earthlink.net domain. I can't set up an SPF record claiming to control who can/can't send e-mail from earthlink.com, because I don't own that domain.

Now, the BIGGER problem is not "faking" an SPF record. It's users who set up their own domain and publish a valid SPF record that includes spammers or zombies. So, I can set up spamdomain.com, and have my SPF record include ZOMBIE1234.earthlink.com as an allowed sender. This means mail COULD come from this zombie that claims to be from spamdomain.com. It's even possible for spamdomain.com to set up an SPF record that says EVERYONE is an allowed sender, and so anyone could send e-mail from spamdomain.com.

So, this won't actually prevent people from spamming. What it WILL do is keep spammers from imitating existing domains in their "from" headers. Which doesn't sound like a lot, but will help with impersonation. It will also make it fairly easy to tell the spam domains. Anyone with an SPF record of "every sender is OK" probably should be blocked as a probable spammer. And anyone claiming to be from a reputable domain actually is. It will also make it harder for viruses that go through your address book for "to" and "from" headers to work.

Anti Spam Measures

[blackholepcs]

It's all well and good that something is being attempted to alleviate spam in this manner, but I think a much bigger problem that needs to be addressed is ISP's selling your email address before you even log in the first time to check your mail. *Cough*Cox*Cough*. I've had 3 seperate email addresses (from 3 seperate accounts) with Cox and each one filled up with junk mail without me giving anyone the email address.

The best thing I have come across so far is Incredimail, but even that is a pain in the ass to right click each spam and choose block sender or bounce to sender. What Incredimail needs to do is come up with an automatically updated block list for known spam. It would help greatly.

In the end, I think that spammers should be beaten, shot, stabbed, hanged, drawn and quartered, eviscerated, castrated, tortured, set ablaze, and be kicked in the shin.

Re:Sender ID = Caller ID = Worthless

[Iphtashu Fitz]

I dunno, but hasn't this failed with caller ID?

Not at all. If I get a phone call and the CallerID says "unavailable", "out of area" or "private" then I don't even bother to answer it.

If my incoming e-mail doesn't have SPF headers then it bumps up the scores in SpamAssassin. If the score gets too high then the e-mail gets filtered & I never see it.

Microsoft related?

[john_is_war]

Hmm... Microsoft announces hotmail will be restricted to user-ID and now it's been passed as an experimental phase. Coincedince? I think not.

Read what ASF had to say...

[tolkienfan]

Sender ID is not particularly trusted by everyone, to say the least.

Example from ASF

Re:It won't work for long

[Iphtashu Fitz]

within a couple weeks at most the spammers will find another way arround it as they have everything else.

Do you even understand how SPF works? It sure sounds like you don't. It's meant to prevent spammers from forging domains, not to put an end to all spam. If you own your own domain then something like SPF can be a HUGE help if your domain name is used to forge spam.

It's one SMALL step

[realmolo]

Both SPF and Sender-ID solve only one problem: faked sender domains.

That's a problem that needs to be solved, but it doesn't account for a lot of spam, and spammers will just stop faking domains in their mass emails.

What we need, and what will NEVER happen, is a central database of mailservers. If you aren't in the "registry" of legit mailservers, then other mailservers won't accept your mail. To get in the registry, you'd have to pay a fee, and prove that your server are secure, and that you aren't a spammer. Obviously, each "legit" server would have to append some kind of digital signature to outgoing emails, so that the verification coudl take place.

In other words, a total revamp of the mail system protocols. ;) We can dream.

Re:It's one SMALL step

[droptone]

What we need, and what will NEVER happen, is a central database of mailservers. If you aren't in the "registry" of legit mailservers, then other mailservers won't accept your mail. To get in the registry, you'd have to pay a fee, and prove that your server are secure, and that you aren't a spammer. Obviously, each "legit" server would have to append some kind of digital signature to outgoing emails, so that the verification coudl take place.

Who would run this registry? Why do you need to pay to get on it (spammers usually generate income and do not mind spending money to further their business while individuals who want to run private/personal webservers would not want to pay money just to be able to send email)? How would you prove that they aren't a spammer when the only email "legit" email servers would receive would be through this registry (thus no trial-runs for the prospective applicant)?

Re:It's one SMALL step

[tolkienfan]

How do you figure you can prove you're not a spammer?
Better is to allow your approval to be revoked, if you are initiating spam.
Plus, the cost of such a system would be prohibitive to say the least.
Who would be resposible? A US company? And I assume all the other countries are just dying to subscribe to that system!

I think a protocol change is in order. Instead of sending the message via SMTP, only send a notification that an email is waiting to be picked up, and it's location. When your email is checked, it loads the email from the server.
This has the following benefits:

1. The cost is on the sender. It would be prohibitively expensive to send millions of messages - you'd have to host all the clients!
2. Zombies would be useless - systems are likely to be behind firewalls now (hardware and software), so incoming email clients would be rejected.
3. It could be rolled out based on current protocols. SMTP could be used for notifications. 1 notification per email, max of 1 notification per 20mins - reset on check? Check email via POP3. Not IMAP since you want to actually retrieve the message.

There are some kinks, but I think it's workable.

Re:It's one SMALL step

[julesh]

I think a protocol change is in order. Instead of sending the message via SMTP, only send a notification that an email is waiting to be picked up, and it's location. When your email is checked, it loads the email from the server.
This has the following benefits:

1. The cost is on the sender. It would be prohibitively expensive to send millions of messages - you'd have to host all the clients!

Huh? You'd still only transfer the same amount of data, it would just become unpredictable how much you would transfer, when.

Plus, you'd be able to tell which of those e-mails were actually being retrieved, therefore you'd get a better form of address verification out of it than is currently available.

Also, it would create problems for people who check their e-mail infrequently (because the originating message might have disappeared).

No thanks, I think the current method is better.

Re:It's one SMALL step

[Wesley Felter]

In bonded sender systems, you would put up a large (>$1,000) bond (like a deposit). If any spam comes from your server, you'd lose the bond and your certificate would be revoked, so you couldn't send any more mail.

Exploiting the weaknesses of such a scheme is left as an excercise for the reader.

Re:It's one SMALL step

[bunnyman]

This article advocates a

( ) technical (x) legislative ( ) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

(x) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
(x) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
(x) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
(x) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
( ) Joe jobs and/or identity theft
(x) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
(x) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
(x) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
(x) Countermeasures must work if phased in gradually
( ) Sending email should be free
(x) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
(x) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

( ) Sorry dude, but I don't think it would work.
(x) This is a stupid idea, and you're a stupid person for suggesting it.

There is no "Experimental Standard"

[pyrrhonist]

According to the records in the IETF's database (here and here), both the SPF and Sender-ID anti-spam proposals were tentatively approved by the IESG (the approval board of the IETF) as experimental standards.

There is no such thing as an "experimental standard". The term "experimental" is a "non-standards track maturity level".

See "The Internet Standards Process":

Not every specification is on the standards track. A specification may not be intended to be an Internet Standard, or it may be intended for eventual standardization but not yet ready to enter the standards track. A specification may have been superseded by a more recent Internet Standard, or have otherwise fallen into disuse or disfavor.

Specifications that are not on the standards track are labeled with one of three "off-track" maturity levels: "Experimental", "Informational", or "Historic". The documents bearing these labels are not Internet Standards in any sense.

The IETF has NOT approved either SPF or Sender-ID as an Internet Standard.

Re:There is no "Experimental Standard"

[pyrrhonist]

If I read the information at the IETF, it *has* approved SPF, however Sender-ID is "Experimental".

The intended status of both documents is "Experimental".

Both documents are in the "Approved-announcement to be sent" state. This means that:

The IESG has approved the document for publication, but the Secretariat has not yet sent out on official approval message.

After the approved announcements are sent, both documents will go to the RFC Editor Queue for publication.

What this means basically means that both documents will be published as a Request for Comments (RFC) as a non-standards track document with the status "Experimental".

Reduce spam?

[taustin]

Since SPF doesn't even claim to be a method of reducing spam, why would anything think it would?

As it happens, a couple of my bosses have been having email rejected recently by the receiver's ISP because we are SPF compliant.

SPF breaks email forwarding, and most mailing lists. It's a bad idea, poorly conceieved, and poorly implemented.

No matter what we do, SPF will cause some of our email to be rejected.

That is a way to help spammers, not hinder them.

A central database is open to abuse.

[CyricZ]

Of course we will never see a central database of mailservers. That has been proposed before, but will always be unsuitable for the Internet. Remember, the Internet is meant to be decentralized. And a centralized database is open to abuse by governments, corporations, and whoever runs it (or provides the funding for it).

There's nothing to stop spammers from infiltrating such a system, via legitimate and illegitmate means. So it just plain won't work.

Between the fact that it is easy to abuse, it just won't work and it won't provide any benefits over existing systems, your system is just a bad idea (no personal offense meant, of course).

Re:A central database is open to abuse.

[gowen]

Remember, the Internet is meant to be decentralized

Several root level DNS servers slide towards the corner of the room and try to look inconspicuous.

Re:A central database is open to abuse.

[ErikTheRed]

Verisign is hardly an integral part of the Internet. Indeed, even then it is difficult to say that they're certification is reliable.

I can reliably say that Verisign is certifiable.

Re:I'd love to see an Apache Project mailserver.

[shane2uunet]

I'd personally love to see the Apache Project coordinate and release a mail server.

Obviously this guy has not heard of Postfix, a truely awesome mailserver

SPF in the real world

[SamMichaels]

I honor spf entries on my mail server. It stops about 1000 emails/day. So far no legit mail being bounced.

Re:SPF in the real world

[ryanvm]

I stopped answering my telephone yesterday. So far nobody has called and complained.

Re:SPF in the real world

[droleary]

I stopped answering my telephone yesterday. So far nobody has called and complained.

Mods are on crack. You're closer to funny than insightful, and it's "funny" as in "so far misguided that it's amazing you're not a Darwin Award recipient." The OP's point is that for domains that already offer an SPF record, checking that record allows them to reject a fair number of messages. Your analogy is like rejecting by default, whereas they are accepting by default and rejecting if verification information is available and fails.

Re:I'd love to see an Apache Project mailserver.

[finse]

Postfix, a truely awesome mailserver

Postfix is fast, flexable and easy to use. In my mind, there is no better mail server for Unix and Unix like platforms.

Re:It won't work for long

[pe1chl]

you obiously have no practical experience...
putting up SPF records has not made any noticable difference in the spam abuse from one of my domains.
obviously, spammers do not (yet) check of a domain they use for joejobs has an SPF listing. this means that little or no receivers are bouncing the spam because of SPF.

Re:What's wrong with this?

[JohnGrahamCumming]

I'm not going to say you're a moron, but how do you allow for legitimate unsolicited email from people?

Currently I receive lots of unsolicited mails from people that I want to hear from. Let's call these people "customers".

Your scheme would have me polling only people I have already talked to.

John.

Re:Sender ID = Caller ID = Worthless

[joebubba]

In Qwest territory, incoming calls with no number never ring here. They are given the opportunity to unblock their number, or to manually key it in. Either choice will let the call ring through.

I've only had the pleasure of one telemarketer bold enough to get through that and my "no solicitation warning". After I got them to give me their information I informed them that my number is on the Fed/State do not call list and I reported them.

It has only happened once. My phone is now forever dead quiet unless it is someone I actually want to talk to.

Um, no.

[Anonymous Coward]

Let's place the blame where it is due. If the recipient's ISPs are rejecting your bosses' mail on the basis of SPF records (as you claim), it means your boss is sending mail through a SMTP server which is not authorized by the SPF records you have published.

Which means your bosses' machines are misconfigured. It's lame to try to lay the blame for that on SPF, which, while imperfect, should never lead to cases like this.

Re:Um, no.

[taustin]

You are completely, totally, and entirely incorrect, and know nothing about the situation.

We are 100% SPF compliant. Tested and everything. The server we send through is on the only IP address allowed in our SPF record.

The address we send to is a virtual domain that does not offer POP3 mailboxes, only forwarding service. This is not under our control. It forwards to a third domain, also not under our control, that rejects based on SPF, because the second server - the forwarding service - does not rewrite the MAIL FROM address (and is not supposed to, so far as I know, under the RFCS).

All three machines are 100% compliant wit the RFCs, and the two using aspects of SPF are 100% compliant with those specs. And yet, legitimate email is being rejected.

And the only way around it is to misconfigure the server not using any aspect of SPF to violate the RFCs regarding how email is supposed to work.

Or, even better, use -all on our SPF, and thus explicitly enable precisely the kind of forgery that SPF is supposed to prevent.

That is the very definition of a broken system.

Pay To The Order Of

[Doc Ruby]

I thought the IETF wouldn't approve patented specs as standards. This MS move to take over the Internet must come bundled with some pretty good checks to "the right people".

Parent is OVERRATED

[Medievalist]

It's all well and good that something is being attempted to alleviate spam in this manner

SPF and Sender-ID are anti-forgery technologies that do nothing to block spam .

There is ample documentation available. Try this if you've got a PDF viewer.

I believe that is the problem with forwarding.

[khasim]

http://spf.pobox.com/faq.html#whichfield
So, this is implementation specific, but it seems that it will compare published SPF record of the domain in the FROM: or the return path with the fully qualified domain name of the sending machine (zombie123.earthlink.net yields "earthlink.net").

So, if the incoming email claims to be from/return-path taco@slashdot.org and slashdot.org publishes an SPF record, that SPF record had better list zombie123.earthlink.net as a legitimate mail server or it will fail.

What, specifically, happens when it fails is also up to the implementation.

The problem appears when taco@slashdot._org sends an email to my old college which offers forwarding services for alumni.

taco@slashdot._org sends to khasim@example._com

mail.example._com forwards that message to my gmail account.

mail.gmail._com checks the From:/return of slashdot._org and checks their SPF record for slashdot._org.

slashdot._org does not list any example._org boxes as a mail server so the message fails the SPF check.

Again, what happens at this point depends upon the implementation of SPF that is being used. It can range from increasing the SpamAssassin score to dropping the connection attempt.

Re:SPF versus Multiple Sites?

[quantum bit]

No, the solution is that your company should have an external authenticated mail relay that is included in the SPF record.

Authenticated is the key word here. Anybody who's roaming uses the company's relay. Hell, use it internally too and you don't have to change any settings while away. I've yet to come across a mail client that doesn't support SMTP AUTH, and many allow you to "use the same password that I do for checking mail" for convenience.

The mail relay should run on the submission port (587), or better yet over SSL (port 465). This gets around the port 25 blocks and transparent redirects that many brain-dead ISPs and hotels have.

Want to stop spam?

[swordgeek]

Arrest the fuckers. Throw Scott Richter in jail for a decade or two for fraud and theft. Break the back of the organised crime syndicates that are profiting. Revoke FDIC/CDIC approval for banks who benefit from mortgage spam. Have the CEOs of explicitly supportive ISPs (MCI, for instance) arrested and fined tens of millions of dollars. Threaten economic sanctions against countries who don't take reasonable action.

Like most crime, the laws exist to stop the small criminals, and have no ability to nail the true sources. Technology is always used to try to fix this problem, and always fails.

Re:I'd love to see an Apache Project mailserver.

[AndersM]

I'd personally love to see the Apache Project coordinate and release a mail server.

Obviously this guy has not heard of Postfix, a truely awesome mailserver

Postfix is not an Apache project, Wietse Venema still runs the show himself. "James" (http://james.apache.org/) is the Apache project's attempt at an email server.