Are You Using SPF Records?

gravyface writes "I've been setting up proper Sender Policy Framework records for all my clients for past year or so, hoping to either maintain or improve their 'reputation' in the email universe. However, there's a lot of IT admins I speak with who either haven't heard of SPF records or haven't bothered setting them up. How many of you are using SPF records for your mail domains? Does it help? How many anti-spam vendors out there use SPF records as part of their 'scorecard'?"

Comments

No, and I won't

[XanC]

SPF is harmful.

Re:No, and I won't

[hedwards]

That's some nice FUD you've got there. There are some valid points, but the basis for the page is inaccurate. SPF is really meant to be used in conjunction with some other technology like DKIM. SPF is just meant to ensure that one aspect of the process, that the machine sending the email is allowed to do so. DKIM for instance is meant to verify the contents of the email hasn't been tampered with.

Unless something has changed in the last few years, DKID doesn't verify that the server is allowed to send email for that particular domain, just that the email itself wasn't tampered with which has its own issues. For instance, while it does verify that the message hasn't been tampered with, it does not help people that are sending legitimate cold emails to a server of say the sales rep for the company.

My point here being that any technology has issues when one tries to use it in a way for which it isn't designed, but saying that because it can be used improperly that it's therefore dangerous is an argument which lacks merit.

Re:No, and I won't

[Anonymous Coward]

While that article raises valid points, I think it goes too far when saying "If you publish SPF records, you are going to be asking people to throw away genuine email which you did actually send." I am perfectly capable of limiting my mail-sending practices to be compatible with SPF, and I am not personally all that inconvenienced by people with misconfigured email systems trying to do both SPF and forwarding.

Of course, people to whom email is just another way to make a quick buck may have different ideas.

Re:No, and I won't

[martok]

Actually, DKIM can be used to guarantee a sender. We're using DKIM here with ADSP. That is:
_adsp._domainkey TXT "DKIM=ALL"
tells a receiver that all emails from our domains should be signed. Since the keys themselves are published in our DNS, a machine not under our control should not be able to send an email purporting to be from our domain.

I'm not sure but I would think that mechanism would make SPF irrelavent. Assuming antispam software actually checked the adsp dkim records.

Re:No, and I won't

[nsayer]

His protest is without teeth. If he really objects to the concept of SPF, then he should publish an SPF record of "?ALL". That way, people will know he's just not being apathetic.

Re:No, and I won't

[Krondor]

How would that work with trusted partners who may send mail on your behalf? With SPF I can use an include:xxx to define relationships with other systems. With DKIM it seems I would need the partnered system to stamp the sent mail or relay off of our originating servers for DKIM attribute addition (something that might not always be possible). Is there an elegant workaround?

Re:No, and I won't

[Albanach]

So the argument is that because you can't forward mail, SPF is broken.

Forwarding mail is almost entirely unnecessary. Every major webmail provider allows you to get mail from third party accounts via POP3/IMAP. Rather than forward mail just fetch it like any other client. It doesn't need anything to be upgraded, works reliably and allows you to use SPF verifying the hosts permitted to send mail from your domain.

Re:No, and I won't

[edman007]

DKIM gets the public cert through DNS, thus to let someone send on your behalf you can do make a private and public cert for the sender and then host the public cert at sender-month._domainkey.example.com, when they send an email they get the public from a DNS server you control which should be easy enough to segregate from other peoples mail servers.

Re:No, and I won't

[Krondor]

I understand that, but it still doesn't address the include:xxx condition I outlined above. If we use an application service provider that sends email on our behalf, I have to get that provider to setup a custom header in the outbound email with a private cert I have generated for them. With SPF I can simply use an include: xxx to specify that I also trust vendorx.com to send mail for mydomain.com. I was inquiring if there is a facility for DKIM to support such a mechanism, which it doesn't seem like there is.

I can take a hardline with the ASPs and require they allow stamp the mail with my DKIM, but if you're not a large enough customer chances are they will say tough deal with it or go somewhere else.

Re:No, and I won't

[edman007]

DKIM verifies the actual message not the server, yes it can be more work to setup, but you end up with end to end security and you can differentiate between different senders on the same IP.

Re:No, and I won't

[Martin Blank]

We don't use the records where I work (too many uncertainties about where government e-mail is coming from, which is a stupid reason to not do it), but we do check them. Except for a few isolated cases of people sending corporate mail via their cablemodem/DSL SMTP servers and the case of ValueWeb[1], we rarely have any problems with it, and drop several tens of thousands of messages per day. That's out of about 2 million dropped or rejected connections, so it's a small fraction, but every bit helps where it doesn't hurt that much. (Spamhaus is responsible for most of the other dropped connections.)

[1] ValueWeb's DNS servers return NXDOMAIN results when doing TXT record queries against them, unless the record explicitly has a TXT record in it, when it should return a NOERROR response with empty results. In addition, asking them to add a TXT record to the zone almost always results in them adding it to NS1, but not to NS2 or NS3, unless the customer specifically requests it. I still would like to know how the NXDOMAIN is generated in these cases.

Re:No, and I won't

[WuphonsReach]

My numbers (measured over a few hundred thousand messages) is that ~61% of all messages had SPF records, and we dropped 1/3 of those due to failing the "-all" check.

(Keeping in mind that we implement a few checks before this point, such as testing HELO records for RFC compliance and a basic DNSBL. So the really screwed up senders are already filtered out.)

Re:No, and I won't

[sprior]

While I can see the reason for your points, I don't agree with the conclusion.

The fact is that implementing SPF comes with a bigger responsibility to account for every machine your email might be sent from. No doubt this can be a big pain and imply compromise.

I have implemented SPF and for me the big downside is that OTHERS don't check it and pay attention to what I've implemented. I still get bounced email which the receiver SHOULD have ignored as forged because it failed SPF checks, but they didn't and bounced it to me anyway.

So my complaint is that being responsible hasn't bought me as much as it should have.

Re:No, and I won't

[Belial6]

So, your saying that the system is technically a good solution, but your partners will refuse to take the steps to protect your reputation?

Re:No, and I won't

[mysidia]

It's not harmful. That post is based on a lie.

So they have an account elsewhere, at a vanity domain or on another computer, and they forward mail from that address to whichever is their current ISP, or their employer.

The keyword is elsewhere.

RFC 1123 deprecated the concept of a proper return path. If it hadn't happened, there would be no issue.

But due to RFC 1123, if you place another mail server's hostname in the "MAIL FROM" line, you are committing forgery, regardless of any ad-hoc justification you may come up with for "permitting" it.

The abuse doesn't make a technology that stops it evil.

Particularly when there is a good workaround called SRS.

Re:No, and I won't

[XanC]

But due to RFC 1123, if you place another mail server's hostname in the "MAIL FROM" line, you are committing forgery, regardless of any ad-hoc justification you may come up with for "permitting" it.

I don't see that in RFC 1123. Can you be more specific? I could easily have missed it.

Re:No, and I won't

[mysidia]

Of course this breaks forwarders ahem, "legitimate" forgers who want to stuff an address they have on someone else's mail server in the "MAIL FROM" line as well...

Referencing someone else's mailserver in 'MAIL FROM' is always forgery, due to the very nature of RETURN PATH. When you place your server name in the Return path, you are claiming that server said they were responsible for the message

Again, prior to rfc1123, this was a lot clearer; every relay in the path the message had passed through would appear in the MAIL FROM line.

SPF and 'MAIL FROM' / Return-Path have nothing to do with 'From' line in the e-mail message itself.

The From: line is user data, not subject to SPF checks, and doesn't have to match the return path.

Using whatever e-mail address you own in the From: line is fine.

Placing someone else's mail server's hostname in the 'MAIL FROM' or Return-Path headers is forgery, and blocking this is one of SPF's best features.

Re:No, and I won't

[mysidia]

Well, the section of interest is 5.2.6 of RFC1123, which killed section 3.6 of RFC 822 with the stated goal of abolishing source-routing of the destination or of the return path.

Basically, the baby was accidentally thrown out with the bathwater, and it is unfortunately way too late to do anything about it..

5.2.6 Mail Relay: RFC-821 Section 3.6
....
the relay function defined in section 3.6 of RFC-821 should not be used.

Now, if you go lookup that section you will find:

... The mailbox is an absolute address, and the route is information about how to get there. The two concepts should not be confused.

Conceptually the elements of the forward-path are moved to the reverse-path as the message is relayed from one server-SMTP to another. The reverse-path is a reverse source route When a server-SMTP deletes its identifier from the forward-path and inserts it into the reverse-path, it must use the name it is known by in the environment it is sending into, not the environment the mail came from ...
If when the message arrives at an SMTP the first element of the forward-path is not the identifier of that SMTP the element is not deleted from the forward-path and is used to determine the next SMTP to send the message to. In any case, the SMTP adds its own identifier to the reverse-path. ...

Re:No, and I won't

[XanC]

Please bear with me, I'm confused.

RFC 821 has specific requirements about the return path, then RFC 1123 says those are void, and so the conclusion is that RFC 1123 says they're still intact?

I often send email from my servers on behalf of third parties, with the return path being the email address of the third party, because that's where I want the bounces to go. Is this illegal according to RFCs currently in effect?

Re:No, and I won't

[MikeBabcock]

No, its being said that setting up signed E-mails hosted by third parties can be a royal pain. I relay mail for my customers as one of those third parties and working out who has control of the domain vs. E-mail vs. apps sending E-mail is a nightmare sometimes.

There's no necessity that a customer does their own domain work, or even controls their own E-mail system, and then wants us to send E-mail on their behalf from their domain, when their domain uses SPF and signed records.

Re:No, and I won't

[Martin Blank]

We do something similar. Bad HELO/EHLO? Disconnected and blackholed for an hour. It then goes against three RBLs, Spamhaus being the most effective. For what we pay, it is by far the best value of the entire system. Anyway, anything matching that is blackholed for an hour. Then we check for spoofed addresses, then SPF records. After that, it's sent on to the anti-spam systems. We figured last that we block about 92% of all attempts to send messages, between dropped connections, rejections, and quarantines.

Re:No, and I won't

[mysidia]

rfc1123 removed the relay function whereby your mail server may list other server's names in the return path.

The replacement is that only your mailserver is listed in the MAIL FROM entry. The RFC never said anything about adding some other server's address to a MAIL FROM line, that was never allowed by the standard. Your mail server can only add its own address as it is known to the mail server it is sending to period (for a new mail from: line).

To list the address in MAIL FROM: means that you (the sending mail server) are claiming you can take responsibility for being able to return mail to the address you list.

That is: by listing an address in mail from, you PROMISE you can immediately return mail to the host you received it from, if there is an error.

There are traditionally only two ways a mail server can take that responsibility: either (1) the item you list in MAIL FROM is you, and you list a local user, then of course you can return the message to your local user, OR:

(2) You are relaying the item, and you received that address in MAIL FROM. You can live up to the promise by delivering the error message to the server that relayed you the message with that address in MAIL FROM, and that server can live up to their promise, and so on.

Well, without source routing (2) is actually a stretch.

The elimination of option (2) as a way to satisfy the promise you make means that you are left with option (1) only.

Re:No, and I won't

[Antique Geekmeister]

Since SPF acts on the bounce address, not the "From:" address, it's not necessarily a big issue. You should set up SRS, which resets the bounce address to be you, and makes sending the bounce message back to the originator your problem, not hte email recipient's.

Re:No, and I won't

[gmack]

And this is why I was forced to turn it back off again. Too many people checking for SPF without SRS caused a lot of my email to be silently dumped.

Re:No, and I won't

[DrSkwid]

Spoken like someone who's never had to deal with big companies IT dept. when you're a little fish.

"But we do X"
"That's nice for you, we don't"

Re:No, and I won't

[arth1]

(Keeping in mind that we implement a few checks before this point, such as testing HELO records for RFC compliance and a basic DNSBL. So the really screwed up senders are already filtered out.)

Excuse me? If you're worried about RFC compliance, you should NEVER reject messages on the basis on the string that the remote presents in a HELO. There are good reasons for this (the most common one being NAT traversal, where the hostname you can look up is one bound to the gateway, and not the sending host's hostname. A less common scenario are alert e-mails sent through literal IP that DNS servers are down, where the hostname won't match because, well, the DNS servers are down...)

By all means, use the HELO text as weights in your spam filtering, but do not block senders based on it. At least not in the name of wanting to be RFC compliant.

Re:No, and I won't

[WuphonsReach]

Go read RFC 5321, specifically section 2.3.5.

The domain name given in the EHLO command MUST be either a primary host name (a domain name that resolves to an address RR) or, if the host has no name, an address literal, as described in Section 4.1.3 and discussed further in the EHLO discussion of Section 4.1.4.

If your mail server talks to the outside world, it must introduce itself properly. There's no wiggle room there.

Re:No, and I won't

[kju]

rfc1123 removed the relay function whereby your mail server may list other server's names in the return path.
The replacement is that only your mailserver is listed in the MAIL FROM entry. The RFC never said anything about adding some other server's address to a MAIL FROM line, that was never allowed by the standard.

This is utter nonsense and a major misunderstanding of the standard on your side. Also RFC821 was deprecated by RFC2821 eight fricking years ago. It is telling that you apparently did not know that, because otherwise you won't argue with RFC821 and RFC1123 which updates RFC821 instead on looking at the later standard which incorporated the changes proposed by RFC1123. It is also important to understand that a RFC is what it names tell: A Request for Comments, not a binding standard. The pure existence of RFC1123 does not mean that everyone has to follow it ideas.

Having said that, and ignoring your misleading statements, the case is very simple. Regarding to RFC2821 - which actually is widely accepted as an standard - the MAIL FROM command specifies " The reverse-path consists of the sender mailbox." It is a no-brainer that a relay is a relay and not the sender of the message and thus does not change the "sender mailbox". So to comply with RFC2821 as relay can NOT change this value and HAS TO use the original address (which is by the way not a server address as you called it). While this is fully standard compliant, the existence of SPF and other spam filtering messages nowadays make it impossible to follow the standard in this, thus mechanism like SRS (sender rewriting scheme) were invented. But while the use of such hacks is widely accepted and understood as a necessity, it is in fact a violation of RFC2821 which mandates to keep the original value.

And for the issue of what a RFC means, you really should read RFC 2026 "The Internet Standards Process". As said, a RFC is not a standard, you might have confused RFC with the BCP (best current practice) and STD (standard) series of documents.

Re:No, and I won't

[mysidia]

RFC1123 is a standard, STD 3 is currently RFC1123. RFC2821 is currently not a standard of any kind. Check the listing in the RFC Index next time.

1123 Requirements for Internet Hosts - Application and Support. R. Braden, Ed.. October 1989. (Format: TXT=245503 bytes) (Updates RFC0822) (Updated by RFC1349, RFC2181, RFC5321) (Also STD0003) (Status: STANDARD)

2821 Simple Mail Transfer Protocol. J. Klensin, Ed.. April 2001. (Format: TXT=192504 bytes) (Obsoletes RFC0821, RFC0974, RFC1869, STD0010) (Obsoleted by RFC5321) (Updated by RFC5336) (Status: PROPOSED STANDARD)

0821 Simple Mail Transfer Protocol. J. Postel. August 1982. (Format: TXT=124482 bytes) (Obsoletes RFC0788) (Obsoleted by RFC2821) (Also STD0010) (Status: STANDARD)

Re:No, and I won't

[kju]

Ok, i haven't looked at that, sorry. But still RFC2821 is widely accepted as a standard and has incorporated the changes from RFC1123. And your main claim (about the value of the MAIL FROM command) is still wrong.

Re:No, and I won't

[XanC]

There are a lot of assertions here, but no quotes or references to specific parts of RFCs.

The RFC never said anything about adding some other server's address to a MAIL FROM line

A server doesn't go in there, an email address does. The address of the sender.

That is: by listing an address in mail from, you PROMISE you can immediately return mail to the host you received it from

Who says? Is "PROMISE" defined like "SHOULD", "MAY", "SHOULD NOT", etc?

Re:No, and I won't

[mysidia]

Oh really.. No. I think it's more of a case of RFC2821 being revisions to the standard proposed to make it more accurately reflect current implementations / current common practice.

If it were generally accepted and widely seen to have no issues, it had plenty of time to cross from PROPOSED STANDARD to Full standard.

Obviously someone (or some people) objected to rfc2821 and hold that further revisions are necessary, before a new standard is arrived at.

Meanwhile, implementors who see advantages of some of the changes in 2821 begin to incorporate some of what they see as the best improvements into their implementations.

Of course Internet Standards can't force anyone to do (or not do) anything.

MTA developers can and should play around with experimental features. It's how new standards get developed, in the first place.

Re:No, and I won't

[WuphonsReach]

RFC 5321 has replaced RFC 2821.

Re:No, and I won't

[arth1]

Yes, there are rules for what the HELO and EHLO should be. Rule for the SENDER, not the RECIPIENT. That the sender hasn't followed the rules gives the recipient no right to also break the rules. "He started it!" didn't work when you were five, and doesn't work now when you're twelve.

The very same RFC that you quote references RFC2821, which quite explicitly states:

An SMTP server MAY verify that the domain name parameter in the EHLO command actually corresponds to the IP address of the client. However, the server MUST NOT refuse to accept a message for this reason if the verification fails: the information about verification failure is for logging and tracing only.

In other words, if you use the HELO/EHLO to block senders, you are the one breaking the RFCs, not them.

If your mail server blocks me when I (correctly) send "EHLO [123.45.67.89]", or "EHLO hidden.invalid", your mail server is misconfigured, and doesn't follow RFCs. Period. To then claim that you do the blocking to protect the RFCs is disingenuous.

Re:No, and I won't

[Eric S. Smith]

How would that work with trusted partners who may send mail on your behalf?

There is no such thing (unless you're in the business of spamming people).

I guess my work does its own on-line order processing. I could've sworn that we outsourced it, and that I'd set up our SPF record to allow it, but I must be wrong.

Re:No, and I won't

[XanC]

The RFC forbids rejecting the message due to the domain of the EHLO string not matching the IP address of the sender.

It doesn't say anything about rejecting based on EHLOs that are missing, badly formed, refer to nonexistent addresses, etc.

So I certainly may and will use the EHLO string to block senders, as long as I don't reject mail for the reason that the connecting IP doesn't like up with the EHLO (which would be stupid for many reasons).

Re:No, and I won't

[hyartep]

So my complaint is that being responsible hasn't bought me as much as it should have.

i think, it's a situation, where few decent "people" have a hard life living in the wild country.
however as the number of decent people grows, being good is more and more rewarding and being bad gets you being isolated.

by being good you help the good side prevail sooner.

Re:No, and I won't

[welsh git]

You really haven't thought this through, have you?

Re:No, and I won't

[DavidTC]

Yes, the email cannot be rejected on the basis of the domain name not matching.

But the RFC is silent on the cases of it not being a real domain.

You aren't required to accept everything there, just actual domains, which you have to accept regardless of them being the rDNS, or the DNS being the IP, of that machine. You can do forward and backwards lookups if you want, but can't reject based on them according to the RFC.

But if they aren't actual domains names, or bracket enclosed IP addresses, you can do whatever the hell you want with them. (Although you might want to consider accepting non-bracket enclosed IP too.)

Of course, anyone who wanted to block non-matching HELOs could just claim that they currently only authorized servers with matching DNS, rDNS, and HELO to access their server, and hence they are blocking based on an 'authentication or authorization-determining mechanism', to follow the exception listed in that section.

Or, to quote the RFC itself at 7.7: It is a well-established principle that an SMTP server may refuse to accept mail for any operational or technical reason that makes sense to the site providing the server.

Which is really how your entire argument about what servers can and can't do falls apart. The SMTP standard defines a protocol. The RFC standards themselves say, quite clearly, that you can block anyone you don't want at all. Pretty much all requirements that say what mail you should accept also says 'Unless you want to block unauthorized people'. Well, if you block them, and reject them with a message saying they are not authorized, they are ipso facto unauthorized.

And then, just to make it clear, it comes out and says 'You can block anyone for any reason you want'

This is because, despite what you seem to think, the RFCs are written to define a protocol and what programs following the protocol do. Programs that follow the mail RFCs have to be able to accept mail from people with incorrect domain names in the HELO, or they do not follow the RFC. There is a bunch of talk about what programs must do to be within the RFC.

But people do not have to configure the programs to do that if they want to reject mail that is like that, as the RFC specifically says.

Re:No, and I won't

[plague3106]

Bull. I have an email address, which I give to everyone. The service I use to get the address provides NO STORAGE, it's only purpose is to forward mail to another address I specify.

This is very useful because I can give this address out, and not have to worry about changing who's hosting my mail.

Yes.

[growse]

Yes, I use an SPF for my domain. No I don't have any idea how effective it is, because my SPF record is used by other people. I haven't had any complaints about people not getting my mails.

Re:Yes.

[oatworm]

Pretty much the same here - SPF records aren't particularly hard to implement, after all. On the receiving side, I just check for SPF failure (i.e. somebody e-mailing from somewhere other than the domain's SPF-registered mail server), and even those just get sent to users' junk mail folders. I'm certainly not bouncing anything because of them. Based on my mail server reports, it looks like the low SPF filtering is catching about 0.5% of the mail volume that flows my direction, which isn't much, but it's 0.5% less than I would be dealing with otherwise and was implemented "for free", so I'm not complaining.

Re:Yes.

[maxwell demon]

Do all people you send mails to expect you to send mails to them?
Did you eventually get no answer to a mail you sent to someone you don't normally send mail to?

People won't complain about not getting your mail if they don't have a clue that you sent them any mail.

Re:Yes.

[pclminion]

I communicate with the homeless by thought projection. I like to let them know that they can come over for steak and beer any time they want. I think these thoughts vigorously every night. I have yet to hear any homeless person tell me they are not receiving my messages.

Re:Yes.

[PopeRatzo]

I communicate with the homeless by thought projection.

Your mother is homeless, you insensitive clod!

Re:Yes.

[dhammabum]

I've checked the number of DNS queries to our domain looking for these SPF records a few months ago - got about 50 requests / hour compared to say 2-300 messages/hour total. Of course there is no direct relation between the number of messages we send and those queries necessarily....

It is easy enough to test for it: dig -t txt domain.tld - I found it funny that most of the large corporates here in Oz don't use it.

Re:Yes.

[fearlezz]

I have had a few complaints about people not getting my mails. But all times, the fault was on the receiving mailservers.

Mostly, the situation was:
[my-mailserver]<------>[their-barracuda]<------>[their-mailserver]
When their-mailserver isn't configured to check the SPF of the second hop, mail gets rejected as their-barracuda clearly isn't one of my servers. I don't feel responsible for other organisations' misconfiguration.

Re:Yes.

[TheRaven64]

I use SPF, but SPF is not meant to combat spam, it is meant to combat Joe Jobs. If someone is sending spam from a botnet and claiming that it's from my email address, then spam filters can use SPF to avoid sending me backscatter. Unfortunately, incompetent idiots set up their mail server to accept mail and then send a 'message rejected' email containing the spam email to whoever happened to be in the From: field. I got a lot of spam from backscatter by some of these idiots recently. Their domain was gmail.com.

I use them

[NormalVisual]

I use them for all of my domains, but I can't really see that it makes the first bit of difference.

Re:I use them

[Aladrin]

Ditto.

Re:I use them

[digitalchinky]

Not just to add a 'me too' but I recently removed SPF completely - mostly because other people couldn't get their entries correct, or just completely failed to update it when they add in extra servers. Legitimate messages were hitting our spam folders. Since I can't train our fine worker drones to actually look in their spam, I opted just to remove it. With greylisting and spamassassin its removal hasn't made any noticeable difference aside from the false positives now being delivered properly.

Re:I use them

[GreyFish]

It may be making a difference for *other people* who can now discard spams that use your domain name.

What me worry?

[P1aGu3ed]

If you maintain your own public DNS server you have no reason not to include SPF records, however many of the public DNS providers support little more than A CNAME and MX.

Re:What me worry?

[imemyself]

Who doesn't let you create TXT records?? I've used DynDNS's Custom DNS service, Slicehost, and GoDaddy's DNS service and I've always been able to create TXT records. I'm not doubting you, but I for one would be royally pissed off if I signed up for a DNS service that didn't actually support all of the normal records.

yes

[zeldor]

it has cut down tremendously on the spam claiming to be from my domains.
any other benefit I am unaware of.

Re:yes

[MightyMartian]

If you're using a lack of SPF records as a determinant in whether or not a message is spam, then I can guarantee you that you're losing mail to false positives. Maybe that isn't a big deal to you, but for the organization I work for, it would be absolutely nuts. The chief reason I have SPF records for my domains is so that the big boys like hotmail.com and GMail don't reject my emails. I use greylisting as my chief anti-spam weapon, and it's far more reliable and far more effective than SPF.

Re:Yes

[sjwest]

Logwatch is a useful tool for monitoring, it seems to work well for our domains, dkim/domainkeys and spf do help.

If people don't want to set them up for a mailserver to use then well thats your choice. If the context they are used in is too hard for them to grok then perhaps they should not be looking after email systems or any it.

There are idiots out there (hey magnatune) who don't sign all there mail servers and that is unfortunate in magnatune's case there office box has it, but buy an mp3/etc thing and it goes bang.

Nobody home when i emailed them about it the once.

I understand that messing about with smtp/lmtp,and av, and then perhaps a disclaimer, before signing a message for mailing might scare some 'administrators' - i would consider them unemployable.

Re:yes

[JWSmythe]

    Folks who do a lot of mail find out the hard way that without SPF records, there are plenty of places that bounce them. I've had them on my domains for years.

    For my old network, where we got a huge amount of spam, we used both graylisting and our own custom blacklist. I didn't trust the blacklist providers, so we did rolling blacklists based on the amount of detected spam (with mailscanner and friends), which worked with the firewall. It set it's own firewall rules, so all traffic was dropped from that IP. On the first offense from an IP, it was blocked for a day. If there were multiple spams detected from the same /24, the whole /24 got blocked. If they were repeat offenders, the durations increased. It protected the mail server from about 90% of the spam, and didn't generate a single complaint. There was a tremendous amount of inbound mail also that was legitimate, so we would have had complaints after their automatic block was lifted.

    It also used some honeypotting. Messages to old dormant accounts that only received spam automatically had the sender blocked. It's not like the accounts were a few days unused, we're talking about more than 5 years, and they were some of the highest traffic accounts on the server.

    An offense was carefully defined, so as not to block legitimate traffic. It worked amazingly well. For it to work though, you have to have a high-load environment, that the spammers are already hitting hard. We would receive upwards of 100k emails/day, which was then reduced to 10k and most were legitimate.

Re:yes

[ls671]

Agreed with both your post and the GP !

I publish SPF records for all my domains for which I know *for sure* the IPs from which mail might be sent from and I take care of using the -all qualifier which is FAIL ( NOT SOFTFAIL which uses a tilde ). This is telling other mail servers using SPF to refuse the email when not coming from the published list of IPs.

I barely take SPF into account to filter incoming email for basically the same reason you have mentioned.

Oh, I do not use greylisting because having emails delayed by 1 to 6 hours, for the organization I work for, "would be absolutely nuts".

Re:yes

[Snover]

Read again.

Spammers can’t use his domain to forge spam, because SPF-aware mail servers reject it. Hence, he doesn’t have to deal with tons of bounces, spam warnings, virus warnings, etc..

Re:Yes

[SmoothriderSean]

But I can tell that Hotmail still ignores SPF since almost all the backscatter that still comes through is from Hotmail. They should know better.

I believe you, but really? Hotmail was THE reason I've implemented SPF for a few domains connected to sites that send alert emails to users. Nothing - from email confirmations to status update type stuff - was getting through to Hotmail accounts until I set up SPF. Some kind of Left Hand / Right Hand mess going on over there?

Re:yes

[2Bits]

The chief reason I have SPF records for my domains is so that the big boys like hotmail.com and GMail don't reject my emails.

I set up SPF records properly, hoping that mails from our domain are not blocked or bounced. It works a little bit, but my frustration has not reduced that much. Now, I don't care any more, the SPF records are still there, but that's all.

The big boys are actually quite nice, and they do care about that. I got in touch with their admins, they did a verification (quite fast, I must say), and made sure mails from our domain went through. The problem is with cocky admins from small IT shops and ISPs who probably know jack about it, and who just chose to block the IP range of a whole continent. Yes, we are in China, but note that not all email servers in China are open relays, just like not all email servers in America or Europe are open relays.

So, no, I don't care about SPF any more. It does not work as it is meant to.

Re:yes

[jonadab]

> If you're using a lack of SPF records as a determinant
> in whether or not a message is spam, then I can guarantee
> you that you're losing mail to false positives.

Yeah, that's not how you're supposed to do it. No SPF record means "I don't know whether this is a valid sender for this domain or not", so you fall back to whatever other method you have for making the determination.

Traditionally, the "other method" generally meant accepting everything and letting the users sort it out in the inbox, but there are other things you can do, such as looking at stuff like whether the EHLO domain matches the sending IP, whether the envelope sender matches either of them, whether the From field matches any of the above, whether any of our users have sent mail to this domain in the last N days, and so on and so forth, each of which criteria can be assigned a weight, which can be used to determine whether to accept the message immediately, graylist it (and for how long), subject it to more rigorous (but computationally intensive) regex-based filtering, check for it on collaboratively-maintained blacklists, etc.

And if the domain *has* SPF records and the sending IP isn't in them, you have your choice of greylisting with a lengthy delay, going straight to teargrube mode, or just sending a 554 response immediately. And you can take note of the IP address for future reference.

> I use greylisting as my chief anti-spam weapon, and it's
> far more reliable and far more effective than SPF.

Greylisting has problems too. Not least, it causes delays, which can run into multiple minutes. (In some situations, even a thirty-second delay will make the natives restless.) Additionally, if too many mail exchangers do greylisting, the spammers will just implement retries (it's not like it's hard; normal mail servers have been doing it forever). All of this doesn't mean greylisting isn't useful, but it's one tool in the toolbox, not the be-all and end-all. There are a lot of things a mail server can do to try to determine whether a message is spam. None of them individually is very reliable, but you don't have to do just one thing.

Re:yes

[man_of_mr_e]

Well, I don't know about right now, but as of about 2 years ago any email I sent from my domain silently disappeared when sent to gmail and hotmail. It didn't even go into the users spam folder, just disappeared. Adding an SPF record fixed it. So maybe you're right, they don't "reject" the mail, they just ignore it.

Re:yes

[JayAEU]

Greylisting has problems too. Not least, it causes delays, which can run into multiple minutes.

That's why I recommend no-listing. Basically you have the first and last MX record point to nowhere, which gets rid of lots of spammers not doing proper MX traversal and retries. The "real" MX entries are "hidden" in between the no-listing ones, priority-wise.

For proper mail servers it's no slowdown at all, while spammers get shot down right off the bat.

Re:Yes

[S-100]

Could very well be. SPF may help mail get through, but I don't have enough info to say one way or the other. But I can definitely say that I still get backscatter bounce messages from hotmail. Always from spam with a forged return address to my domain. Just got one today from above.net, but what was once a frequent occurrence is now rare.

Re:yes

[Bigbutt]

Actually my big problem is with a couple of Canadian mail ISPs (shaw.ca and sympatico.ca). They use some black list service which doesn't have a way of getting a mail server off of their list. I changed my mail log rolling so it would save mail for a year. I ran a check on the logs for the past year and was able to account for every message I sent. A majority of incoming e-mail were open-relay attempts from Taiwan. I finally just blocked Taiwan entirely which dropped the attempts off quite a bit.

Microsoft is a bit crazy. Every few months they flip-flop as to whether they'll receive mail from my domain or bounce it. The only option that seems to be available to me is to pay them to have my domain added to their advertisers list. Since I'm not advertising anything (it's a Mosaic and Stained Glass forum), I won't be paying them.

Carl

Re:yes

[gladbach]

This happens for sure, however I am willing to bet they were blacklisting you for some other reason, and decided to trust you again once they detected the spf records published for your domain. I see this situation all the time with yahoo and gmail.

Re:yes

[yuna49]

Now there's a strategy I haven't heard before. Is it considered RFC-compliant to list MX servers you know for sure aren't authoritative for your domain? I guess in practice it only slows your own inbound connections, though it does add marginal overhead to the network as all those servers must issue the initial failing queries. Even with locally cached records there's still computational time involved in asking the question.

Some spammers routinely choose to mail to secondary servers, perhaps because they find those servers have less vigilant defenders athwart the walls. I'm going to try this strategy:

          IN MX 0 mail.example.com.
          IN MX 10 mail.example.com.
          IN MX 20 mail.my_real_primary.net.
          IN MX 30 mail.example.com.
          IN MX 40 mail.my_real_secondary.net.
          IN MX 50 mail.example.com.

The first two and the last protect protect against the most obvious strategies. The 30 record between the real servers may be overkill.

Re:yes

[MightyMartian]

Only a small fraction of spammers, from what I can tell, have ever gone for retries. In particular bot attacks are just buzzing through addresses as fast as possible. They ain't sitting around waiting to find out if the server accepted the message or closed the connection.

I long ago explained to my users that there are numerous reasons why an email will not be delivered instantaneously, and that if they want instantaneous communications, there are a number of chat clients out there designed specifically for that purpose.

The nice thing about pure greylisting is it doesn't wait for the message. It's looking at the SMTP commands and the connecting IP address.

Re:yes

[scottv67]

>Nobody rejects mail because there is no SPF record. Obviously you have no idea how it works. Please read up on the subject before commenting next time.

Yes, organizations *do* reject email when a valid SPF record does not exist. There is a large "educational institution" in the Milwaukee area that completely stopped accepting email from my employer because we did not have an SPF record. Not simply "marked as spam", they completely stopped accepting email we were sending.

Re:Yes

[Degrees]

Seconded. After putting in my SPF records, the amount of backscatter dropped a huge amount. Presumably the rest of the world got a little better too, as they could tell who the forgers are.

Re:yes

[sjames]

I have found that it cuts down a lot on bounce messages where my domains were forged. It's hardly a cure-all, but it helps iand takes very little effort.

Re:yes

[Serious Callers Only]

it has cut down tremendously on the spam claiming to be from my domains.

If you're using a lack of SPF records as a determinant in whether or not a message is spam

Well, if they were, that would be a problem wouldn't it. But that's not what the OP said. I also use SPF, and use it because it allows other mail servers to decide mail is spam based on the fact it claims to be *from* my domains, when it doesn't come from a mail server authorised for those domains. So if everyone used SPF, no spammers would be able to impersonate me without first hijacking my server.

That's exactly the opposite of using the _lack_ of SPF to determine anything, it's using the positive existence of SPF records for a domain to exlude all other messages claiming to be from that domain.

Re:yes

[DavidTC]

That's what I do, also. At some point, one of the domain names I'm in charge of accepted wildcard emails, although I'm not sure when. This was way back in the early days of companies getting on the internet, 1998 or so. And we got dictionary attacked, apparently.

As a result, there are hundreds of email addresses on spammer's lists that have never actually had a person behind them...and haven't accepted email for at least a decade. Also, there are addresses that, as far as I can tell, are off-by-one errors of addresses, with missing first, or even first and second, characters. Some of them, hilariously, are off-by-one errors of the dictionary addresses. Some of them are clearly people typing gibberish in somewhere, like a bbbbbbb@ one and a qqqq@ one.

So I have plenty of addresses that can't get any legit mail, at all. And aren't vaguely close to any sort of reasonable typo.

Well, we accept email for them. ;)

I have written a program that actually goes along and pulls mail out the maildir all these accounts deliver too, and runs it through spamassassin learning, and then throws the IP in an ipset table so that spammers can't even connect to our mail server. I'm pretty unfascist about anti-spam rules in general, with weighted rules and very few absolutes, but with the 'never actual valid addresses', hey, they're clearly spammers, period.

My only regret is that the company won't let me use greylisting, because whiney people would rather have ten times as much spam if it means their messages get there instantly, or something. If I could let the spamtrap mail in instantly, but defer other mail, that would be perfect.

Re:yes

[DavidTC]

Yeah, I can't use greylisting for the same reason.

As we're essentially in one time zone, I've played with the idea of greylisting only at night. Haven't seen a program to do that, but it shouldn't be that hard. (I guess I could always rewrite a postfix lookup table every time it changed.)

Re:yes

[DavidTC]

If you're worried about speed or something, just add the MX last record.

That causes absolutely no overhead 99.999% of the time. But plenty of spammers go straight to the last record. (On the theory that backup MX servers might not be running as much antispam check.)

Also, I'm not sure that listing a fake address is a good idea. Might want to list a real one, that you actually own, that doesn't have a mail server on it.

Re:yes

[DavidTC]

But spammers implementing retries would slow down their spam sending by quite a lot, as they'd actually have to keep track of it.

And it would not help at all in combination with spamtrap blacklisting, either local or remote. There are plenty of people, myself included, that run their own 'bad address' blacklist...you send to a certain invalid email address, you're blacklisted for a while. And there are the big, publicly accessibly lists that operate essentially the same way.

If you use that on top of greylisting (Which, sadly, I cannot because of the delay) by the time anyone gets around to getting back to you, that IP is blacklisted.

Re:yes

[DavidTC]

Oh, and while you're at it, make sure your mail server isn't on the IP that the A record points to.

Spam software is designed by total morons, apparently, and there is indeed some percentage that just totally ignores MX records, sending to the domain's A record.

Re:yes

[JWSmythe]

    That's pretty much the way we did it. If you hadn't already done the work, I would have said to use mailscanner. :) I used an addon withit, that logged in a database for me. The iptables rules were done with another script that ran once a minute to block new spammer. Either way. :)

    You might want to check out graymilter. If I remember right, you could whitelist known good senders, and it would whitelist good ones on it's own. I believe it's rules were dropped (other than your defined whitelist) if it was restarted. Pretty much, there would be a little hiccup in mail delivery when you first start it, and then it would run fine forever, or until you restarted the graymilter daemon or rebooted the machine.

    I know I looked at a whole bunch of solutions for graylisting. Some worked well. Some worked terribly. This one worked just as I'd like.

    After I got everything set up and tuned perfectly, I did get complaints. People said they weren't getting anywhere near as much spam as they usually got, and to them that showed how well the mail server was working. :) Graylisting for 30 minutes is effective, but graylisting for 30 seconds makes it so people barely notice. The first message is delayed for a relatively short time (depending on the remote server), but usually only about 5 minutes on the initial message, and no delay after that.

    Now, I don't run anything near as robust as this, because I'm not in control over any mail servers that have a need like this. The biggest mail server I run now has up to a 15 second delay, as it scans every inbound message, drops the high spam score messages, and delivers the low score messages. If they're remotely spammy, it tags them, so the client can decide how to deliver it, with it's own filters.

Re:yes

[ls671]

> I've played with the idea of greylisting only at night.

Interesting ! ;-)

But maybe you would have to publish that fact anyway just in case one of the big bosses stays up all night exchanging emails with somebody in order to finalyze that "very important" contract ;-))

I remember one of my bosses using email as a chat client during a very important power grid failure crisis ;-))

Re:yes

[DavidTC]

I would only do it if I could figure out how to run the 'put on the whitelist' part during the day. So anyone that's sent email at any time that's gotten in would be already cleared. (Cleared to not be greylisted, I mean, not whitelisted in general.) Run that for a week, collect 'tuples', and then turn it on only at night. Most greylisting programs can run in 'collection only' mode.

I've also looked at the idea of greylisting, only by IP plus domain for all free email services.

I.e., if some IP says they're gmail.com, we'd greylist that IP, until they retry that email (With the same sender and recipient) a bit later.

Then we'd let that IP give us any gmail.com email, to anyone.

I could even grep the mail log and find some starting IPs.

Sadly, I've seen no program that does anything like this. I think I could do it with a postfix map that only sends certain domains to be greylisted, though.

I publish but I don't check them

[chrisj_0]

I publish spf records. But I don't check them for any incoming mail. I have seen some email rejected by spf checking. Last time was a internal contractor that had our domain's emails forwarded to godaddy hosted email. I would never reject mail based on an spf but I do publish if others if their dumb enough to reject mail.

I use it

[Deltaspectre]

Yes, and it's not very effective in the places that matter. My school has recently transitioned to Zimbra, which has been automatically sending anything from any of my domains into the Spam folder. (I also have DKIM set up, but that didn't help. As far as I know my IP isn't on any blacklists, so it should be getting through fine.... )

Re:I use it

[kosmosik]

Where is logic in that?

Two facts:
- you use SPF for own domains
- your shool's Zimbra installation scores mails from your domains as spam

Based on above facts how have you come to conclusion that SPF doesn't work in general? The fact that your school's Zimbra scores your mail as spam is just a single cases and most probably not related to SPF in general.

Have you looked at headers of these message marked as spam? Have you contacted the postmaster?

Re:I use it

[Deltaspectre]

I should have added that this is just my personal experience. I have looked at the headers, but they are gibberish to me that I can't find information on via Google (X-Spam-Track in particular). As far as contacting the postmaster, I received the helpful reply to have all recipients whitelist my domains in their filters.

I use them, but mainly for deniability

[e9th]

My SPF records have gotten me un-blacklisted a few times, after I've pointed out that those machines in Brazil weren't authorized to send email from my domains. But I think DomainKeys, DKIM, etc. will make eventually make SPF unnecessary.

Re:I use them, but mainly for deniability

[MightyMartian]

And yet none of these solutions will actually do very much good at all. This was all hashed over several years ago. SPF, DomainKeys and so forth are little more than feel-good half-measures. If the sole reason you're using any of them is so that Google doesn't reject your email, then I think that's pretty much demonstrated the worthlessness of them.

Re:I use them, but mainly for deniability

[e9th]

I've never had problems with gmail. It's smaller providers/businesses that don't use DK or SPF that have given me problems.

Re:I use them, but mainly for deniability

[WuphonsReach]

DKIM solves a different problem, the two solutions (SPF vs DKIM) are not mutually exclusive.

Re:I use them, but mainly for deniability

[MightyMartian]

Email is never a guaranteed instant delivery anyways, and once a tuplet is in the database, your done. That sender won't be delayed again. I don't even use other spam filters or AV filters on my mail server anymore. Greylisting has pretty much done in well over 90% of that stuff. Since it works at the initial connection level rather than during the delivery phase it's far less resource intensive.

Re:I use them, but mainly for deniability

[GPLHost-Thomas]

The fact that big providers are using DKIM shows that it must do some good to their server, so no, it's NOT useless. Also, as an email service provider, I can show you my server logs clearly showing LOADS of spam blocked by dkimproxy or tumgreyspf (which, by the way, I maintain both in Debian, and for a reason...).

Sure, it's not PERFECT, as emails are still going in. But both SPF and DKIM never claimed to be so. They just wanted to avoid people from doing fraud with domain names that they don't own. And both (especially DKIM if you ask me) are doing this job very well.

I personally see DKIM as the evolution of SPF, which is to my view obsolete.

nope...

[stokessd]

It's winter so there isn't much sun or exposed flesh to worry about. My record for SPF is 50 when I'm bicycling in the noonday sun in the summer.

http://en.wikipedia.org/wiki/Sunscreen

Re:nope...

[caluml]

Typical northern-hemisphere-centredness.

Yes

[S-100]

Yes, I used SPF records on all the domains that I host that have email accounts. SPF records I believe have cut way down on backscatter. Before SPF, accounts would get dozens to hundreds of bounces when their email address was forged as the reply-to address in spam. Now the backscatter is almost completely gone.

But I can tell that Hotmail still ignores SPF since almost all the backscatter that still comes through is from Hotmail. They should know better.

Having valid SPF records also helps outgoing mail get through. I would frequently have to deal with large ISPs that would flag my mail or my domain as a spam source, based on their misinterpretation of forged headers. But since I have SPF records in place, this has not happened. I also check incoming SPF. If the SPF check fails, the mail is dumped. If SPF passes or there's no SPF, it goes through. Works great as one step in spam control.

Hosting Services don't make it easy...

[Jah-Wren Ryel]

SPF has been around for at least a couple of years, but at least one very large hosting provider - hostgator.com - hasn't made it any easier to implement. They still require that you email them and request that they set it up for you.

http://forums.hostgator.com/custom-mx-and-spf-records-t58820.html

Re:Hosting Services don't make it easy...

[drinkypoo]

By contrast, justhost not only has an option to do it for you, but actually detects if your nameservers are THEIR nameservers so that it will actually work. And the tool creates the SPF record and shows it to you, so if you are using another host for DNS you can plug the SPF record in there.

I'm not in love with justhost but they don't have THAT problem.

no

[Uzik2]

and spamhaus put me on the pbl as well. (I don't send spam)

Re:no

[GuruBuckaroo]

Spamhaus didn't put you on the PBL, your ISP did. The PBL is made up of netblocks owned by ISPs who specifically don't want mail coming from those blocks. I use sbl-xbl instead of zen because the PBL has too many "false" positives.

Re:no

[Uzik2]

Thanks. I wasn't aware of some of the politics behind this.

Re:No

[nsayer]

You should provide an authenticated SMTP server for your clients to use. And yes, this can be done even when their ISP blocks port 25 egress - set up an authenticated-only SMTP listener on the Submit port (587). I do this for my home domain, which means that I absolutely get to use "mx -all" for SPF.

Re:No

[julesh]

Most of my smaller clients ISPs do publish SPF, notable clients are running their own outbound.

Maybe now. When I was doing this in 2007, it was nearly impossible.

SPF is working as expected in that scenario. RFC821 FROM should be list@listdomain.tld, broken mail list software that joe-jobs sender@senderdomain.tld is a problem for the list operator.

It doesn't matter if it's working as expected. The expectation that I'm able to get a mailing list operator to change the behaviour of their list software is broken.

Got it on Google's advice

[cerberusss]

Four years ago, I got hit by a Joe-job, i.e. some spammer used my domain in the 'From' field. I deleted the thousands of resulting messages in the following days and then didn't think about it anymore.

Two years ago, I shut down my mail server and moved it to Google Apps. Basically it involves creating a Google Apps account which tells you to point your domain its MX (mail exchange) records to GMail. The second, optional, step was to add SPF records. I thought about the Joe-job. Since the GMail wizard is good and explains everything, I just executed that step. It's actually really simple.

Anyone else have this experience? I.e. creating SPF records was too easy to just skip it?

Re:Got it on Google's advice

[cerberusss]

why does google recommend ~all instead of a hard -all?

I believe it has to do with the fact that some mail providers do not correctly parse SPF records.

When the DNS standard actually will integrate SPF records (instead of the current implementation which uses TXT records), and mail providers use the DNS standard, then the above will probably be a thing of the past.

Re:Got it on Google's advice

[cerberusss]

Well, yes, it does break forwarding. The forwarder will have to switch from forwarding to remailing, where the envelope sender is changed.

Re:Use DomainKeys..

[NormalVisual]

Yahoo, Gmail, MSN/Hotmail, and AOL pretty much require that you have DomainKeys implemented if you want to email their users

I don't have DomainKeys set up, and I've never had any difficulty getting mail to users of any of those services.

Some spam filters score on SPF

[kosmosik]

Some spam filters score on SPF. So not having SPF increases chance of false positives for your legitimate mail when you don't have SPF. And since SPF is free and painless to implement (just few DNS records) I don't see any reason not to use it. Also not like it is something that much significant either.

Re:Some spam filters score on SPF

[wvmarle]

I'm running SpamAssassin on Debian, pretty much in default settings (just set it up, training on the go). It checks SPF and scores a little for failures, no points for matching SPF or missing SPF.

That's the few spams that get past greylisting... which in itself blocks about 90% of my spam before it even reaches me.

Re:Some spam filters score on SPF

[characterZer0]

+1 for greylisting. I implemented in a few weeks ago, and my spam volume dropped by about 80%.

Bless me server, for I have sinned

[ndogg]

It's been, umm, a very long time since I've been to confession.

It's true, I don't use SPF. I've at least got the TXT line in my DNS hosts file.

But I'm using exim, which only has experimental support, and I'm too afraid to use something experimental like that.

What should I do, server?

Re:Bless me server, for I have sinned

[element-o.p.]

What should I do, server?

Move to Postfix ;)

On a completely off-topic, well...topic, I love the "#include" in your sig

Anti-spam vendor's perspective

[gujo-odori]

I work for a major anti-spam vendor.

Yes, SPF records are part of the mix at many anti-spam vendors.

However, they aren't part of reputation. Reputation, to describe it simply and without giving away any secrets, is determined by the kind of mail a host or network emits. Whether it has SPF records and/or DKIM-signs its mail does not affect reputation; if you emit junk, your reputation will be junk.

SPF and (moreso) DKIM have value in assessing whether a given mail is a forgery or not (think phishing and related scams). They are not weighted overly much, since people do foolish things like put their work email address into their webmail account all the time, and it causes FPs, for some value of false positive. That is, it's not an FP per se, but the mail is technically legit, so dropping it on the floor isn't the desired action.

In short, don't expect having SPF and DKIM to improve your deliverability much, if at all. That's not where the value-add is. The value-add is helping to separate the sheep from the goats among mail that purports to be from your domain. If you want recipients to be able to (theoretically, since most of them don't/won't check) have greater confidence that a mail that claims to have come from your organization really did so, then yes, implement both SPF and DKIM.

If you're an organization whose customers might be phishing targets, definitely do both. Orgs I've seen targeted for phishing include financial institutions of any size (even a single branch!), various government agencies, educational institutions (not just universities, either), BBB, auto clubs, World of Warcraft accounts, Vonage, Craig's List, all the free webmail providers. If it has a login, and anything a phisher could find to be of value (for practically any value of "value"), there will be phishing attempts.

If your company is one of those - or even if it's not, really - I recommend both SPF and DKIM.

Re:Anti-spam vendor's perspective

[__aadhrk6380]

I agree. The only point at which SPF or DKIM comes into play is the last few percentage points of filtering and even then other measures can suffice. For instance, I use a Barracuda Spam Firewall and out of the box it catches probably 80% with no false positives. Train the Bayesian filter and pick up easily another 10%. For my use, I can do some TLD blocking without worry such as CN, BR, and RU to name a few and I pick up additional percentage points. A few Regex for things like Viagra and Rolex net me a few more points. Doing a little header blocking for things like XMailer: The Bat and I am down to around 1% spam or less.

Doing things like NOT white-listing your own domain work just as well as SPF or DKIM if you implement quarantining. When you put email handling rules into play like Junk or Spam boxes and allow a per user quarantine with personalized Bayesian settings you can really knock the junk down to virtually nothing.

I think the thing here to take into account is that things like DKIM and SPF are not major solutions to spam. They can help reduce a point or two on percentages depending on your overall configuration, but they are nowhere near global solutions within your enterprise.

Re:Anti-spam vendor's perspective

[Shimmer]

"people do foolish things like put their work email address into their webmail account"

Why do webmail providers allow this? In fact, why would a webmail provider allow you to specify any "From:" address other than the user's actual webmail account?

Re:Anti-spam vendor's perspective

[DNS-and-BIND]

if you emit junk, your reputation will be junk.Yeah, well how do you tell what is junk? I operate a 100% legitimate email list. ALL THE TIME, I get people who click "this is spam" in gmail or hotmail or yahoo. At least 2-5 every email list run. Why? My list isn't spam. It's just that people get tired of the emails, and they would rather not click on "unsubscribe" and instead click on "spam". They volunteered for the emails! And meanwhile yahoo/gmail/hotmail take this as a vote that my email list is spam, and it ends up in new subscribers' spam folder. Yippee.

Re:Anti-spam vendor's perspective

[primus1024]

One thing to note is, is that people are lazy and they easily forget they've subscribed to lists. I don't know what kind of list you have, but i would advise you to put very noticeable unsubscribe at the start of the list instead at the end of it. IE: i get some mails that i know i have subscribed to, but aren't what they were supposed to be. It would be easy to mark them as spam (people react emotionally to disappointment) especially since they hide unsubscribe options with the smallest font possible at the end of the message.
So my suggestion would be: make it easy for people to do the right thing and they'll do it. If you make it difficult people are very resourceful at doing the unexpected.
Another thing to be noted is that people were "trained" to not click unsubscribe option ("don't click unsubscribe or you'll just confirm your mail address as active to spammers" advice we've heard many times).

Again, I don't know what kind of list parent runs, these are just general observations from the lists I encountered.

Re:Anti-spam vendor's perspective

[Barnett]

You said that SPF records "aren't part of reputation", but rather "if you emit junk, your reputation will be junk." But if others are allowed to forge your email addresses, then their junk will be attributed to you, and so affect you reputation. So it sounds to me like SPF can in fact affect reputation. There seems to be a contradiction here...

Better for Sent Items then Received

[Krondor]

I use them, and what I've found is that they have a very marginal effect (if any) on spam catch rates on your inbound mail. However, they do have a great side benefit. They significantly reduce backscatter, keep yourself off of blacklists, and provide some control of you, your employer, or your client's identity on the web. SPF records provide a mechanism to limit who can spoof as you (as long as recipient servers adhere to them). If you have a risk to yourself or interested parties that someone might spoof your domain (banks!), then SPF provides a means to insure the chain of custody (to an extent).

I do think overall SPF has helped to prevent forged domain letters, but those are less and less common (for those that publish spf). The spammers now either rely on forged domains without DKIM or SPF (why not use both!!) or they send from their own controlled botnet domains and publish legit SPF for themselves as well.

dkim; convincing individual mail providers

[bcrowell]

DKIM (formerly known as Domain Keys) is more sophisticated and worth looking into in addition to SPF. I'm using an implementation called DKIMproxy, which runs as a daemon and is specifically designed to work with postfix. I've been fairly happy with it. What's helpful about it is that if I get mail from someone who implements DKIM, I can be sure that it's really from them, and likewise if I get joe-jobbed, I can convince the recipient that the spam wasn't actually from me. The biggest and best known users of DKIM are gmail and yahoo, but I'm seeing it used elsewhere as well. For example, I recently got spam from lulu.com, and the good news was that it was DKIM-signed, so I could be sure it wasn't a joe job.

I understand what you mean about establishing a good reputation in terms of the email you send. Actually many of the big email providers have a policy of blacklisting all domains by default these days, and waiting for the domain operators to contact them and ask to be allowed to send mail to them. Both AOL and yahoo seem to do this. With yahoo, you can fill out a form to convince them you're not evil, and if the info on the form satisfies them, they stop blacklisting you. One of their criteria is that they're more likely to approve you if you implement DKIM. If you tell them you're using DKIM, then they won't accept mail from your domain that isn't DKIM-signed; this is to your advantage, because then their users won't be clicking on the spam button on mail that claims to be from you but isn't.

Re:dkim; convincing individual mail providers

[MillionthMonkey]

With yahoo, you can fill out a form to convince them you're not evil, and if the info on the form satisfies them, they stop blacklisting you.

Your post advocates a
technical (*) legislative ( ) market-based ( ) vigilante ( )
solution... aaah, never mind.

Re:dkim; convincing individual mail providers

[GPLHost-Thomas]

Are you using Debian? If so, can you try my latest package for it, before it's sent to SID and report if it's running well or not? :) http://ftparchive.gplhost.com/debian/pool/lenny/main/d/dkimproxy/

Re:dkim; convincing individual mail providers

[bcrowell]

Are you using Debian? If so, can you try my latest package for it, before it's sent to SID and report if it's running well or not? :)

Hi -- I'd be glad to take a look. First could you drop me an email using the address given at http://www.lightandmatter.com/area4author.html ? I want to make sure we have two-way communication before I put any effort in.

SPF is usless

[DarkOx]

Its not helpful in reducing SPAM unless or until every uses it. Why because you can't toss out mail from domains without SPF records you'd loose to much HAM. You can only uses it to detect and reject spoofs from domains with SPF.

Its not good as an anti spoofing technique in general because there are lots of ways you could make it look like you were sending from the correct host. Possibly in conjunction with DNSSEC (something only being slowly adopted) and some enhancements to BGP you could get there buy SPF alone does not do it.

A public private key scheme on the message bodies would, be much much more secure, and reliable for the anti spoof use.

Sometimes you want to temporarily run your mail out a different IP or relay from another domain, and if you used SPF and your recipients have the dns record cached you are kinda screwed if you need to do anything in a hurry.

SPF is an infective solution at best and really amounts to needless complexity which can only cause problems at worst.

The SPAM and tamper issues are both better solved with message signing.

Re:SPF is usless

[WuphonsReach]

Sometimes you want to temporarily run your mail out a different IP or relay from another domain, and if you used SPF and your recipients have the dns record cached you are kinda screwed if you need to do anything in a hurry.

You solve that issue by running your SPF DNS records with a TTL of about 2 hours (or maybe 4 hours). Even in the freak accident category, I'm hard pressed to come up with a situation where your primary mail server goes up in flames (or the outbound ISP goes up in flames) and you can't both (a) move to a new host/IP range and (b) setup new SPF records in under 4 hours.

And if you can't manage to publish SPF records correctly, then don't. Full stop.

The rest of us will publish them. And manage our email infrastructure in a way that takes SPF records into account. Which means we'll have to deal with less joe-job forgeries then you will over the long run.

Yes

[Kevinv]

I use it on all my domains, and check it on all inbound mail. I especially make sure i define no servers are valid for several domains I have that are web pages only, or use for throwaway e-mail addresses (i receive e-mail at that domain, never send from that domain.)

I do support a domain hosted on google apps and setting it up for that ends up with a less firm ~all option that allows bogus senders to slip through.

I can see SPF fails in my logs so it looks like many other domains are using it as well.

Nope

[menelaus]

I don't use them personally and we have very few customers at my current job that will request them.

I used to work for an anti-spam company and the request would come in from time to time to have SPF checking built into our appliances. As developers, we did see the benefit of it. But at the time, there was the SPF vs SenderID vs Domain Keys battle going on. Who would win out?

As it appears years later, no one really did.

The problem with the technology is adoption rates. Unfortunately, many of these technologies are not being adopted by the masses. I'm not saying its hurting you by having these in place, but it also might not be doing as much good as you think that it is.

Re:Nope

[WuphonsReach]

The problem with the technology is adoption rates. Unfortunately, many of these technologies are not being adopted by the masses. I'm not saying its hurting you by having these in place, but it also might not be doing as much good as you think that it is.

A quick check of mail volume:

151,000 messages checked in the log files that I looked at
58,800 (39%) did not have SPF records ('none')

So 61% of our inbound mail has SPF records that we can test. That is a pretty decent rate of adoption. (Note that we only test SPF for emails that make it past our first set of filters.)

Of the 92,200 messages with SPF records
51,650 (56%) passed their SPF check ('pass')
30,700 (33%) failed (-all)
5,150 (5.6%) resulted in a softfail (~all)
3,100 (3.4%) resulted in 'neutral' (?all)
1,500 (1.6%) permerror (DNS contains an invalid SPF record)

Killing off 33% of those messages with SPF records due to being obvious forgeries is still worth quite a bit. That lets us drop 20% of inbound spam at SMTP time without getting into the really heavy lifting of spam scoring or content analysis.

(We don't block on softfail, neutral or permerror conditions.)

i use it.

[Ruede]

i use spf and DKIM. works fine :) well it is a low traffic domain :)

the bigger issue will be ppl that forward all their shit to other mailboxes... it is always "great" to see lots of it being rejected due "spam"

disable forward - enable pop3 fetch.

Use SPF.

[Anonymous Coward]

If you aren't using SPF, you aren't doing your job. I've become merciless in my dealings with other sites who don't use SPF, or even worse publish incorrect SPF.

Yes

[marquinhocb]

SPF is the way to go. Most public email out there (GMail, Hotmail, Yahoo) will mark email as spam if an email is sent from a server that isn't listed on the SPF record.
Obviously this isn't the only technique to fight spam (You validate that the sender really belongs to X.com, not that X.com isn't a spammer), but it helps.

As for the link to "SPF is harmful", that's about the biggest load of bull I've ever seen. It's inaccurate, and is an uncommon case (how often does mail forwarding happen these days with everyone using non-ISP-bound free email services?). It's like saying we should shutdown the internet because it's not completely accessible to devices with black&white screens.

As I said before, all the major free email service providers take SPF into account (test it out yourself - setup your domain with SPF, and send an email to your gmail/hotmail from an unauthorized IP).

That said, SPF is pretty easy to setup. Just a quick little txt in your domain and you're good to go. This site will help you with generating your SPF:
http://www.openspf.org/Tools

I use it so I don't get blacklisted

[Coolcom]

Before implementing an SPF record, my mail server IP was getting put on all sorts of blacklists. People were sending mail claiming to be from my domain and it was causing me to get on the BLs; all the major ones like CBL, Barracuda, Spamhaus, etc. Anytime anyone would try and send mail to me it would be rejected. I don't use SPF for checking incoming mail, I have the mail checked against a few RBLs and anti-spam software.

They help, but only slightly!

[DNX Blandy]

I also use SPF records for all my domains, most are simply: "v=spf1 a mx -all". "-all" as in hard fail. I don't know why there is a soft fail "~all" option, if it's not from a known host / IP, it should fail. What's the point in returning an unknown response? Like as if there was no SPF record in the first place? It's amazing how many domains actually use soft fail. Anyone know why? They only help stop backscatter and other IPs from sending emails from @youdomain.com as long as the other mail server does a SPF lookup. We have become dependant on the email protocol and the way it works, pitty it's in such a mess :( Damn you SPAMBOTS!!!

Re:They help, but only slightly!

[WuphonsReach]

The point of the ~all was so that you could start testing the waters.

We ran with ~all for a few years, but have recently switched everything over to -all.

So far, I've seen only 1 or 2 false positives where the SPF check failed - even when sent from our own mail servers. I'm guessing that the destination mail server had DNS troubles when it tested our message.

We've also started 5xx (rejecting) at SMTP time if the inbound message fails its SPF check. SPF has been around for long enough at this point, that mail admins who have implemented it *should* have the bugs worked out. (The time for excuses are past, and if we don't reject at SMTP time, the origin mail admin won't know things are broken.)

Re:They help, but only slightly!

[DNX Blandy]

Clearly I do not suffer from "Asperger's Syndrome" lol, and I do actually ask the question as to why it's needed, which the comment below happy provides :)

Re:They help, but only slightly!

[DNX Blandy]

I have multiple mail servers from different companies from which I check the SPF lookups via the logs but, if I was having problems, I would use the soft fail. As yet, I've not had to. I Agree with you, the time for excuses are past.

Sometimes, sometimes not

[93 Escort Wagon]

In the summer I like to use SPF-15 or higher. In the winter it's pretty cloudy around here, so I don't bother.

Yes

[cybersquid]

I do use it on the handful of domains I admin.

Re:Use DomainKeys..

[Curmudgeonlyoldbloke]

MSN/Hotmail's postmaster guidelines don't seem to mention DomainKeys, but do mention SPF:

http://postmaster.hotmail.com/Guidelines.aspx

"4. Authenticate your outbound e-mail: Publish Sender Policy Framework (SPF) records"

Re:Use DomainKeys..

[nacturation]

I don't have DomainKeys set up, and I've never had any difficulty getting mail to users of any of those services.

Does your mail server deliver tens of thousands of messages per hour to those services? If you're talking about the occasional email, you're probably not hitting the threshold at which your delivery will be affected.

Yes! Prevents forged Froms

[bziman]

SPF is great. It's one of the technical means of making sure that the IP address that is trying to send you a message is authorized to use the sender that it claims to be from. That means you can automatically reject spam that claims to be from any of the big mailers.

One common problem right now, is misconfigured mail servers. An e-mail admin configures the SPF entry in DNS, and then forgets about it. Then they change their IP address, or they outsource their e-mail to a third party, and suddenly, SPF is saying that all of their legit mail is not legit. The other problem is when a company has (for example) an order fulfillment system that generates its own e-mails, instead of routing them through the proper mail server. If that system isn't identified in the SPF entry, those messages can be rejected.

Another "problem" is when organizations send messages on behalf of other individuals or organizations (like the legit message that avon.com tried to send me this morning that was being generated by filltek.com, but without the permission of avon.com's SPF entry). I put "problem" in quotes, because really, third party messaging services should not forge the From line of the message.

On the other hand, it's great, in that it blocks all those stupid e-cards, because they claim to be from your.friend@gmail.com, when really they're being sent by stupid-e-card.com.

The biggest problem is dealing with "forwarding" services, like your @acm.org e-mail address. On my server, I have to keep a list of domains that "bypass" SPF checks, because any message sent to a forwarded address is going to arrive at your mail server from the forwarded (i.e. mail.acm.org), but it's going to have the header information associated with the original message. OpenSPF.org talks about some ways to deal with this, but I haven't look at it in a while.

Since SPF is still not universally accepted, it has a "soft fail" option that you can use for testing, until you're sure that it works the way you want it to. It's not the be-all-and-end-all, but it is a useful piece of the puzzle.

Re:Yes! Prevents forged Froms

[A Non-MS Coward]

SPF implementations are not supposed to check the From: header in the email, they are supposed to check the SMTP MAIL FROM address (which you might recognize as the Return-Path: header).

Situations like your avon/filltek scenario, greeting cards, 3rd-party mailers, etc, can still function as intended with proper SPF records, as long as the MAIL FROM SMTP command from the sending MTA software doesn't misrepresent itself. (it should not match the From: header in these cases)

Re:Yes! Prevents forged Froms

[Eric S. Smith]

...3rd-party mailers, etc, can still function as intended with proper SPF records, as long as the MAIL FROM SMTP command from the sending MTA software doesn't misrepresent itself.

They can still function as intended, but the chance that they've been developed by someone who both knows and cares about the difference between the RFC 2821 envelope sender and the RFC 2822 To: line isn't terribly high. The "send an e-mail" function of the library they're using probably doesn't even let them set the two separately. This is no great loss with e-cards, but some "send this story to a friend" features of news sites operate the same way.

Even the forwarding features of real MTAs don't get this right; SPF promoters recommend SRS, but unless your whole business is forwarding mail, you've probably never heard of it.

To answer the article's original question, I use SPF on some of my own domains and implemented it at work years ago. It's not a foolproof barrier to spam or to backscatter, but it mitigates the risk of a joe job.

Now to get sites to read the SPF records.

[Animats]

I have strict "-all" SPF records on all my web sites. But I still get mail bounces from joe-jobs that the recipient host should have rejected during the SMTP session from the spammer.

Yes, but only "neutral"

[adaviel]

Last time I looked, forwarding was a show-stopper. Sender rewriting was complicated, and user's mail client configs would be broken if they used a local SMTP server at home or while travelling.

Reduces 100% "from self" spam and 50% other

[misnohmer]

I've been running a few emails domains of my own for years. Adding SPF checking on the receive end provided me with a few decent benefits:
1. Adding SPF record got rid of backscatter almost completely (a benefit mentioned by another poster already)
2. Adding SPF checking on the receive end got rid of "addressed to self" spam 100% - that was 2-15 emails a day for different mailboxes
3. Rejecting emails per SPF record actually manages to reject over 50% of what would have ended up in my junkmail folder anyways, makes it much easier to scan through junk
4. I setup my server to actually reject the emails with an informative message. This means that if a valid email gets rejected, the sending server (not my server) should send a delivery failure to the sender. Without this that email would have likely ended up in an overcrowded junkmail folder anyways, which means I may have just deleted it - better that the sender knows
5. The SPF result on the receive end is also factored-in by the Intelligent Message Filtering of the Exchange, which assigns a spam likelyhood for spam. I setup a threshold for the spam likelyhood which also rejects the email during the receive, leaving the burden of sending non delivery message to the sender server (so valid servers do it, spam bots don't).
6. Tarpitting also helped a bit for spam rejection, though unrelated to the SPF record.

PS> This is about usefulness of SPF, not about my choice of servers, but if you really want to know I use both Exchange and Postfix to route my mails to appropriate mailboxes. Both have features the other doesn't (e.g. Postfix wildcarding, unlimitted mailboxes, etc | Exchange Blackberry Enterprise Server integration, calendar, contacts, SPF integrated with IMF, OWA, etc).

It's caused us some problems.

[Fross]

I work for an organisation that has a private email system (private as in hardware, network lines). SPF works fine on that, though is also redundant. However, the network is accessible to other networks (ie the internet, as in, people can send mail to regular mail addresses, and vice versa), and SPF breaks here.

Due to the jump to the network, the "sender" is always the provider who handles said connectivity, where our area of the private network touches the internet. Thus we've had to completely disable SPF as it always comes back with negative results.

A good idea in principle, but fails when the two mail servers cannot immediately talk to one another. You'd need something like a validation chain to allow that scenario to work.

Yes & Yes

[Iphtashu Fitz]

Yes, I use SPF to identify the MX's of three domains I own, and Yes I use SPF as one of the things SpamAssassin uses for identifying spam. Granted these domains are tiny in the grand scheme of things (one is for family, one for some shareware I wrote, and one for a non-profit my brother is involved in), but it definitely helps. I wrote a script that sends me monthly stats of spam, and here are the results for the last month:

sa score : 1 messages :299
sa score : 2 messages :194
sa score : 3 messages :235
sa score : 4 messages :299
sa score : 5 messages :477
sa score : 6 messages :597
sa score > 10 messages : 31678
highest sa score = 57

total probable spam (sa score of 5 or more) : 32752
total spam blocked outright by sa : 37110

e-mail blocked via SPF : 3007
Unique IP's that passed SPF check : 1389

We only block spam if the SpamAssassin score is above 10, but we tag anything above 5 as spam so the end users can decide what to do with it. As far as SPF goes, in the last month over 3000 bogus e-mails were dropped due to SPF failures, and 1389 other e-mails that were accepted were approved in part because the domains had SPF records that passed the check.

Re:Yes & Yes

[kjart]

As far as SPF goes, in the last month over 3000 bogus e-mails were dropped due to SPF failures

How do you know those were all bogus?

I've been using SPF for several years...

[dpilot]

...in conjunction with my DynDNS vanity domain. When I first set it up, there was a rush of backscatter, then it tapered off and went away, never to return.

More recently I've started having problems of a different sort. I've been on a certain mailing list for over a year, though not posting very often. Last week I posted to a thread, and got an SPF violation notice from what looks like AOL in Australia, on behalf of someone with 2 apparent domains, neither of which is AOL. The violation notices seem to think that MY mail is originating from an AOL server, so the AOL server is generating an SPF fail. These notices are being generated for only one list subscriber, for every time I post to the list. It looks like a misconfigured AOL server (Would you expect anything else?) to me. Still, that's one aspect of SPF and presumably DKIM - other peoples' misconfigured machines.

I set up SPF and regret doing so

[mi]

I set it up and regret it. First, it broke things for one of my correspondents (at least this one — who bothered to tell me about it), who forwards all e-mails to his cell-phone. Because the messages are forwarded by his e-mail provider, but appear as if from me, his cell-phone service rejects them — because his e-mail provider is not listed in my SPF-record. So, he finds my messages in his mailbox, but is not alerted about them (as he is used to) by his phone...

Then, it turned out, my SPF-record is set slightly incorrectly, which — bizarrely enough — causes outright rejection by many servers. In this respect, people with buggy SPF-records are treated worse, than people without it... This is partially my fault, so this second item is just here for general interest.

And lastly, I am still a victim of "Joe jobs" every once in a while, as spammers send spam with the "From" set to my domain — my having an SPF record is not much of a deterrent, evidently.

Overall, the broken forwarding is, probably, responsible for slow adoption, which, in turn, makes it ineffective for the adopters...

Care about your domain's reputation? Do it anyway.

[harr2969]

If you're in business, or if you care about your domain's reputation, you should be implementing SPF to prevent others from sending mail (aka joe-jobbing) as your domain.

Even if you DON'T care about your reputation, your life will be easier if you don't have to deal with the back-scatter (complaints, threats, invalid postmaster replies, out of office messages, etc) from a massive joe-job/spamming effort which is spoofing your domain.

You CAN make a substantial dent in these types of attacks with SPF. There are levels of SPF "certainty". In order to be most effective, you need to list all your sending servers with a dash "-all" for example, a major financial uses:

text ="v=spf1 ip4:207.162.228.0/24 [shortened] -all"

On the receiving side, most SPF implementations will (and should) respect the certainty of the senders SPF record. In the above example the financial implemented the "-all" qualifier, so if mail comes in from a place not on that list, based on their assertion I can safely drop it as spam. If they used a "?all" or other, I might only increment the spam probability or tag it [maybe spam].

When implementing your DNS SPF record, it can take time to make sure you've identified all the legit sender's of mail with your domain name if you're a large company. Keep at it and come back here and let me know, I'll give you a pat on the virtual back for doing THE RIGHT THING.

http://www.openspf.org/

I have it with -all

[Nuitari The Wiz]

A few people are inconvenienced because they have to connect to a different port then the default due to ISP firewalling.

I would really really like it if more ISPs were checking them and silently discard anything that is flagged as spam _AND_ fails SPF instead of bouncing it back.

We get thousands of bounces addressed to non-existant users, which in turn makes into a double bounce. Of course now I've set our system to silently delete them instead. Else it's just a colossal waste of resources.

Re:Use DomainKeys..

[Antique Geekmeister]

This is nonsense, at least for Gmail. I have no difficulty sending to their domain from unregistered hosts.

SPF is not an anti-spam tool: it's an anti-spoofing tool. It also helps prevent 'backscatter' from certain types of forged spam, the bouncing of forged emails from scattered SMTP servers around the world, because its classic form blocks the message before it is even fully transmitted when the bounce address is published. It still has issues with sites that do email forwarding, since most of them simply repeat the original sender's bounce address, and their forwarding host is not usually authorized to send mail from that bounce address.

And make no mistake. SPF isn't about the "From:" line, it's about the bounce address. This confuses many people who have to deal with it.

Wow, what a bunch of clueless responses

[BlortHorc]

A very restricted SPF TXT record that specifies _precisely_ which IP addresses an incoming SMTP for a given domain an email _must_ come from cannot hurt. At best, your IT admins have wasted a half hour, at best they have significantly improved the chance of your outgoing email being not treated as suspicious by bulk email handlers such as yahoo, gmail and hotmail (especially hotmail).

You want proof? Check your shit. http://postmaster.live.com/Services.aspx . And no, I don't work for MS, but damn they provide the best postmaster tools on the interweb for monitoring shit like email deliverability. Don't even talk to me about the pestilence that is Yahoo!, those pricks remind me of my evil DM who used to make up pointless forms just to pass the time.

Why/What I use SPF for ...

[BitZtream]

SPF serves multiple purposes for me.

Why I add SPF records to our DNS servers:

First and formost, it tells everyone who my mail senders are and that they should only accept mail from my users from those servers. Thats really all it does, but that results in the following things for me:
My remote users always configure their outbound mail server as our gateway, rather than their own ISP or something like that, which means that all that mail piping through me means I can do all sorts of sanity checking on the server for my users. It doesn't force them to use our mail server, but if they do, they'll end up sending to someone at Google or Yahoo who'll reject the message, at which point my user will generally have the problem fixed.

More important however is that it cuts down tremendously on backscatter spam I get.

Why I look for it when receiving mail:

Helps stop our auto-response mail addresses from producing a bunch of backscatter when they get spammed.

Stops scammers who use email addresses from large well known businesses in an attempt to make their messages look more legitimate. Pretty much every major company worth its weight in air publishes SPF records.

Problems:
It seems that services which host frontend stores for things like hotel rooms and travel seem to not understand that the server sending all their confirmation emails should probably be included in the SPF list.

Thats the only thing I've run into, on hotels and ticket purchasing web sites that are managed by 3rd parties, and send messages as if they were actual company. Universal Studios has had this problem with their ticket ordering system for at least the last 3 Halloween Horror nights :( Seen it with a couple hotel sites as well.

Re:Use DomainKeys..

[Jazz-Masta]

I don't have DomainKeys set up, and I've never had any difficulty getting mail to users of any of those services.

Does your mail server deliver tens of thousands of messages per hour to those services? If you're talking about the occasional email, you're probably not hitting the threshold at which your delivery will be affected.

Some of my servers have been flagged at about 100 emails a day to multiple users at yahoo. Hotmail seems to be around the same too.

Yahoo even sends you a message saying they've blocked you and to check out their website for options.

Re:Use DomainKeys..

[Daniel_Staal]

The one I manage at work doesn't have DomainKeys, and it does send tens of thousands of messages an hour to those services. We get deferred by Yahoo regularly, but that's about it.

Reduced Backscatter Significantly

[Anonymous Coward]

I didn't used to use SPF records for my domains. I always felt that it was a pretty half-assed way to prevent forged spam from my domain since it required the admins of other mail servers to enable SPF checking in the first place.

Then I started getting a ton of backscatter in my inbox. Just out of the blue, some spammer had decided to start using my domain name for cialis spam and next thing I know, my inbox is flooded with rejected messages bouncing back to me. I set up SPF as kind of a desperation play more than anything else and the backscatter disappeared almost overnight. I'm sure someone out there is still receiving spam which appears to be sent from my domain, but the volume of backscatter I'm getting isn't even a tenth of what it once was. SPF is good for something.

Re:Reduced Backscatter Significantly

[badger.foo]

I set up SPF as kind of a desperation play more than anything else and the backscatter disappeared almost overnight. I'm sure someone out there is still receiving spam which appears to be sent from my domain, but the volume of backscatter I'm getting isn't even a tenth of what it once was. SPF is good for something.

The end of backscatter is more likely both a temporary thing and an indicator that they moved on to the next domains in their lists. They will be back sooner or later. In the meantime, you could use those backscatter addresses productively for such things as greytrapping (see eg http://www.bsdly.net/~peter/traplist.shtml, featured here at /. at various times). It is worth noting that the domains involved there had valid spf records before those backscatter storms started happening.

We publish and use SPF records

[dskoll]

We both publish and use SPF records. We publish them in an attempt to limit backscatter from joe-jobs, but that's not very successful. Nevertheless, I like the idea of being able to declare which machines are legitimately allowed to send mail for my domain.

We also use SPF records, but in a careful way. We add lots of points for SPF "fail" results from certain domains like paypal.com, ebay.com, etc. We add a moderate number of points for SPF "fails" from domains not in that list. We subtract points for SPF "pass" results from certain trusted domains.

We certainly do not subtract points for SPF "pass" from some random domain; we have no reason to trust it. In fact, for a while, an SPF "pass" result was a mild indicator of spam, as spammers registered throwaway domains and published SPF records.

Graph showing adoption

[kingradar]

I am one of the admins for the free email service Lavabit. We have a graph on the net showing adoption, built from about 150k messages a day. (We don't include messages for users who have disabled this inbound check, or for messages which are blocked for some reason other than SPF.)

http://lauren.lavabit.com/export/graph_162.html

Yes and it helps

[vvaduva]

I use them for all the domains I manage (maybe about 200+ domains) and forged spam has disappeared since. It doesn't take that much time to set it up, so why not do it?

We use postini, and postini doesn't use it.

[zerofoo]

We use Postini, and postini handles the delivery of our mail. We have yet to have any organization block our mail while it is delivered by postini. It seems that most mail admins implicitly trust that postini's servers aren't spewing spam.

As far as postini's position on using SPF to identify spam:

Postini has investigated SPF and has decided not to implement it as a
feature for inbound mail processing. Implementing SPF would add
significant processing overhead without adding any appreciable
effectiveness to the spam filtering. Almost all mail that would be
blocked by SPF are also identified as spam by our spam filters.

In addition, Postini tracks the IP addresses of Fortune 500
corporations and the most popular internet sites such as Yahoo,
Hotmail, eBay, etc. Adding these domains to the Approved Senders list,
particularly at the organization level, is not usually needed and can
result in spam appearing to be sent from those domains inadvertently
getting to users' mailboxes. For this reason, Postini recommends
against using the Approved Senders list in this way; rather, it should
be used only for mail from senders that has previously been falsely
quarantined as spam.

The other reason I have not published and SPF record: Verizon is hosting our DNS services, and when I asked their business services about adding SPF records to my domain the guy on the other end of the telephone had NO idea what I was talking about.

After 3 or 4 call transfers I just gave up.

-ted

Only once ...

[Tux2000]

... in my last job, we had a lot of clients using Microsofts mail services. M$ gave you basically two choices: Implement SPF or have your mails delivered to the spam folder or refused. So, we made our DNS provider add SPF records and the problem was gone.

Tux2000

Kinda

[MBGMorden]

I'm "kinda" using it, in that yes, I setup an SPF record for my domain at work, but I'm not actively checking the SPF records of any incoming mail. I kinda question whether it's of any use at all. I set up our record because it seemed the wise thing to do, but honestly given how many domains don't have SPF records setup I'm not sure ANYONE is actively checking them for incoming mail. Without more usage the system is kinda useless.

As postmaster for a large public university:

[Anonymous Coward]

...No, we do not use or publish SPF records in any way. Spamassassin assigns a score of zero to the SPF_PASS rule for incoming email.

Every 6-12 months or so, someone asks why. The reasons have not changed in the several years that I've been here:

* For a large, non-centralized research institution like ours, any SPF record we could conceivably come up with would have to be permissive to the point of rendering it useless. We have countless departments, nearly a hundred thousand users, and they all have their clients (both on and off campus) configured about a hundred different ways. Some relay through our servers, some through their ISPs (either by choice or due to a heavy-handed ISP filtering policy that restricts outbound SMTP on TCP/25 and/or TCP/587), some through departmental servers, some through sendmail running on their own workstations (damn you, CS faculty). The work necessary to construct, publish, and maintain an SPF record for our purposes has been deemed not worth the effort.

* SPF has been embraced widely by spammers, perhaps more so than legitimate institutions. This alone keeps us from assigning any stock to SPF records when making delivery decisions on our own incoming mail.

We have yet to encounter any notable issues getting our mail delivered to the "big boys" like Hotmail, Google, Yahoo, etc. based on our lack of SPF kool-aid consumption.

Screw SPF and the admins who use it

[synthesizerpatel]

The only times I've come across SPF servers it's been an employee at my company asking why they got an email from a foreign server 'warning' them that because _we_ don't use SPF there's something wrong with _our_ email system.

1. If everyone isn't using it, it's not a standard.

2. Soft-warnings to uneducated people result in busy-work for IT people. Worse yet, try explaining to a marketing person that no, in fact, OUR email system works fine, it's the remote guy's server that's got the issue.

SPF is a good idea in theory and if everyone used it (and it worked properly) it'd be great. But we don't and I won't be switching until a real solution is available that gets adopted by everyone. Sorry folks, email isn't going to be changing drastically any time soon.

If we were smart we'd set everyone to be white-list email only, if an email bounces the sender is told to call you on the phone and get a whitelist one-time-code to get added to your list. Everything else is just going to be statistical attempts at solving this problem which will never be 100% correct. Either spam gets through or important email gets lost.

Re:Setting up DKIM

[WuphonsReach]

We run a multi-step system.

First off, your mail server has to follow RFCs regarding HELOs. That means it has to be a valid HELO name, has to be a FQDN, and it has to map back to an address in a public DNS A/MX record. (We do *not* check beyond that as per the RFCs due to mail servers often being multi-homed and the IP address in the DNS record frequently won't match the IP address of the server that's talking to us.)

The HELO checks are cheap and will block 10-20% of inbound connections. Make sure you have a quick and easy way to whitelist servers which are improperly configured. You'll have to whitelist a few dozen during the first 2-3 months, then it will trickle down to about 1 per month.

Then I'd recommend blocking on a very very good DNSBL (like Spamhaus Zen). We personally choose not to.

After that, SPF checks and A/V filtering *during* SMTP time so that you can 5xx reject and not have to quarantine or deal with creating backscatter bounce messages. If you block using the Zen DNSBL, then you probably won't see many inbound viruses. But it's good to have A/V filtering at this point on inbound anyway. About 61% of our inbound messages have SPF rules that can be checked, and 33% of those get blocked at SMTP time due to "-all" failures.

At this point, we're dealing with maybe 40-50% of the original mail volume that was hitting our server. So now we feed it into a spam filtering system (we use SpamAsassin with amavisd-new). If you use Amavis, setup a 'soft' whitelist where you subtract 2 points for domains that you commonly do business with and subtract 5 points for specific email addresses which tend to score high.

We tag at 5.0 (adding a '[SPAM]' to the start of the subject line along with headers) and we quarantine at 9.0. The quarantine is simply done by moving the message into the user's Junk filter on the IMAP server using server-side Sieve filters. If the user wants to check their junk folder, they can either use webmail (which talks to the server via IMAP) or if they're already using an IMAP mail client, they can simply look in the subscribed Junk folder right in their mail client.

To give you an idea of numbers. Without the HELO checks and SPF checks, I would personally be getting about 6x more spam then I did before we implemented SA. We were killing off 83% of all spam just with HELO and a DNSBL check. With SA in the picture, I only see maybe 1/10th of what's left make it into my inbox. So we're keeping about 98% of all spam from hitting the user's inbox.

I could probably push that to 99%, but the risk of false positives would go up too much.

Outgoing Mail Gets Spammed

[Torrance]

I've just today been reading up on SPF records. My problem is that mail coming from a new server we've set up is being spammed by Big Mail (Gmail, Hotmail, etc.).

The server has a primary domain (domain.com) but many other domains are hosted on it too (eg. mine.com). When I try to send mail with the from field set to me@mine.com (as opposed to me@domain.com) it gets spammed.

We're not on any blacklists, the A records for mine.com point to domain.com as do the MX records, domain.com has PTR records, and I've just put in a SPF record for mine.com.

So far no luck. Sigh.

One Word

[fast turtle]

"Katrina"

Remember that huricane? Nocked lots of companies offline for more then 4 hours and people couldn't change an SPF record if they wanted to due to the evacuations. Another was Loma Prieta in 1991 during the World Series Game. San Francisco, CA 6.9 Earthquake with lots of damage. Another was the St. Helens Eruption in Washington. I could go on but I think you've got the message, which is Natural Disasters. These could affect large areas and result in SPF being absolutely useless because people simply can't transfer their Email service even temporarilly and I'm not talking the Mom and Pop shops, I'm talking places like RacSpace and other Hosts like them. Get a major disaster in a location where some major hosting providers are and you're talking an email meltdown because of SPF only but if combined with DKIM, you might have a chance.

Re:One Word

[atamido]

All of that is taken care of by using a DNS provider with the basic amount of redundancy, ie, two locations. If a meteor takes out your email hoster's one location (which in my mind means you're probably just not going to do email for a bit) and one of your DNS hoster's locations, then you set up another email provider and change the SPF record at the DNS hoster's other location. I can't believe this isn't completely obvious to you.

We use sender-ID

[osssmkatz]

for incoming mail to a college campus with 3,000 students, and I presume outgoing. Nobody complains. (I did have one complaint, and it was weird. Not at all related I don't think, but it was "My mail's not getting through.")

--Sam

Yes

[mysidia]

I publish SPF records.

I administer SurgeMail mail software.

Senders that don't publish SPF records are dodgy sources, and they get an automatic penalty, unless the source IP matches the MX record and listens on port 25.

Automatic +5.0 to spam score. Basically assures the sender will be hit by graylisting, and the message may ultimately be classified spam.

Fail to publish proper SPF records at you and your users' peril.

a bit

[Matthew Weigel]

I use greylisting to reduce spam volume, and I whitelist outgoing mail servers for domains that a) have trouble with greylisting and b) publish SPF records. In other words, I use SPF given existing trust for a particular domain, but only if not relying on SPF causes problems. I thought I hadn't set up SPF records for my own (vanity) domains, but apparently I have... not that I particularly notice. It's just not a big deal.

Re:Use DomainKeys..

[rmm4pi8]

I'll second that. Y! defers everybody, it's just a fact of life. But I send about 2 million emails a day to the big-5 and don't use DKIM. I hope we don't end up having to, b/c it sounds like a real PITA.

spf - seems to reduce forged spam

[Teunis]

I've used this on various servers. A couple had high rates of forged email before and it was reduced after. Mind I also put a lot of other email security in place so it could have been that too. We had hard requirements for validity internally and soft for external - which helped a lot more in tracing down some internal computers that had been compromised.

SPF vs. DKIM/DK

[buss_error]

I run a server farm somewhere between a /14 and a /17.

All authorized mail servers have SPF records. Ranges that clearly have no legitimate business sending email are clearly identified with XXX-XXX-XXX-XXX.dynamic.TLD and listed with SpamHaus's PBL.

No servers have DKIM/DK. The software to do so is opaque, testing is difficult to impossible, and the benefits over SPF are unclear at best, dubious at worst.

On about 1/3 of the servers, all Yahoo email is blocked out of hand due to the disgust and irritation of the server owner over Yahoo!'s blocking/delaying/spam problems. One server owner told me, "My mail TO them is blocked or delayed. But unless I use DKIM/DK, they won't tell me what the problem *is*. Since my own spam load is roughly 40% FROM yahoo!, screw 'em."

Yahoo!'s insistence on DKIM/DK is highly suspect in the cases, like mine, where a responsive, active abuse desk that will address a spam issue if it's from our clearly identifiable ARIN allocation is available.

For those customers that choose not to accept Yahoo email, we return an error message generally worded like so:

"We're sorry, but due to Yahoo! polices we strongly disagree with, we will not accept your email. Please use another email service that doesn't have it's head up it's ass."

It isn't phrased quite so bluntly, but the flavor is still there.

When I get complaints that Yahoo! won't take a customer's email, I tell them, "Yahoo! is a free service. Their customers are getting all they pay for. I'd like to help you, but frankly, I can't get them on the phone or to give a reasonable response via e-mail. Your best bet is to require a contact method that refuses or bypasses Yahoo!. They aren't in the business of giving their customers reliable email service."

Do I have problems? I'm sure I do. But since Yahoo! won't discuss them without jumping through their useless DKIM/DK hoops, I'll just ignore it and move on.

Re:SPF vs. DKIM/DK

[edmin]

The problem isn't understanding DKIM, but rather finding it worth the effort. It might make sense for a bank to protect the content of its messages with a signature, but for spam, phishing, and other high-volume abuse, there are simpler and more effective methods.

SPF is good stuff.

[jafo]

SPF is not an anti-spam measure, it's about preventing hijacking of domains. People often seem to say "but spammers publish SPF records", and that is true, but it doesn't mean that SPF is not effective.

SPF allows me to publish information about what systems will legitimately send e-mail using that domain. It also allows me to act on that information published by other third parties.

What this means is that I have to deal with dramatically less backscatter spam. Since implementing SPF, I have not woken up to find 100,000 messages in my box that were bounces or outraged replies to spam sent by someone else. Back in 1995 that exact issue happened to me, and to a lesser degree it happened regularly until SPF.

There are, of course, some difficulties with SPF, but despite those I have chosen to use and advocate SPF.

You do have to deal with legitimate third-parties sending mail from your domain. We use an outsourced accounting package and have had to include their servers in our SPF records. No big deal.

As a recipient, if you have one account forwarding to another, and the destination account implements SPF, then you either need to white-list the forwarding machine(s), or you need to implement SRS there.

DKIM and it's variants is, IMHO, useless because it only allows you to prove that e-mail came from an authorized sender for a domain, it does *NOT* allow you to tell if e-mail came from an UNAUTHORIZED system for a domain. You cannot use DKIM to tell if a sender address is forging the domain.

So DKIM is *NOT* a "better SPF". They *ARE* compatible though. If you get a message claiming to be from a specific domain which fails the SPF check, you probably still want to allow it if it passes DKIM. I don't know of any mail programs that do that though. The unfortunate thing about this is that SPF-only can be implemented entirely at SMTP time (RECV FROM) where SPF+DKIM would have to be implemented after receiving the message (after DATA).

Sean

Re:SPF is good stuff.

[Eric S. Smith]

DKIM and it's variants is, IMHO, useless because it only allows you to prove that e-mail came from an authorized sender for a domain, it does *NOT* allow you to tell if e-mail came from an UNAUTHORIZED system for a domain. You cannot use DKIM to tell if a sender address is forging the domain.

Someone who really cares about DKIM can check your domain to see if it publishes a key and reject messages that lack a valid signature. I don't see how that's much different from SPF; I do agree that it's not necessarily better.

Yahoo

[NtrlBrnThrila]

Yahoo absolutely uses SPF in conjunction with other methods to authenticate senders with high volumes

I am, but maybe not much longer...

[vyrus128]

I currently use SPF, and am thinking about dropping it. It causes me a massive pain in my ass every time some dumbass with a misconfigured forwarder doesn't understand SPF or SRS, and tries to blame me for the fact that they can't receive email from me. There just aren't enough large sites sending SPF-enabled mail for misconfigured receiving sites to realize they're doin' it wrong.

Just get everyone to use PGP or GPG

[Orion Blastar]

and sign their emails with public keys. That way you can store their public keys on your system to verify it is a valid email.

I really am not sure why PGP or GPG isn't added to Email servers to verify email. Most email clients work with them and if email clients and servers are modified to use PGP or GPG encryption to connect and send out messages and automatically sign them then the servers can verify the sender via the private key and passphrase and lock out the spammers and scammers. Anyone who does send scams or spams can be identified by their signature key which would be required to send an email.

Most scammers and spammers use Botnets that have built in SMTP servers that they can use via remote control to send email and fake the headers and SPF and fake being from a domain that isn't black listed. Not only would you have to change how email clients and servers work, but you'd have to find a way for millions of Microsoft Windows clients to avoid being infected with trojans to be part of a Botnet that fakes SMTP messages.

Yes, but no

[slimjim8094]

I have Dreamhost. They provide a copy and paste line for a DNS entry. See http://wiki.dreamhost.com/SPF

It's one of those things that won't be useful until just about everybody has implemented it. The way it works is by defining which IPs can send email purporting to be from a domain; if you receive an email "from" a yahoo address but coming from some cable modem, you can block it. And as long as not everyone has SPF, you can't just block emails that fail a SPF check...

So yes - I do use it. But it's mostly altruistic, as it really has no effect on incoming email unless you just block no/invalid SPF.

I don't understand why everybody doesn't just enable it - it's a few minutes' worth of a TXT field.

Re:Use DomainKeys..

[danomac]

Yahoo, Gmail, MSN/Hotmail, and AOL pretty much require that you have DomainKeys implemented if you want to email their users

I send mail to these services without DomainKeys. In fact, we started getting bounces from them saying we don't have a SPF record for our domain. I've added one for our domain and haven't had any issues since.

You ARE asking people to throw away your own email

[CritterNYC]

The thing is that due to forwarding and vanity servers, you ARE asking people down the wire (which you can't predict and have NO control over) to throw away mail you sent. When you send to a buddy's vanity domain (that you don't know is a vanity domain) and it forwards your email to their ISP or work account that does check SPF records, your mail gets ditched. And you usually won't get any notice, so your email just disappears.

Using SPF increases the likelihood of your email getting sent into the ether to not return.

The SPF folks themselves acknowledge this as an issue and recommend using SRS to combat it. Of course, since no one uses SRS, it's still an issue.

I do use it, but only to some level.

[KingRobot]

I use SPF, but only to block known or likely forgeries. That is, an e-mail that claims to be from somehost.com AND somehost.com has SPF records, but the message fails the SPF test. In this case, either the message is forged, or the admin of somehost.com hasn't properly educated his users. I also block a very limited case of messages that claim to be from our own domain, but are in fact spoofed. It has to do with a mismatch of the From: header and the MAIL FROM response. You can read more about it here: http://www.eisbox.net/2009/07/23/2395-mail-from-vs-from-vs-sender-exploiting-spf/

Re:I do use it, but only to some level.

[TommydCat]

Wouldn't implementing this stop most external mailing lists from sending one of your user's email back to himself (or worse, that email from your user to another user of your system)?

My system has been receiving a bunch of spam as described looking like it comes from domain but the envelope is external. Several users (including myself) are active on various mailing lists, so it seems I have my hands tied unless I construct a white list - which would be high maintenance and piss users off until it hit critical mass.

Yahoo's 17 Questions

[Newt-dog]

I've had this floating around on my hard drive for quite a while, and have used it on occasion. Not sure if Yahoo still sends it out, but you get the drift from their questions....

Yahoo's 17 Questions

1. Do you rent, lease, buy or otherwise obtain email lists from companies, individuals, organizations, or websites (other than those you own) that do not indicate that the customer will be subscribed to this specific email list?
a. If yes, do you explicitly send an opt-in confirmation email to the email addresses you have acquired?
-- If yes, please send a text-only example of this email.
b. If no, please explain how you obtain email addresses.

2. How do you verify that the true owner of the email address you have obtained is valid?

3. Do you offer list management services for other companies (i.e., as an ASP)? If so, please provide us with your standards for accepting your clients' email lists.

4. Do you rent, lease, sell, or otherwise give email lists to other companies, individuals, organizations, or affiliates without providing notice to the email users that they will be subscribed to the buyer's specific email list?

5. Please indicate the information below pertaining to email sent to Yahoo! mail.
a. How frequently do you send email to Yahoo! users in a given month and how many emails are sent in the average mailing?
b. If you send email to multiple addresses, how many addresses are sent to, for an average mailing?
c. If you are an ASP, what has your average client mailing frequency been over the past six months?
d. Are you emails informational and subscriber based (newsletters)?
e. Are your emails for marketing to other than existing customers?

6. Please specify your policies pertaining to both soft (4xx) and hard (5xx) SMTP response codes or bounce messages.
a. Do you remove email addresses from your mail server or list if emails to them bounce?
Soft:
Hard:
b. How many bounced emails are required before you consider an email address to be inactive and subject to removal from your list?
Soft:
Hard:
-After an email address reaches your bounce limit, how long (i.e., minutes, hours, etc.) does it typically take to remove the email address from your list?
Soft:
Hard:
c. Are there any circumstances under which you ignore the standard definitions (4xx) being temporary and (5xx) being permanent, and instead apply your own non standard interpretation? If so, when/what/how?

7. If a user requests removal from your email list, how long (i.e., minutes, hours, etc.) does it typically take to remove the email address?
When user clicks an unsubscribe link (if applicable):
When user requests removal:
Other:

8. If a user is removed from your email list, what happens to that email address in your database?

9. Please copy and paste a text-only example of a recent mailing, having the delivery issue, including full Internet headers. Include the entire error message if email is being returned or undeliverable.

Within a Yahoo! Mail account, you can display this information by clicking the "Full Headers" link located within the message in the bottom right-hand corner.

10. Please provide all of the active email IP address(es) and domain names you are currently using to send your mailings including notes with regards to dedicated or shared status for each. We do request email administrators to describe which of their clients corresponds to each IP address. Please submit this information in the following format:

IP Address: xxx.xxx.xxx.xxx
Mail Server Domain Name: server_name.domain.com
Notes: dedicated IP, domain/list server
At this time we can only consider active and correctly configured mail servers/IPs for possible addition to the whitelist.

11. Are these IP addresses dedicated solely for your company's mai

No, we don't use SPF

[Lazy Jones]

... the last 3 times we attempted to, we had too many recipients rejecting our e-mails due to broken forwards.

Yes I am

[frambris]

Yes it does

Use it if you want trouble

[trendzetter]

If your mail does not arrive your supposted to get a helpful answer here: www.openspf.org/Why
It's been down for ages.

No

[julesh]

I tried to implement SPF for our clients not long after the spec was released, but found two problems with it:

* Most of our clients send e-mail through their ISP's MXs. Most of the ISPs wouldn't provide an SPF record with a list of all their outbound MXs that I could refer to.
* Even when I was able to set it up because the ISP was cooperative, the client often ended up complaining that messages they sent to mailing lists were being rejected.

Because of these two problems I wrote the system off as useless. The first is an administrative hassle that I cold do without, but the second produces false positive swhich is an absolute fail, IMO.

See NANOG archives

[lorand]

There was a recent thread on the topic on the NANOG mainling list, see http://www.merit.edu/mail.archives/nanog/msg02874.html

Re:Use DomainKeys..

[Kakao]

How lucky you are. I have both SPF and DomainKeys implemented and one of my domains has plenty of problems with Hotmail and Yahoo. I send no lists. Never sent anything with a resemblance of spam. Only email confirmation emails. My IP is clean everywhere.

I would but .....

[DrSkwid]

http://www.openspf.org/ has been down all week and it has all the instructions

3 Install postfix-policyd-spf-perl

Next we download postfix-policyd-spf-perl from http://www.openspf.org/Software to the /usr/src/ directory and install it to the /usr/lib/postfix/ directory like this:

Re:Use DomainKeys..

[characterZer0]

I do not use SPF or DKIM. Many of the addresses on my server simply forward to other addresses, and SRS is way over the head of most of the recipients.

I send very low volumes - fewer than 100 messages per month.

Gmail delivers all of them.

Yahoo and AOL mark some of them as spam until the recipients add me to their contact list.

MSN/Hotmail, in violation of the SMTP RFC, accepts and promptly deletes them with no notification to the sender or recipient.

Re:Use DomainKeys..

[Lost Race]

Same here. Yahoo is a fucking disaster.

use public key infrastructure

[viralMeme]

Use Public key infrastructure, register an email with a public directory. If the email sent from that address doesn't match the digital signature, then reject it - case closed ...

Do penance

[Chemisor]

The server will forgive your sins, my son, if you say three Hail Linuses.

Not much

[bussdriver]

The SPF headers in emails could help with the spam filter - I have mail rules that use the spam rank # in the header (from the server) combined with the SPF to mark emails as junk. This helped reduce a lot of spam: raise the bar for poor SPF results. I also filter spam from my own server using SPF, this cuts down on SOME spam; however, for some technical reasons I can't have a strict SPF record on all my domains and some spammers exploit that so I didn't remove all of it.

Also, I've noticed a huge majority of emails with the SPF header containing: "Maximum nesting level exceeded" and I've not noticed a single legit email with it. I just filter those out now.

When I momentarily tried stricter SPF policies I noticed all of the spammers who get past the normal filtering were using SPF; probably the 1st people to use SPF were the spammers. You know, SPF includes a "loose" mode where you essentially say: these are my mail servers but I might break from it. Strict SPF use stops spam (outside DNS spoofing) but I rarely see strict SPF and everybody would have to use it along with blacklists to stop spam. So many servers lack SPF completely that I couldn't ever require it.

Although, spammers adapt to any popular trick quickly - its their job to do so.

My wish is that more software use the X-Spam email headers... Would be nice if clients would indicate spam rank as well. So I still see the low chance spam but it is color coded (by degree) and the high chance spam is auto filed out of sight.

Re:Not much

[DavidTC]

I don't like idea of the the X-Spam header showing up in clients, because spammers will just start adding that. Like, uh, they have.

If we're going to do that, we need to define an actual header that you're not allowed to send in mail messages, that the receiving mail server has to strip out. (And optionally add their own.)

Actually, what we might want to do is add a header that lists which other headers were added by the receiving mail server. Or just to mark a place above which new recipient headers are added.

Re:Not much

[Anonymous Coward]

I never thought the purpose of X-Spam was actually for email - but a means for server programs and client programs to communicate by attaching data to the email in a standards based way.

If you were going to add a feature to extend email would you redo the protocol or data format or would you use the existing header format?

A spam service could use this to help your mail server and you'd have to add an exception-- although you'd already have to configure that service into the process so this would just be implicit or another configuration option.

I admit I stopped before I got into trying to make postfix rewrite headers or validate the Received headers which indicate the servers (that made note of it) or play with any other header ideas for that matter.

The problem with Received headers is they can be spoofed which is ALSO a problem with delimiters for new header additions (and even if these 'marks' were useful they would require a change in server behavior everywhere to work while using a header attribute is not position dependent and therefore widely compatible.)

Re:Not much

[DavidTC]

The problem if it's a standard client feature than the second in shows up in clients, spammers will start adding it to their email. (As, in fact, they already do.)

So if everyone client is reading it, receiving mail servers need to strip existing ones even if they themselves aren't writing one. Otherwise, you just invented a way to trick mail clients.

However, mail servers are not supposed to strip X-Spam headers. They aren't, in fact, supposed to alter any header they're not specifically supposed to alter.

Ergo, to actually do this, you'd want to create a standard saying a certain header SHOULD be stripped on receipt. Which, incidentally, some already are, and replaced, like Delivered-To and Return-Path. (I don't know of any that get stripped even if not being replaced, but there's no reason why not.)

And no standard header is called X- anything. If defined, it would get an actual name, along with other rules, like whether or not you can have multiple ones.

Or, as I said, an interesting idea might be to make a header called 'Recipient-Headers', which lists trusted headers that were added by the local mail server.

This would, of course, require the local mail server to strip that out, but you'd only need to worry about adding that standard requirement once.

And, from then on, if you want to add a X-Spam header, or a X-Attachment-Saved-To header, or whatever, you can simply stick the name of that header in the Recipient-Headers line, which asserts the mail server stripped out any instances of that header to start with, and thus anything in it is trusted.

And any client would only trust X-Spam if there was a reference to X-Spam in the Recipient-Headers header.

As for the delimiter, I was actually saying that because there is a current way to see what headers were added locally...they'll be before the the Received line added by your local mail server. Lines are added to mail at the start.

The problem is that, for example, you will end up with something like, for the top of the email:

Return-Path: <sender@example.net>
X-Original-To: david@example.com
Delivered-To: david@example.com
Received: from localhost (localhost.localdomain [127.0.0.1]) by example.com (Postfix) with ESMTP id E9576738090 for <david@example.com>; Fri, 18 Dec 2009 19:57:35 -0800 (PST)
X-Virus-Scanned: amavisd-new at example.com
Received: from example.com ([127.0.0.1]) by localhost (example.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N2pKNQAv+HqD for <david@example.com>; Fri, 18 Dec 2009 19:57:34 -0800 (PST)
Received: from lrcmmta08-srv.example.net (lrcmmta08-srv.example.net [10.0.0.1]) by example.com (Postfix) with ESMTP for <david@example.com>; Fri, 18 Dec 2009 19:57:33 -0800 (PST)
...other Received headers, and then the other email headers.

There's a received header after my local server does a virus scan. (Which is also where it would add an X-Spam header if I wanted that...I don't actually want the virus scan header either, I should get rid of that.) Also, in theory, there could be a local added header before the second received one. The bottommost header I showed is where it actually enters my control.

So an actual simple solution to this would be for all local mail servers to simply add 'X-Received-Locally' or something like that, at the top of the headers, the second they get an email, and then not add anything after that. Then we simply believe the topmost one of those headers. And, what's more, you could do this without any changes to mail servers...essentially all mail servers can add custom headers, and they always add them at the top.

Re:Use DomainKeys..

[nacturation]

Our Mail Servers don't use DK/DKIM and have no problem sending tens of thousands of messages an hour to the Big 5. But then again, we have setup Loopback/Feedback with the Big 5 and are Whitelisted with them. All it takes is having your ABUSE@ accounts operational (and monitored) and filling out a single Web Form for each. It's not Rocket Science, and it is far easier to do than to setup DKIM for thousands of domains.

It definitely helps, but I've been involved in operations which were responsible for sending out newsletters for major brands (think Apple, Nike, etc.) and despite DKIM, SPF, ISP whitelisting, and signing up for their feedback loops, there are occasionally still issues. Especially where you could be delivering sometimes several million an hour at peak times, with the majority being to the big providers. Exceed a certain complaint threshold and an entire /24 can be jeopardized. Some people "report spam" for things they no longer want, rather than just for the "grow your wand" type emails.

Re:I Cannot, Really

[nsayer]

You should set up an authenticated-only SMTP listener on the submit port (587). See RFC-2476.

Re:SPF is Bad

[nsayer]

Port 25 egress blocking is no excuse. See RFC-2476.

Subzones of com, net and org using SPF

[huskymo]

Our annual survey of the Internet's DNS infrastructure measures (among other things) the percentage of a random sample of com, net and org subzones that use SPF. In the October 2009 survey, we found that 12.2% of the sample used SPF records. For more information, see http://dns.measurement-factory.com/surveys/200910.html

SPF, DKIM, Best Practices, none of it matters...

[TFGeditor]

...if a handful of AOL users flag your email as spam, AOL will not whitelist your server. This includes double-opt in email sent to verify registration for a newsletter or service. I swear to Bob, AOL use will mark these as spam then complain because they cannot register for your site.

We now simply tell potential registrants who enter an AOL address, "Sorry, get a real email service then we will talk."

The way to stop all backscatter

[edmin]

is add an authentication code to the return address on every message you send out. If you get a bounce to an address without the code, you know it was forged.

Re:Fake bounces from Hotmail

[edmin]

No doubt you thought of this, but is it possible the bounces are fake - i.e. not really from a Hotmail transmitter? Could also be that Microsoft really is screwing up their SPF checks, and the reason you are seeing more bogus bounces from them than everyone else, is that you have a specific spammer targeting your domain and not others.

Re:You ARE asking people to throw away your own em

[Nerd of all trades]

Personally, I don't mind it at all. If a remote email address isn't set up correctly to receive emails with proper SPF records, it is their problem to deal with, not mine. My responsibility regarding email only extends to the point where the remote server accepts the email. Once that is done, it is their problem if they can't relay it properly.

SPF Almost Eliminates Backscatter

[herbierobinson]

I started using SPF because the backscatter from spammers forging my domain was getting to be 5-10 times more than the amount of spam I was getting. The backscatter stopped almost completely and it stopped immediately. Every once in a while I get a small burst of backscatter, but it doesn't last long.

I don't know this for sure, but I suspect that the spammers are checking for SPF before using a domain for forgeries. It would make sense, because using a domain with SPF records for spam makes it possible for anybody to determine it's spam. In particular, if any tier one suppliers are using SPF combined with mail volume to identify spam, they could spot the spam almost instantly -- no wait for complaints to come in. In particular, the spam could be spotted quickly enough to shut down the sender. It probably doesn't happen that much, but if one was sending spam, why would one forge a domain with an SPF record when there are so many others out there with no SPF record.

SPF; You know something better?

[WhiteWiz]

We all know that SPF is an imperfect patch to an old, old system; SMTP We can't make a change that fixes spam but breaks SMTP so we are left with opt-in fixes or a total re-write (ala Google Wave) I have implemented SPF because for now it's all we have.

Yes, I have, and it cut down spam by 90+ %.

[Doctor O]

See subject. Nuff said.