Slashdot Mirror
Are You Using SPF Records?
gravyface writes "I've been setting up proper Sender Policy Framework records for all my clients for past year or so, hoping to either maintain or improve their 'reputation' in the email universe. However, there's a lot of IT admins I speak with who either haven't heard of SPF records or haven't bothered setting them up. How many of you are using SPF records for your mail domains? Does it help? How many anti-spam vendors out there use SPF records as part of their 'scorecard'?"
it has cut down tremendously on the spam claiming to be from my domains.
any other benefit I am unaware of.
If I could walk that way I wouldnt need cologne.
That's some nice FUD you've got there. There are some valid points, but the basis for the page is inaccurate. SPF is really meant to be used in conjunction with some other technology like DKIM. SPF is just meant to ensure that one aspect of the process, that the machine sending the email is allowed to do so. DKIM for instance is meant to verify the contents of the email hasn't been tampered with.
Unless something has changed in the last few years, DKID doesn't verify that the server is allowed to send email for that particular domain, just that the email itself wasn't tampered with which has its own issues. For instance, while it does verify that the message hasn't been tampered with, it does not help people that are sending legitimate cold emails to a server of say the sales rep for the company.
My point here being that any technology has issues when one tries to use it in a way for which it isn't designed, but saying that because it can be used improperly that it's therefore dangerous is an argument which lacks merit.
My SPF records have gotten me un-blacklisted a few times, after I've pointed out that those machines in Brazil weren't authorized to send email from my domains. But I think DomainKeys, DKIM, etc. will make eventually make SPF unnecessary.
Of course, people to whom email is just another way to make a quick buck may have different ideas.
It's winter so there isn't much sun or exposed flesh to worry about. My record for SPF is 50 when I'm bicycling in the noonday sun in the summer.
http://en.wikipedia.org/wiki/Sunscreen
Yes, I used SPF records on all the domains that I host that have email accounts. SPF records I believe have cut way down on backscatter. Before SPF, accounts would get dozens to hundreds of bounces when their email address was forged as the reply-to address in spam. Now the backscatter is almost completely gone.
But I can tell that Hotmail still ignores SPF since almost all the backscatter that still comes through is from Hotmail. They should know better.
Having valid SPF records also helps outgoing mail get through. I would frequently have to deal with large ISPs that would flag my mail or my domain as a spam source, based on their misinterpretation of forged headers. But since I have SPF records in place, this has not happened. I also check incoming SPF. If the SPF check fails, the mail is dumped. If SPF passes or there's no SPF, it goes through. Works great as one step in spam control.
Four years ago, I got hit by a Joe-job, i.e. some spammer used my domain in the 'From' field. I deleted the thousands of resulting messages in the following days and then didn't think about it anymore.
Two years ago, I shut down my mail server and moved it to Google Apps. Basically it involves creating a Google Apps account which tells you to point your domain its MX (mail exchange) records to GMail. The second, optional, step was to add SPF records. I thought about the Joe-job. Since the GMail wizard is good and explains everything, I just executed that step. It's actually really simple.
Anyone else have this experience? I.e. creating SPF records was too easy to just skip it?
8 of 13 people found this answer helpful. Did you?
Yahoo, Gmail, MSN/Hotmail, and AOL pretty much require that you have DomainKeys implemented if you want to email their users
I don't have DomainKeys set up, and I've never had any difficulty getting mail to users of any of those services.
Please stand clear of the doors, por favor mantenganse alejado de las puertas
Some spam filters score on SPF. So not having SPF increases chance of false positives for your legitimate mail when you don't have SPF. And since SPF is free and painless to implement (just few DNS records) I don't see any reason not to use it. Also not like it is something that much significant either.
It's been, umm, a very long time since I've been to confession.
It's true, I don't use SPF. I've at least got the TXT line in my DNS hosts file.
But I'm using exim, which only has experimental support, and I'm too afraid to use something experimental like that.
What should I do, server?
// file: mice.h
#include "frickin_lasers.h"
I work for a major anti-spam vendor.
Yes, SPF records are part of the mix at many anti-spam vendors.
However, they aren't part of reputation. Reputation, to describe it simply and without giving away any secrets, is determined by the kind of mail a host or network emits. Whether it has SPF records and/or DKIM-signs its mail does not affect reputation; if you emit junk, your reputation will be junk.
SPF and (moreso) DKIM have value in assessing whether a given mail is a forgery or not (think phishing and related scams). They are not weighted overly much, since people do foolish things like put their work email address into their webmail account all the time, and it causes FPs, for some value of false positive. That is, it's not an FP per se, but the mail is technically legit, so dropping it on the floor isn't the desired action.
In short, don't expect having SPF and DKIM to improve your deliverability much, if at all. That's not where the value-add is. The value-add is helping to separate the sheep from the goats among mail that purports to be from your domain. If you want recipients to be able to (theoretically, since most of them don't/won't check) have greater confidence that a mail that claims to have come from your organization really did so, then yes, implement both SPF and DKIM.
If you're an organization whose customers might be phishing targets, definitely do both. Orgs I've seen targeted for phishing include financial institutions of any size (even a single branch!), various government agencies, educational institutions (not just universities, either), BBB, auto clubs, World of Warcraft accounts, Vonage, Craig's List, all the free webmail providers. If it has a login, and anything a phisher could find to be of value (for practically any value of "value"), there will be phishing attempts.
If your company is one of those - or even if it's not, really - I recommend both SPF and DKIM.
I use them, and what I've found is that they have a very marginal effect (if any) on spam catch rates on your inbound mail. However, they do have a great side benefit. They significantly reduce backscatter, keep yourself off of blacklists, and provide some control of you, your employer, or your client's identity on the web. SPF records provide a mechanism to limit who can spoof as you (as long as recipient servers adhere to them). If you have a risk to yourself or interested parties that someone might spoof your domain (banks!), then SPF provides a means to insure the chain of custody (to an extent).
I do think overall SPF has helped to prevent forged domain letters, but those are less and less common (for those that publish spf). The spammers now either rely on forged domains without DKIM or SPF (why not use both!!) or they send from their own controlled botnet domains and publish legit SPF for themselves as well.
DKIM (formerly known as Domain Keys) is more sophisticated and worth looking into in addition to SPF. I'm using an implementation called DKIMproxy, which runs as a daemon and is specifically designed to work with postfix. I've been fairly happy with it. What's helpful about it is that if I get mail from someone who implements DKIM, I can be sure that it's really from them, and likewise if I get joe-jobbed, I can convince the recipient that the spam wasn't actually from me. The biggest and best known users of DKIM are gmail and yahoo, but I'm seeing it used elsewhere as well. For example, I recently got spam from lulu.com, and the good news was that it was DKIM-signed, so I could be sure it wasn't a joe job.
I understand what you mean about establishing a good reputation in terms of the email you send. Actually many of the big email providers have a policy of blacklisting all domains by default these days, and waiting for the domain operators to contact them and ask to be allowed to send mail to them. Both AOL and yahoo seem to do this. With yahoo, you can fill out a form to convince them you're not evil, and if the info on the form satisfies them, they stop blacklisting you. One of their criteria is that they're more likely to approve you if you implement DKIM. If you tell them you're using DKIM, then they won't accept mail from your domain that isn't DKIM-signed; this is to your advantage, because then their users won't be clicking on the spam button on mail that claims to be from you but isn't.
Find free books.
Where is logic in that?
Two facts:
- you use SPF for own domains
- your shool's Zimbra installation scores mails from your domains as spam
Based on above facts how have you come to conclusion that SPF doesn't work in general? The fact that your school's Zimbra scores your mail as spam is just a single cases and most probably not related to SPF in general.
Have you looked at headers of these message marked as spam? Have you contacted the postmaster?
I don't use them personally and we have very few customers at my current job that will request them.
I used to work for an anti-spam company and the request would come in from time to time to have SPF checking built into our appliances. As developers, we did see the benefit of it. But at the time, there was the SPF vs SenderID vs Domain Keys battle going on. Who would win out?
As it appears years later, no one really did.
The problem with the technology is adoption rates. Unfortunately, many of these technologies are not being adopted by the masses. I'm not saying its hurting you by having these in place, but it also might not be doing as much good as you think that it is.
Actually, DKIM can be used to guarantee a sender. We're using DKIM here with ADSP. That is:
_adsp._domainkey TXT "DKIM=ALL"
tells a receiver that all emails from our domains should be signed. Since the keys themselves are published in our DNS, a machine not under our control should not be able to send an email purporting to be from our domain.
I'm not sure but I would think that mechanism would make SPF irrelavent. Assuming antispam software actually checked the adsp dkim records.
His protest is without teeth. If he really objects to the concept of SPF, then he should publish an SPF record of "?ALL". That way, people will know he's just not being apathetic.
I also use SPF records for all my domains, most are simply: "v=spf1 a mx -all". "-all" as in hard fail. I don't know why there is a soft fail "~all" option, if it's not from a known host / IP, it should fail. What's the point in returning an unknown response? Like as if there was no SPF record in the first place? It's amazing how many domains actually use soft fail. Anyone know why? They only help stop backscatter and other IPs from sending emails from @youdomain.com as long as the other mail server does a SPF lookup. We have become dependant on the email protocol and the way it works, pitty it's in such a mess :( Damn you SPAMBOTS!!!
Pretty much the same here - SPF records aren't particularly hard to implement, after all. On the receiving side, I just check for SPF failure (i.e. somebody e-mailing from somewhere other than the domain's SPF-registered mail server), and even those just get sent to users' junk mail folders. I'm certainly not bouncing anything because of them. Based on my mail server reports, it looks like the low SPF filtering is catching about 0.5% of the mail volume that flows my direction, which isn't much, but it's 0.5% less than I would be dealing with otherwise and was implemented "for free", so I'm not complaining.
How would that work with trusted partners who may send mail on your behalf? With SPF I can use an include:xxx to define relationships with other systems. With DKIM it seems I would need the partnered system to stamp the sent mail or relay off of our originating servers for DKIM attribute addition (something that might not always be possible). Is there an elegant workaround?
Spamhaus didn't put you on the PBL, your ISP did. The PBL is made up of netblocks owned by ISPs who specifically don't want mail coming from those blocks. I use sbl-xbl instead of zen because the PBL has too many "false" positives.
Poor means hoping the toothache goes away.
I don't have DomainKeys set up, and I've never had any difficulty getting mail to users of any of those services.
Does your mail server deliver tens of thousands of messages per hour to those services? If you're talking about the occasional email, you're probably not hitting the threshold at which your delivery will be affected.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
SPF is great. It's one of the technical means of making sure that the IP address that is trying to send you a message is authorized to use the sender that it claims to be from. That means you can automatically reject spam that claims to be from any of the big mailers.
One common problem right now, is misconfigured mail servers. An e-mail admin configures the SPF entry in DNS, and then forgets about it. Then they change their IP address, or they outsource their e-mail to a third party, and suddenly, SPF is saying that all of their legit mail is not legit. The other problem is when a company has (for example) an order fulfillment system that generates its own e-mails, instead of routing them through the proper mail server. If that system isn't identified in the SPF entry, those messages can be rejected.
Another "problem" is when organizations send messages on behalf of other individuals or organizations (like the legit message that avon.com tried to send me this morning that was being generated by filltek.com, but without the permission of avon.com's SPF entry). I put "problem" in quotes, because really, third party messaging services should not forge the From line of the message.
On the other hand, it's great, in that it blocks all those stupid e-cards, because they claim to be from your.friend@gmail.com, when really they're being sent by stupid-e-card.com.
The biggest problem is dealing with "forwarding" services, like your @acm.org e-mail address. On my server, I have to keep a list of domains that "bypass" SPF checks, because any message sent to a forwarded address is going to arrive at your mail server from the forwarded (i.e. mail.acm.org), but it's going to have the header information associated with the original message. OpenSPF.org talks about some ways to deal with this, but I haven't look at it in a while.
Since SPF is still not universally accepted, it has a "soft fail" option that you can use for testing, until you're sure that it works the way you want it to. It's not the be-all-and-end-all, but it is a useful piece of the puzzle.
Not just to add a 'me too' but I recently removed SPF completely - mostly because other people couldn't get their entries correct, or just completely failed to update it when they add in extra servers. Legitimate messages were hitting our spam folders. Since I can't train our fine worker drones to actually look in their spam, I opted just to remove it. With greylisting and spamassassin its removal hasn't made any noticeable difference aside from the false positives now being delivered properly.
Yes, I use SPF to identify the MX's of three domains I own, and Yes I use SPF as one of the things SpamAssassin uses for identifying spam. Granted these domains are tiny in the grand scheme of things (one is for family, one for some shareware I wrote, and one for a non-profit my brother is involved in), but it definitely helps. I wrote a script that sends me monthly stats of spam, and here are the results for the last month:
sa score : 1 messages :299 :194 :235 :299 :477 :597
sa score : 2 messages
sa score : 3 messages
sa score : 4 messages
sa score : 5 messages
sa score : 6 messages
sa score > 10 messages : 31678
highest sa score = 57
total probable spam (sa score of 5 or more) : 32752
total spam blocked outright by sa : 37110
e-mail blocked via SPF : 3007
Unique IP's that passed SPF check : 1389
We only block spam if the SpamAssassin score is above 10, but we tag anything above 5 as spam so the end users can decide what to do with it. As far as SPF goes, in the last month over 3000 bogus e-mails were dropped due to SPF failures, and 1389 other e-mails that were accepted were approved in part because the domains had SPF records that passed the check.
While I can see the reason for your points, I don't agree with the conclusion.
The fact is that implementing SPF comes with a bigger responsibility to account for every machine your email might be sent from. No doubt this can be a big pain and imply compromise.
I have implemented SPF and for me the big downside is that OTHERS don't check it and pay attention to what I've implemented. I still get bounced email which the receiver SHOULD have ignored as forged because it failed SPF checks, but they didn't and bounced it to me anyway.
So my complaint is that being responsible hasn't bought me as much as it should have.
I'll second that. Y! defers everybody, it's just a fact of life. But I send about 2 million emails a day to the big-5 and don't use DKIM. I hope we don't end up having to, b/c it sounds like a real PITA.
U.S. War Crimes blog. Email for free Mandriva support.
I run a server farm somewhere between a /14 and a /17.
All authorized mail servers have SPF records. Ranges that clearly have no legitimate business sending email are clearly identified with XXX-XXX-XXX-XXX.dynamic.TLD and listed with SpamHaus's PBL.
No servers have DKIM/DK. The software to do so is opaque, testing is difficult to impossible, and the benefits over SPF are unclear at best, dubious at worst.
On about 1/3 of the servers, all Yahoo email is blocked out of hand due to the disgust and irritation of the server owner over Yahoo!'s blocking/delaying/spam problems. One server owner told me, "My mail TO them is blocked or delayed. But unless I use DKIM/DK, they won't tell me what the problem *is*. Since my own spam load is roughly 40% FROM yahoo!, screw 'em."
Yahoo!'s insistence on DKIM/DK is highly suspect in the cases, like mine, where a responsive, active abuse desk that will address a spam issue if it's from our clearly identifiable ARIN allocation is available.
For those customers that choose not to accept Yahoo email, we return an error message generally worded like so:
"We're sorry, but due to Yahoo! polices we strongly disagree with, we will not accept your email. Please use another email service that doesn't have it's head up it's ass."
It isn't phrased quite so bluntly, but the flavor is still there.
When I get complaints that Yahoo! won't take a customer's email, I tell them, "Yahoo! is a free service. Their customers are getting all they pay for. I'd like to help you, but frankly, I can't get them on the phone or to give a reasonable response via e-mail. Your best bet is to require a contact method that refuses or bypasses Yahoo!. They aren't in the business of giving their customers reliable email service."
Do I have problems? I'm sure I do. But since Yahoo! won't discuss them without jumping through their useless DKIM/DK hoops, I'll just ignore it and move on.
Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves.
SPF is not an anti-spam measure, it's about preventing hijacking of domains. People often seem to say "but spammers publish SPF records", and that is true, but it doesn't mean that SPF is not effective.
SPF allows me to publish information about what systems will legitimately send e-mail using that domain. It also allows me to act on that information published by other third parties.
What this means is that I have to deal with dramatically less backscatter spam. Since implementing SPF, I have not woken up to find 100,000 messages in my box that were bounces or outraged replies to spam sent by someone else. Back in 1995 that exact issue happened to me, and to a lesser degree it happened regularly until SPF.
There are, of course, some difficulties with SPF, but despite those I have chosen to use and advocate SPF.
You do have to deal with legitimate third-parties sending mail from your domain. We use an outsourced accounting package and have had to include their servers in our SPF records. No big deal.
As a recipient, if you have one account forwarding to another, and the destination account implements SPF, then you either need to white-list the forwarding machine(s), or you need to implement SRS there.
DKIM and it's variants is, IMHO, useless because it only allows you to prove that e-mail came from an authorized sender for a domain, it does *NOT* allow you to tell if e-mail came from an UNAUTHORIZED system for a domain. You cannot use DKIM to tell if a sender address is forging the domain.
So DKIM is *NOT* a "better SPF". They *ARE* compatible though. If you get a message claiming to be from a specific domain which fails the SPF check, you probably still want to allow it if it passes DKIM. I don't know of any mail programs that do that though. The unfortunate thing about this is that SPF-only can be implemented entirely at SMTP time (RECV FROM) where SPF+DKIM would have to be implemented after receiving the message (after DATA).
Sean
We do something similar. Bad HELO/EHLO? Disconnected and blackholed for an hour. It then goes against three RBLs, Spamhaus being the most effective. For what we pay, it is by far the best value of the entire system. Anyway, anything matching that is blackholed for an hour. Then we check for spoofed addresses, then SPF records. After that, it's sent on to the anti-spam systems. We figured last that we block about 92% of all attempts to send messages, between dropped connections, rejections, and quarantines.
You can never go home again... but I guess you can shop there.
The thing is that due to forwarding and vanity servers, you ARE asking people down the wire (which you can't predict and have NO control over) to throw away mail you sent. When you send to a buddy's vanity domain (that you don't know is a vanity domain) and it forwards your email to their ISP or work account that does check SPF records, your mail gets ditched. And you usually won't get any notice, so your email just disappears.
Using SPF increases the likelihood of your email getting sent into the ether to not return.
The SPF folks themselves acknowledge this as an issue and recommend using SRS to combat it. Of course, since no one uses SRS, it's still an issue.
Portable versions of Firefox, GIMP, LibreOffice, etc
rfc1123 removed the relay function whereby your mail server may list other server's names in the return path.
The replacement is that only your mailserver is listed in the MAIL FROM entry. The RFC never said anything about adding some other server's address to a MAIL FROM line, that was never allowed by the standard. Your mail server can only add its own address as it is known to the mail server it is sending to period (for a new mail from: line).
To list the address in MAIL FROM: means that you (the sending mail server) are claiming you can take responsibility for being able to return mail to the address you list.
That is: by listing an address in mail from, you PROMISE you can immediately return mail to the host you received it from, if there is an error.
There are traditionally only two ways a mail server can take that responsibility: either (1) the item you list in MAIL FROM is you, and you list a local user, then of course you can return the message to your local user, OR:
(2) You are relaying the item, and you received that address in MAIL FROM. You can live up to the promise by delivering the error message to the server that relayed you the message with that address in MAIL FROM, and that server can live up to their promise, and so on.
Well, without source routing (2) is actually a stretch.
The elimination of option (2) as a way to satisfy the promise you make means that you are left with option (1) only.
How lucky you are. I have both SPF and DomainKeys implemented and one of my domains has plenty of problems with Hotmail and Yahoo. I send no lists. Never sent anything with a resemblance of spam. Only email confirmation emails. My IP is clean everywhere.
2011. The year Gnome decided Linux will never be on the desktop.
The domain name given in the EHLO command MUST be either a primary host name (a domain name that resolves to an address RR) or, if the host has no name, an address literal, as described in Section 4.1.3 and discussed further in the EHLO discussion of Section 4.1.4.
If your mail server talks to the outside world, it must introduce itself properly. There's no wiggle room there.
Wolde you bothe eate your cake, and have your cake?
This is utter nonsense and a major misunderstanding of the standard on your side. Also RFC821 was deprecated by RFC2821 eight fricking years ago. It is telling that you apparently did not know that, because otherwise you won't argue with RFC821 and RFC1123 which updates RFC821 instead on looking at the later standard which incorporated the changes proposed by RFC1123. It is also important to understand that a RFC is what it names tell: A Request for Comments, not a binding standard. The pure existence of RFC1123 does not mean that everyone has to follow it ideas.
Having said that, and ignoring your misleading statements, the case is very simple. Regarding to RFC2821 - which actually is widely accepted as an standard - the MAIL FROM command specifies " The reverse-path consists of the sender mailbox." It is a no-brainer that a relay is a relay and not the sender of the message and thus does not change the "sender mailbox". So to comply with RFC2821 as relay can NOT change this value and HAS TO use the original address (which is by the way not a server address as you called it). While this is fully standard compliant, the existence of SPF and other spam filtering messages nowadays make it impossible to follow the standard in this, thus mechanism like SRS (sender rewriting scheme) were invented. But while the use of such hacks is widely accepted and understood as a necessity, it is in fact a violation of RFC2821 which mandates to keep the original value.
And for the issue of what a RFC means, you really should read RFC 2026 "The Internet Standards Process". As said, a RFC is not a standard, you might have confused RFC with the BCP (best current practice) and STD (standard) series of documents.
RFC1123 is a standard, STD 3 is currently RFC1123. RFC2821 is currently not a standard of any kind. Check the listing in the RFC Index next time.