Slashdot Mirror
Does SPF Really Help Curtail Forged Email Headers?
Intelopment asks: "My Domain name has recently been used a lot in the 'Reply' field by some inconsiderate spammer, and my ISP has suggested that I consider using the Open SPF service as a way to stop spammers from using my domain name for in their mail headers field. From what I can tell, it requires the receiving mail server to actually participate in the SPF service, which is where I have my doubts. Does anyone have any experience with this service? Does it work? Are many ISPs using Open SFP?"
I know of at least one ISP that checks SPF records. SPF costs very little to implement in most cases and does not break email for someone who is not using it. Based on that there is really no reason *not* to implement it. It won't completely solve the problem, but it does enable someone who is SPF-aware to filter those emails.
... but only if you use it.
Add SPF to your domain, and whatever subset of ISPs / mailservers that use it probably won't bug you. The only downside of using SPF is that you may have to change your DNS records if you want to use a new mailserver, but most people that I know only use one or two servers for outgoing mail for any one domain.
One DNS line to potentially stop a joejob against you - it's a no-brainer, even if you "have [your] dobuts". Go to the SPF Setup Wizard, fill in your servers and copy the IN TXT line.
See if it works, and proceed from there. If it doesn't, go back to the ISP and complain.
Several ISPs use SPF, for example, AOL does.
http://www.postmaster.aol.com/spf/
Several ISPs don't.. For example, yahoo is busy pushing the competing standard of domainkeys.
Many open source spam scanners use it, ie: SpamAssassin.
However, even if not everyone supports SPF, at least some folks do, and that means if and when your domain does get forged by a spammer, there will be fewer folks receiving it, fewer mailservers accepting it and fewer bounces/complaints heading your way.
And of course, SPF is more-or-less cost free.. All you have to do is add a TXT record to your DNS, which probably won't cost you anything unless your DNS is hosted on some oddly billed 3rd party service.
I'd say the ROI on it is pretty good.
Many folks will immediately bash SPF as a poor spam control technology. Well, they're right, but that's not the point, and it's not what SPF is for, and it's not what your trying to get out of SPF.
SPF isn't a "cure-all" for spam that some folks think it is and others bash it for not being, but SPF IS a reasonable start at controlling forgery, and it's quite effective at it.
-Matt
Some can, but be sure to make sure that it blocks both UVA and UVB spam.
- RG>
Hey pal, this isn't a pleasantforest, so don't waste my time with pleasantries!
I used to receive 30 bouncebacks a day due to spam. I switched to SPF, and it didnt immediately make a difference. After several weeks I noticed I was receiving maybe 1 or 2 bouncebacks a day.
I cannot be certain whether this is due to the spammer observing my implementation of SPF and no longer using my domain as a return address, or whether the spammer still uses my domain but mail servers have stopped sending me the bouncebacks.
Either way I+internet won, spammer lost.
SURELY NOT!!!!!
I was initially like 'Why do I care?' but once I finally realized that it could help prevent people from using my domain name to spam -with- (rather than -to-), I was all for it. Especially since, as you note, it costs me nothing but a bit of time to set up. (And not much, since I use Google's mail servers, and they practically push the information on you.)
It may not have a huge effect, but as a domain owner, I have had my domain 'used' a few times as the return address. It hasn't happened since I set up the SPF record. (Likely spammers don't think I'm as nice a target now.)
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
Yes, it does work. I started using SPF and almost immediately stopped receiving spam backscatter. Besides that, I activated SPF check in my SMTP server and since then, we drop a lot of forged mail headers too. Its ridiculous easy to implement, consumes nothing more than a DNS record and can be fine tuned. Besides that, every single big mailer is already using it.
We checked SPF on all incoming mail to our ISP, it worked for a while, but eventually it wasn't worth the effort of dealing with legit mis-configured companies. Not to mention the fact customers wouldn't believe it wasn't our fault. Yes even banks make mistakes.
The best part of using SPF, for me, is responding to automated mailers that send me messages saying "Your message to us failed an SPF check!" I always have great fun explaining that failing an SPF check means that they would have a better chance of reaching the person who actually sent the message by picking a random address on a random other domain.
how to invest, a novice's guide
I think the spammers check the SPF records, and if there is one they don't forge your address.
I had lots of problems with my e-mail address being forged by spammers.
When I put in an SPF record, it stopped immediatly.
... little to filter incoming mail. To protect your outgoing mail, all you have to do is publish a special DNS record - that's it, done, no need to change it as long as your MX servers don't change. It's That Simple.
On the incoming side, a lot of ISPs are using it - and a great number of corporations are using it, even if they don't realize it. Spam Filter boxes like those from Barracuda (Can't recommend these guys enough), or software like SpamAssassin, can easily check SPF records. I think Barracuda's do by default, but I could be wrong - it's been a few years since we installed our Barracuda.
Granted, it's only one part of a good anti-spam system. I use SPF, DomainKeys/DKIM, SpamAssassin, and a nifty little feature of Sendmail called "greet_pause" (check it out if you use Sendmail for inbound email). It's cut down on my junk mail by an ungodly amount.
Poor means hoping the toothache goes away.
For several years I've been running LogSat Software's Spam Filter ISP in front of my Exchange server. It uses SPF, blacklists, and Bayesian filtering to keep spam out, and between SPF and the blacklists, about 97% of the incoming spam connections I used to get are now disconnected immediately. The savings in bandwidth (and in processing power and storage space on my mail server) has been enormous.
It allows me to set up a whitelist of the legitimate email addresses in my domain, and if an email tries to come in to an address that isn't on the whitelist, the connection is immediately dropped. So no more endless stream of "abernathy@mydomain.com,abraham@mydomain.com..." spam clogging up my badmail folder. YMMV, but I tried a number of different antispam products before settling on this one, and I'm a very happy camper.
My truck is like a series of tubes.
It took me maybe an hour to fully implement it a few years ago (most of which was just reading), so why WOULDN'T you try it? There's really no downside other than having to maintain a list of outgoing mail servers. And unlike something like domainkeys, you don't have to go around messing with your mailserver configuration. If it gives you some kind of problem, it only takes a few minutes to disable it.
AccountKiller
Barracuda (Can't recommend these guys enough)
Recommend? Those bastards, their asshat defaults, and their RTFM-impaired users are responsible for some 40% of the shite in my mailbox right now (though that is unusually high, I grant you). It is NOT acceptable to bounce "back" to an innocent victim. It is NOT acceptable to advertise the piece of shit responsible in the subject header either - though I like to imagine competent sysadmins the world over vowing not to buy the product as a direct result.
If everyone set up a rule to forward anything with "Message you sent blocked by Barracuda" to sales@barracuda.com with a "please fix your defaults", would that constitute a DDoS or just a mass appeal? (Yeah, I posted an email address. I figure they should be able to handle it, no?)
I got a couple hundred bouncebacks yesterday, despite having an SPF record. They don't seem to help much.
m l
There are quite a few domains that have SPF records, like AOL, but having a record, and bit bucketing mail that SPF says is forged, are two very different things. Very, very few domains do the latter. It would be nice if more did. Does AOL? Well...
I'm still getting hammered.
One spam I received yesterday suggests:
A spammer tried to send spam
From: @elvey.com
To:@aol.com
AOL's response was:
554-: (ISP:B2) http://postmaster.info.aol.com/errors/554ispb2.ht
554 TRANSACTION FAILED
The spam was transmitted from/via mail2.infoquesthosting.net [65.61.1.49], a Barracuda Spam Firewall 3, which is broken by design.
You thought Barracuda was helping FIX email? They're making the problems worse! But let's keep on topic...
So, the Barracuda takes the spam and sends it to @elvey.com, the innocent victim forged in the From:, despite the elvey.com SPF record.
This is typical. Like I said, a couple hundred bounces yesterday...
If the server of a so-called leader in the antispam arena is sending me spam despite appropriate SPF records, what does that say?
Make 'em pay! http://Payola.org #include "stddisclaimer
I routinely get bounce headers (for invalid destination addresses) from places where they are receiving mail which has been forged under one of the domain names I own (I happen to own over 20 domains and only about 2 or 3 of them would ever send any mail). All of the domains that I would send mail from have three IP addresses listed in the SPF records in the DNS: the fixed IP address that I get DSL supplied from Cavalier Telephone (for mail sent by my computer); the IP address of Cavalier Telephone's mail server (for mail my computer sends through them); the IP address of the webserver that holds my blog (when it sends out mail messages). No other IP address should be considered authentic and my SPF records indicate this. Yet I routinely get bounce messages from places sending bounces to me for spam which has been joe jobbed by someone else using one of my domain names. Maybe some places are rejecting forged mail from other places, but I still see places that apparently don't bother to check SPF records when they do exist.
Paul Robinson — My BlogThe lessons of history teach us - if they teach us anything - that nobody learns the lessons that history teaches us.
Certain major western Canadian cable ISPs who shall remain nameless have been known to silently trap SMTP connections, and then forward the messages on to the destination.
Now, that's all fine and good if you're willing to "include:" their SPF records... but unfortunately this steadfastly anonymous western Canadian cable ISP also happens to be one of the most prolific botnet havens in the world, so you're not really cutting down the number of people that can claim to be you by a useful amount.
Your best bet is to run a VPS mail server, choose a different port for your client to connect to, and use SSL. Alternatively, you can outsource your mail service to someone like Google (who provides SPF guidance).
I rarely criticize things I don't care about.
Supporting SPF doesn't put an end to spam, but it's one of those best-practices things that can really make your life simpler down the road.
For outgoing mail service:
-It becomes immediately apparent when "surprise" mail servers pop up. This can be a web server that's sending outgoing mail directly, or someone sending mail through their ISP's mail servers when they should be connecting and authenticating to your servers, etc. Tracking down mail problems in these situations can be very frustrating.
-It helps prevent forged messages claiming to be from your domain. Not all recipients support this, but even after the fact it's helpful to be able to have an answer for what can be done about it that doesn't get any blame on you.
For incoming service:
-Even a moderately strict SPF policy can help prevent bounce-spam from being sent via your servers.
-It helps protect your users from scams.
It's not a perfect solution, but it puts your network in a better defined state. And that helps keep things running smoothly.
I rarely criticize things I don't care about.
SPF operates on SMTP MAIL FROM and HELO domain names, potentially allowing receivers to reject before DATA which may contain arbitrary headers. The Microsoft PRA nonsense abuses SPF records for mail headers.
Since Mar 26th 2007 I've gotten dns requests for SPF (type 99) records 35 times, and text records (possibly/probably? for SPF) 692 times.
So, someone is checking.
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
Consider migrating your mail servers to Gmail for your Domain:
https://www.google.com/a/cpanel/domain/new
I had these sorts of "Joe Jobs" against my domain for 2 years. The last straw was when I actually had a client upset at me over spam sent on my behalf from a different server. I explored a lot of different ways of stopping it, and ultimately arrived at moving my MX records to Google servers as part of the above Google Apps for your Domain. It uses SPF, and presumably Google's other tools they use to protect core Gmail users. The Joe Job emails stopped (I'd repeatedly get emails about send failures sent to me in regards to the Joe Jobs prior, and the occasional complaint). Not 1 more complaint or send failure notification.
It should be noted that the SPF settings that Gmail for Domains provides uses ~all instead of -all. What this means in layman's terms is that an anti-spam filter is supposed to use the SPF record as a hint, rather than a requirement. Gmail does this most likely so that people who use their ISP's SMTP server to send out mail will still have their mail go through.
The point is, if you exclusively use Gmail for SMTP, you should change the record to -all for better protection.
Well, I found it funny!
As far as its effectiveness goes, in one analysis where we sampled a set of messages in which the purported sender's domain was that of a major ISP, we found that if the SPF authentication check returned 'softfail', the probability of the message being junk was near 100%. When we checked our MTAs "Received" headers, they indicated that the messages were being sent from IP addresses in different countries and domains (as one would expect). Of those messages that passed, only about 30% of the messages were junk. Clearly there is 'signal' in the SPF score.
Interestingly, of those messages that passed and yet were junk (those that composed the 30%), all appeared to be sent by a legitimately registered user at the ISP. This is the double-edged sword of authenticating your messages if you are an ISP: if your own user base is sending junk, other ISPs and recipients will be able to figure it out. And you might be perceived poorly.
Yet this is exactly what should happen; it's the point of authentication. There should be motivation for ISPs, either financial or brand-related (which ends up being financial), to establish and operate procedures that screen members or deter them from sending unwanted messages. Reputation (or concern of damage to it) is a great motivation.
The real promise in sender-authentication though is DKIM. While SPF is easier to implement for senders than DKIM, SPF is rather fragile; it doesn't survive forwarding without re-writing the envelope-from. Too few systems are set up to do this (list management software is the exception), and although changing the behavior of MTAs is just software, doing so will effect the efficiency of status (bounced mail) reporting. Messages that would be delivered 'point to point' in the past end up being 'source routed' with many unnecessary hops, increasing the odds of failure. DKIM is a little more involved to set up, but doesn't have these fragility issues (setting up checks when receiving is about the same level of difficulty for SPF and DKIM).
At Boxbe, we check both DKIM and SPF. The reason is that strong sender identity gives a pre-approval policy its teeth. We quarantine messages which fail EITHER form of authentication, but because DKIM is "forward-friendly" and SPF is not, if a message passes a DKIM check but fails an SPF check, we let the message pass (according to our member's preferences). Using both has merit as each type is a little different. Gmail has been signing/authenticating with both DKIM and SPF for quite some time. We also use both forms of authentication when we send out messages or forward messages to our members.
As other organizations adopt sender authentication (Comcast has announced it is implementing DKIM by year end) it will become a very effective tool.
--
Thede Loder
E: thede@boxbe.com
Agreed.. I didn't mean to imply "odd" was particularly rare.. Just odd.
"We'll provide you 50MB of webhosting for $10/month.. Oh, you wanna add another 60 bytes to your zonefile for another DNS entry.. sure thing, that'll be another $2.50/month".
-Matt
Hey, at least AOL is rejecting it at SMTP delivery time.. If they were queuing it, then post delivery bouncing, I'd call them evil, but right now they're doing the right thing.
:)
After all, if the recipient didn't exist, they'd issue a 5xx error too. Nearly every sane domain will do that. This isn't really any worse for you. Or do you expect them to accept delivery of and swallow those too?
Quite frankly, this still makes your life *MUCH* easier when compared to the post-delivery bounce case.
All you have to do now is blacklist the servers that are being used *sources* and *relays*, as opposed to all the servers that are *destinations*. And quite frankly, the server that's acting as a spam source or relay deserves to be blacklisted anyway.
What could be a DDoS (backscatter from many servers) is in this case, just a single source DoS that can be handled by blacklisting a single IP address.
What's even better, is if you blacklist 65.61.1.49 at the SMTP layer, those spam messages will double-bounce and end up in infoquesthosting's postmaster mailbox. Seeing as it's their spam problem in the first place, it should be fitting that they end up with it.
So do yourself a favor and add 65.61.1.49 to your SMTP access list today.
-Matt
Comment removed based on user account deletion
SPF is cost-free as long as the SPF records remain accurate for your domain. The problem is that things change, servers get moved, new websites get created, and if the SPF records aren't updated to reflect those changes, then some emails are going to go missing. A problem like that could go undetected for months, and that bears a cost.
The fact is, in larger companies the left hand doesn't alway know what the right hand is doing, while in smaller companies IT can get farmed out to multiple providers who can be blissfully unaware of what is happening in the big picture.
SPF can be an effective technology for companies that have their IT act together. But it can be a source of headaches for everyone else, which I expect is the majority.
I work at a hosting company, which means I deal with people frustrated over email forgery all the time.
SPF records are in no way a perfect solution (though if everyone implemented it it'd be good enough), however it is pretty much the ONLY solution that is effective at all at this point. They cost nothing and they benefit you and the millions of people who will not be receiving the SPAM because of the SPF records. Do it, do it now, and tell everyone else you know to do it too. The more people that use them, the more likely the providers are to implement it.
And closer to being on topic tham many +5 insightful comments seen around here.
Since we're on the subject... I think the system could use a -1 metadiscussion moderation to boot.
Platform advocacy is like choosing a favorite severely developmentally disabled child.
The problem is that things change, servers get moved, new websites get created, and if the SPF records aren't updated to reflect those changes...
...then some emails are going to go missing. A problem like that could go undetected for months, and that bears a cost.
Yes, it's possible that all the other DNS records could get updated and just that one be ignored but it's pretty unlikely.
Having had occasion to mistype an IP in an SPF record once or twice I can assure you that it does not go undetected for months. A few hours perhaps, (pft, more like minutes). There are enough ISP's and private spam filters using SPF now that if there's a glitch it shows up pretty damn quick in the form of an irate user, "I was able to email this person yesterday, ZOMGWTFBBQ?!?"
are responsible for some 40% of the shite in my mailbox
For what it's worth one of the updates recently disables bouncebacks triggered by SPF check. They heard that complaint from a lot of folks and finally got around to fixing it.
Interesting to hear that AOL is using Barracudas now, I knew they were switching but didn't know who won the contract.
FYI - see my post above regarding Barracuda's SPF bouncebacks, that's been taken care of recently.
I've seen it happen it twice. In both cases you had company.com with SPF on its domain running a website alternatebranding.com on a colocated server, but which sent out automated emails using @company.com From addresses. When the server was moved to a different IP, the DNS on alternatebranding.com was changed, but the company.com SPF wasn't, and emails started to go missing. In one case it went undetected for over a month, and in the other it was well over a year.
Its possible the spams originated from inside the 'trusted network(s)' that the barracuda mailserver allows relays from.
i.e. a webserver that has/had an exploitable site or a internal machine infected with some malware.
Well, as new sites and servers are created you'd assume that new dns records would be created for them too, so while your there editing your dns records you can update the spf too.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
I reject such bounce-backs at SMTP time. Any email server / software that accepts virus / spam laden mail and then later bounces it is the cause of all collateral damage. Incompetent sites that operate such servers should be blacklisted.
Its part of the game. You either have your act together or you don't. If you don't have your act together, then you can only blame yourself.
Lord High Crapflooder The Right Honourable Vlad Craig Esther McDavenpherson III
Destroyer of Mercatur.Net
- Perform an SPF check, and know to a reasonable certainty that you are not the sender of the message, and then
- Send you a mail containing an advert for their product?
If they do, then the first part should be enough to show that they don't have prior business relationship with you (and are aware of this fact) and the second part shows that they are sending commercial email to you. I am not a lawyer, but it sounds like they are quite nicely fitting the description of unsolicited commercial email as defined by a number of laws worldwide, and could probably be sued quite easily if you had a chat with a lawyer.I am TheRaven on Soylent News
The fact is, in larger companies the left hand doesn't alway know what the right hand is doing, while in smaller companies IT can get farmed out to multiple providers who can be blissfully unaware of what is happening in the big picture.
This is true, as I am all to personally familiar with the situation.
However, such organizations are going to have dozens of other problems causing loss of mail, network outages, etc.
An IT team that does not properly communicate and/or oversee its own networks is a disaster waiting to happen. SPF might exacerbate this problem, but it's hardly a root cause.
So, I'll agree with you. If your network is poorly managed, don't bite off more by adding SPF to the mix. First spend time getting your existing network under control.
-Matt
Comment removed based on user account deletion
http://www.zoovy.com/ Zoovy.com is an e-commerce provider that requires all customers using their mail service to use restricted SPF records for their domains. This has cut down on our SPAM being sent both to and more importantly *from* our domains by spammers considerably.
... my prediction is it will probably happen sometime after IPv6 is rolled out.
The problem is most ISP's and other hosting providers don't control the entire e-mail application stack enough to implement it without an army of technical support people, it's just not economical. That and diagnosing mail problems is too freaking difficult for low level helpdesk people.
It's like credit card fraud, the entire system will need to be retrofitted before it can be significantly reduced or even eliminated, but the short term of cost of dealing with fraud outweights the long term upfront cost of retrofitting billions of dollars worth of swipes, magstrip readers, and point of sale systems.
Eventually the problem will get bad enough and/or a big mail provider (hotmail, gmail, yahoo) will grow a pair and start flagging email that arrives at domains without SPF as spam. Either that or something like Y2K will happen again and require everybody to update to stuff that supports SPF, this could be as soon as 2010 when we run out of IP addresses.
Wouldn't hold my breath though
Consider the following:
S: 200 happy to meet you sir
C: helo example.com
S: 220 happy to meet you
C: mail from:
S: 220 ok
C: rcpt to:
S: 220 ok
C: data
S: 220 begin
C: Subject: v1ag7a
C: From: customersupport@ebay.com
C: To: you@yourdomain.com
C:
C: message body
C: .
You see how the mail from envelope can be manipulated to hold a domain that differs from the message body headers. This is ok for SPF since otherwise it would break email lists.
What it might do is help prevent back scatter spam from hitting your domain if the original recipient's mail server DOES check the SPF for the mail from... that is all.
Why UNIX?
Yes, the 5xx is a big improvement, you're right. What we need is a Blacklist that ... well, backscatterers.com is up, but doesn't seem to be operational. It should be easy to drop all mail from from IPs in such a BL, while treating the rest normally.
Make 'em pay! http://Payola.org #include "stddisclaimer
I have no idea why you were modded flaimbait, because you are largely correct.
With the current epidemic of joe jobs going on, it makes absolutely no sense to bounce emails back when they are not coming from an SPF or domainkeys server. Even when they are it doesn't make a whole lot of sense.
Both of my email providers use SPF, and one of them uses domain keys as well, but as you so aptly stated, if the other servers are ignoring the additional information, the benefit in that area is small.
Even in those cases, the benefit to one more server using SPF or domainkeys does exist, it is one more server that represents a microscopic move towards the newer authentication methods, and one less domain which will have messages trash canned for not having the information.
Does this then apply just to SMTP? Or is it applicable to the webbased email as well?
On the bright side, last time I checked gmail also supported domainkeys.
Companies have websites with dead links that don't get caught for years too. People make mistakes under any/all circumstances, not just involving SPF records. Sounds like these guys weren't exactl on the ball though, any automated system should have some sort of reporting/monitoring with a live human on the other end.
I reject such bounce-backs at SMTP time.
:)
So how do you know when it's a legitimate bounce-back? Or do you just reject all bounce-backs out of hand? (Sure hope that's a home network!
Content scanning. I maintain a regex blacklist of known collateral damage "notifications".
I'm not saying that their default settings are right, but the product doesn't magically work out-of-the-box. The system administrator has to go into the config and set it up--if some no-talent ass-clown doesn't properly set the bounce settings (aka backscatter), then it's the no-talent ass-clown's fault. The same's true elsewhere: Smith & Wesson sells guns that will shoot bullets, but you have to put the bullets into the gun before it can do anything. If you fail to set the safety after putting in the bullets, is it your fault or Smith & Wesson's when you shoot yourself in the foot?
Blame incompetent mail server admins for not researching best practices, not Barracuda (who make a powerful anti-spam appliance for those of us who know what we're doing).
No, but it does help protect against sunburn.