SPF Design Frozen

Semi offtopic, but... by Kethinov · 2003-12-15 16:41 · Score: 2, Interesting

I've always wondered how a spam filter system based on authorization might work. Your mail server could automatically send out a verification request to the email address that sent the email, then if the email address exists, an authorization would be sent back to your mail server. All mails that weren't confirmed by a returned authorization could be automatically deleted. This way, you could only get mail from active email addresses. Could cut down on email spoofing because anyone spamming you would have to use a real email address which would allow you to complain to the domain owner. Of course, all mail servers in the world would have to be upgraded to this new protocall for it to work, or everything would be considered spam.

Does any of this make sense?

--
You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!

Re:Semi offtopic, but... by Anonymous Coward · 2003-12-15 16:48 · Score: 0

no
Re:Semi offtopic, but... by Anonymous Coward · 2003-12-15 16:50 · Score: 0

yes
Re:Semi offtopic, but... by Anonymous Coward · 2003-12-15 16:53 · Score: 0

depends
Re:Semi offtopic, but... by SpinningAround · 2003-12-15 16:56 · Score: 2, Insightful

I was thinking that something like this would be a great idea... sort of an automatic whitelist.
And then it occured to me that spammers would probably find some devious way to use this feature to generate lists of valid email addresses. Rather than sending a zillion emails based on a dictionary attack, they could send a zillion emails to known good addresses.
Unless all the email servers in the world worked on the new protocol all the servers with the validation functionality would be MORE vunerable to spam and not less.
Re:Semi offtopic, but... by bluGill · 2003-12-15 16:57 · Score: 1

Not really. Spammers have already forged my email address in the from line. You can check with my mail server, but my email address exists so it will report good.
Don't suggest that hashes of emails sent be saved, while it could be done, my ISP for email is from years ago, everything else is done though a different ISP, including sending email. Basicly you can ask my ISP if I sent something, but they have no way of knowing if I did because none of my outgoing email goes though them.
Re:Semi offtopic, but... by Kethinov · 2003-12-15 17:03 · Score: 1

Yes, but under the new protocall, it would be impossible to forge an email address because the email address sending the mail would have to be able to recieve mail in order to send or it would be automatically deleted.

--
You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!
Re:Semi offtopic, but... by chromatic · 2003-12-15 17:27 · Score: 1

Instead of impossible, you might more accurately write ineffective only in delivering messages only to servers which perform this check. Spammers don't particularly care about bounces and forged headers now. This scheme would only be effective if almost everyone switched to it.

--
how to invest, a novice's guide
Re:Semi offtopic, but... by Kethinov · 2003-12-15 18:04 · Score: 1

I agree, but if everyone did switch to it, it would virtually stop all spam. Anyone who did spam you could be positively ID'd based on what domain they came form and a complaint could be filed against that domain name. Wouldn't be perfect but it would be a hell of a lot better than the way things are now.

--
You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!
Re:Semi offtopic, but... by Anonymous Coward · 2003-12-15 19:59 · Score: 1, Informative

SMTP contains a VRFY command to check the validity of an email address. You could connect to the sender's SMTP server and use VRFY to check the validity. Except that the command is often disabled, apparently because spammers used it to collect valid addresses.

Could cut down on email spoofing because anyone spamming you would have to use a real email address which would allow you to complain to the domain owner.

It wouldn't cut down on spoofing - a spammer would just need to spoof a valid address, which is trivial to find using the verification requests you described. The verification just proves an address is valid, it doesn't prove that a user or even a mail server actually sent an email.

I'd rather have mail servers reject unsigned messages sent from my email address, and unencrypted messages sent to it. But that seems unlikely to happen.
Re:Semi offtopic, but... by sperling · 2003-12-16 00:34 · Score: 2

The whole concept is based on a mechanism deciding whether ip X is or is not permitted to send mail from address Y, so this will indeed cut down on spoofing.
Read the article ;)

--
The next great MMORPG.
Re:Semi offtopic, but... by dodobh · 2003-12-16 05:39 · Score: 1

Its called Challenge Response, and there is plenty of documentation on why this is a really bad idea.
Just hunt it down in google

--
I can throw myself at the ground, and miss.
Re:Semi offtopic, but... by rthille · 2003-12-16 05:50 · Score: 2, Insightful

Are you suggesting this in concert with SPF? If not, then spammers would just forge the mail as being from a valid address.
What you describe sounds sort of like a challenge-response system (http://tmda.net), but more automated. The trouble with that is the same trouble with VRFY. That is, it can be used to determine whether an address is valid or not for future spam attacks. That's called RCPT harvesting.

My brother runs thille.com using sendmail and he didn't disable VRFY. Nearly all the accounts on there got harvested and we started getting tons of spam. I run thille.org (on qmail) and haven't had that trouble.
Since robert@thille.com forwards to my mail server, I just setup TMDA on the address it forwards to.

I've also had trouble with being 'joe-jobbed', where someone sent spam, and forged the 'From:' (and probably the envelope as well) as from me. I got lots of bounces, but luckily no hate mail. SPF would help with that.

--
Awesome furniture, accessories and cabinetry in Santa Rosa, CA: http://humanity-home.com/
Re:Semi offtopic, but... by Anonymous Coward · 2003-12-16 06:58 · Score: 0

The comment was based on its parent comment "Semi offtopic, but...", which describes a new idea for spam filtering and is unrelated to the article.
Re:Semi offtopic, but... by esquimaux · 2003-12-18 16:08 · Score: 1

And plenty of people who are using it quite happily. See EarthLink's SpamBlocker for a related approach that uses a kind of challenge to help the user populate his whitelist / address book.

There's always plenty of documentation on why something is a really bad idea. "'Considered Harmful' Considered Harmful". I clean out my spam folder once a week or more, and haven't found fewer than 1000 messages in it, usually 2000, for as long as I can remember. These are desperate times.
Re:Semi offtopic, but... by stinkycat · 2003-12-22 05:31 · Score: 1

I've been spoofed a several times (a spammer put MY email address in the reply box of their message.) If anyone replied to the spam message it came to me (along with the 1,000s of auto-replies "message can not be delivered to xxx address" etc.) It was absolutely horrible. If an authorization system was used, spammers may just select an actual address at random from their spam lists and put it in the reply to trick the system. I wish Gonorrhea upon all spammers. -C

I like it, but.... by hawkbug · 2003-12-15 16:47 · Score: 3, Insightful

This is a great idea. I'm all in favor of it. I would update my companies DNS to this new standard immediately. But, I envision these issues, correct me if I'm wrong:

1) Increased network traffic at all points - where one mail server gets the email, and the network of the domain being sent from or forged. Imagine how this might increase AOL's or hotmail's network traffic, while they gain nothing from it. Every mail server in the world could be trying to contact their dns servers to check if they allow the mail. I hope you like lag if you use AOL.

2) Spammers tend to use made up domains anyways. This is bad with this method for several reasons. The first being that you will have delayed email receiving times because your mail server will be trying to contact dns servers that don't exist. The timeout would have to be short for this to work.... then on the other hand, if the timeout is too short, and a busy mail server can't respond in time, the email is rejected, which is just as bad as a real email getting flagged as spam, aka a false positive.

While I agree in practice with this technology, I'd like to see how people can solve these issues before I would use it at my company.

Re:I like it, but.... by Anonymous Coward · 2003-12-15 17:05 · Score: 0

Are you worried about delayed mail receiving times, or delayed spam receiving times ? Because point #2 with the timeout seems to delay mostly mail I don't want, which is going to be rejected by my mail server anyway.
Re:I like it, but.... by eakerin · 2003-12-15 17:15 · Score: 3, Insightful

1) this shouldn't generate that much traffic, it's only one (or possibly more, depending on how you have it setup) query per domain, most likely the information will be cached as well (depening on the TTL set).

2) Most mailservers don't accept mail from domains that don't exist (eg, they already query the DNS servers of the domain in question) and since the timeout for a DNS response is rather short, it shouldn't affect mail receipt that much.

Actually the whole thing could be handled in 1 query, just look up the SPF record for the domain in question, if it comes up with NXDOMAIN, that domain didn't exist in the first place. if it didn't fail, but didn't find the SPF record, then we try TXT, if that dosn't show up, then we just allow the mail through.

if a mailserver has a problem accepting mail it also returns an error message, basically telling the remote server to try later, or another mailserver. so no mail is lost (unless the servers remain loaded down and un-responsive for about a day)

Oh, and I've already setup the record for my domain, and I'll be updating my MTA to query them soon!
Re:I like it, but.... by Piquan · 2003-12-15 17:15 · Score: 2, Insightful

1) Increased network traffic at all points
I expect that after SPF gets into wide usage, that spammers will no longer forge from those domains. So by adding SPF records for your domain, you lower the spam bounces and joe-job BS that you get. Besides, a DNS query is cheap, much cheaper than an email, particularly since it can be cached. And you are using your ISP as a cache, right?
Worst case, no caching, you add about 400 bytes of traffic per email. That's nothing; the rest of the traffic involved in sending or receiving an email ends up about about 16k minimum anyway. Who cares about 400 bytes?
Big picture-wise, widespread implementation of measures that prevent spam are going to lower the traffic that AOL, hotmail, and other major email providers have to pass. Have you looked at the statistics? Add in labor etc, and I'd think that AOL, hotmail, etc would be jumping at the chance to help lower spam.
2) Spammers tend to use made up domains anyways.
Not in my experience, not anymore. Many MTAs will, by default, bounce mail that doesn't come from a genuine domain. So spammers use real domains, or real email addresses.
...you will have delayed email receiving times because your mail server will be trying to contact dns servers that don't exist.
No, if a spammer sends mail from a non-existent domain, then you get an NXDOMAIN from the root domain servers, and instantly know it's bogus. Why would you try to contact any DNS servers below the root? You'd never have any IPs to try to contact, and you don't send no packets without no IP.
The timeout would have to be short for this to work....
Why? If you're waiting for a response, you're not burning CPU. The timeout can be 1 second or 120, and you still use the same amount of CPU.
[If] a busy mail server can't respond in time, the email is rejected,
Why can't it be deferred (which normally happens with most DNS errors), or even failsafe so it gets passed (which normally happens with most anti-spam DNS measures)?
Re:I like it, but.... by hawkbug · 2003-12-15 17:42 · Score: 1

" 2) Spammers tend to use made up domains anyways.

Not in my experience, not anymore. Many MTAs will, by default, bounce mail that doesn't come from a genuine domain. So spammers use real domains, or real email addresses. "

Well, most of my spam comes from forged domains which don't exist, usually with a few constants here and there with a bunch of numbers thrown in.

" The timeout would have to be short for this to work....

Why? If you're waiting for a response, you're not burning CPU. The timeout can be 1 second or 120, and you still use the same amount of CPU. "

I guess I'm just not sure of how much extra load it adds to a server if there is a huge que of messages sitting there to be verified. How often will the server try to recheck these?

" [If] a busy mail server can't respond in time, the email is rejected,

Why can't it be deferred (which normally happens with most DNS errors), or even failsafe so it gets passed (which normally happens with most anti-spam DNS measures)?"

In my business, deferred email, even 4 hours, is just as bad as not getting it most of the time. Email is a benefit to our business because it's instant. So, delaying it for hours is a bad thing for us.
Re:I like it, but.... by jhunsake · 2003-12-15 19:21 · Score: 1

Well, most of my spam comes from forged domains which don't exist, usually with a few constants here and there with a bunch of numbers thrown in.

Then your mail server needs to be reconfigured. There is absolutely no point in accepting email from bogus domains. I've never used a mail server that did so.

In my business, deferred email, even 4 hours, is just as bad as not getting it most of the time. Email is a benefit to our business because it's instant. So, delaying it for hours is a bad thing for us.

Email is not instant! Sure it may act that way a majority of the time, but no where, in any specification or implementation is email determined to be instant. If you need instant communication, you need to be using something else.
Re:I like it, but.... by Firehawke · 2003-12-16 02:45 · Score: 1

I work at a free email service and we get on average dozens of spam complaints a day. Unfortunately, NONE of these spams originate from our service-- we just happen to be spoofed very frequently by one or more spammers.

That's at least 20 emails a day I have to sit down and respond to, educating users on how to read email headers. Then, on top of this, 99.9% of the spam we RECEIVE is from spoofed addresses--- THAT adds up to the hundreds of megabytes per day category.

This system, even if it adds 400 bytes of overhead to every email transaction, would still cut back overall bandwidth usage by a hundredfold if we could convince even 50% of the ISPs out there to support it.

We signed up a month ago.

Internet does not work that way by bluGill · 2003-12-15 17:02 · Score: 3, Informative

Your points are both invalid.

1) Most mail servers already to a return DNS lookup on the IP of who the sender is. (The recived from lines in the headers) DNS takes so little bandwidth compared to normal activity (even compared to the payload of the email it is tiny, not consider all the web browsing, DNS is trivial)

2)DNS works by asking the root servers who owns a domain. The root servers respond either with the DNS for the domain, or with a no such domain. (Ever hear of Verisign's sitefinder? Verisign runs the root servers, and they started saying anything unowned belonged to them) Essentially no overhead is involved in this.

Re:Internet does not work that way by hawkbug · 2003-12-15 17:54 · Score: 1

"1) Most mail servers already to a return DNS lookup on the IP of who the sender is. (The recived from lines in the headers) DNS takes so little bandwidth compared to normal activity (even compared to the payload of the email it is tiny, not consider all the web browsing, DNS is trivial)"

I realize dns is not a major network usuage offender, but more traffic is more traffic - I'm not saying it would be enough of a reason to not do it. I'm all about killing spam as soon as possible. If this helps, I'm all for it. I simply wanted to know how those issues are addressed.

"2)DNS works by asking the root servers who owns a domain. The root servers respond either with the DNS for the domain, or with a no such domain. (Ever hear of Verisign's sitefinder? Verisign runs the root servers, and they started saying anything unowned belonged to them) Essentially no overhead is involved in this."

Yes, I have heard of Verisigns dirty tactics. What don't understand is how the root servers can return the DNS, when I change ours constantly, and I don't allow domain transfers to root servers... I allow transfers to specific dns server, so do root servers get a transfer by default? I assumed that if a domain name exists, the dns request is passed onto the authorative dns server... is this incorrect?
Re:Internet does not work that way by Linux_ho · 2003-12-15 17:57 · Score: 2, Informative

Also don't forget that DNS caches, so SPF data for popular domains would be cached all over the Internet. Your local DNS server would only make one query to the authoritative server every ttl period.

--
include $sig;
1;
Re:Internet does not work that way by Cyber+Bear · 2003-12-15 18:17 · Score: 2, Informative

Yes, I have heard of Verisigns dirty tactics. What don't understand is how the root servers can return the DNS, when I change ours constantly, and I don't allow domain transfers to root servers... I allow transfers to specific dns server, so do root servers get a transfer by default? I assumed that if a domain name exists, the dns request is passed onto the authorative dns server... is this incorrect?"
Yes, that is incorrect. The root DNS servers hold the DNS glue records for each registered domain. DNS glue records are the NS records created from the DNS server information you specified when you registered the domain. So, you may be changing A, PTR, and CNAME records all you want, but the DNS glue records for your domain don't change unless you make a change with your domain registrar.
Re:Internet does not work that way by hawkbug · 2003-12-15 18:41 · Score: 1

Ok, so what I'm saying is that the MX records will have to get pulled from my dns servers, correct? If so, wouldn't the SPF records also be pulled from my local dns?? If so, then that will equal greater traffic on my network.
Re:Internet does not work that way by Anonymous Coward · 2003-12-15 19:15 · Score: 0

Ok at this point you have demonstrated to the whole of Slashdot that you are a fucking moron.

Thus you should give-up and save what self-respect you have left.
Re:Internet does not work that way by kelnos · 2003-12-15 22:19 · Score: 1

correct, but that wasn't a response to the increased traffic point. you originally made a point (an incorrect one) about domains that do not exist. you said that, for this to work, a receiving mail server would have to contact a nonexistant domain, and have some sort of timeout after which it decides the domain doesn't exist. you went on to say that the timeout would have to be short to decrease lag in mail delivery, but if a server is overloaded, it may miss the timeout, and a legitimate mail would be dropped.

this just isn't how it works. the receiving mail server would make a normal dns request for idontexist.com. the local dns server obviously doesn't know anything about it, so it either tries another dns server directly higher in the chain, or it consults the root servers. one of those servers is going to return "NXDOMAIN" - nonexistant domain. the buck stops there. the domain doesn't exist, and thus the From: header is forged. let me make that clear - a _lack_ of a response (i.e. via a timeout mechanism) _cannot_ indicate that a domain doesn't exist. only a specific dns reply from an authoritative source saying "this domain is not registered to any entity in the world" will make a mail server believe that the domain doesn't exist. there is no timeout involved.

--
Xfce: Lighter than some, heavier than others. Just right.
Re:Internet does not work that way by boneshintai · 2003-12-15 22:25 · Score: 1

Start here.
Re:Internet does not work that way by hawkbug · 2003-12-16 06:35 · Score: 1

No, I was talking exactly about what I was responding to:

"Yes, that is incorrect. The root DNS servers hold the DNS glue records for each registered domain. DNS glue records are the NS records created from the DNS server information you specified when you registered the domain. So, you may be changing A, PTR, and CNAME records all you want, but the DNS glue records for your domain don't change unless you make a change with your domain registrar."

I'm not an idiot - I realize that the root servers register the basic information about a domain name. What my point was, is that the MX information is stored on my LOCAL dns server. Thus, more requests to my local server would equal more traffic, no matter how slight if someone was trying to spoof email from my domain name. That was and still is my point.
Re:Internet does not work that way by bluGill · 2003-12-16 13:19 · Score: 1

If your network cannot handle the additional load from this DNS extnetion then you have bigger problems than this extention. DNS is such a tiny amount of traffic, even after this is implimented that it won't appear on your statistics.
You can't do a DOS attack on DNS with email in this way, because DNS is caching. If someone is spamming your domain all over, DNS servers will remember after the first time what you returned last time and just return the cached result. (Unless you have rediculiously low timeouts, but if you do and it DNS now places too much load on your network, you deserve it for your stupidity)

Not helpful to me by Anonymous Coward · 2003-12-15 17:16 · Score: 1, Interesting

My main spam related problem is that I can't send mail from my residential IP to aol.com, rr.com, and some others. Will this provide such a great spam filter that those ISPs will stop blocking me ? I suspect not, because I suspect those people just want to force me to get a higher priced fixed IP. (I suspect RR doesn't care if I spam for a little side income, as long as I pay more and don't do it enough to cost them more than I am paying.)

Besides my residential IP with its dyndns name, I maintain two other mail servers for small businesses (less than 4 employees each). Is there any reason I should employ this solution ? Do I get anything I don't get by running spamassassin at home and for my clients ?

Come on, sell me on this. However, you have an uphill battle, because my main problem isn't spam, it's the people who treat me like a spammer. (Some analogies to terrorism and John Ashcroft may come to mind.) It costs me and my clients the same whether the network is slow from a spam flood (usually virus related) or not. The human attention factor is not large, with filters applied and updated.

Re:Not helpful to me by Piquan · 2003-12-15 17:27 · Score: 1

I use a couple of sendmail rules to relay mail to aol and hotmail through my ISP, and everything else goes directly. Maybe you could do something like that.
Re:Not helpful to me by Anonymous Coward · 2003-12-16 08:26 · Score: 0

You could implement it, or, you could just turn it on in SpamAssassin once it's included (which will be shortly) since your already using SpamAssassin. Implementing it yourself however will save bandwidth, because the SPF check will be performed before the payload of the email is received, whereas SpamAssassin performs it's check after.
Re:Not helpful to me by gconnor · 2003-12-16 12:37 · Score: 1

There are some places that block residential IPs... usually this is based on the reverse DNS (feed your IP in and try to get back a name) - it contains numbers like d22-164-99 or whatever. One solution is to set up your home server to send out through your ISP's server which should have a proper reverse DNS without so much numbers. Some smaller ISPs will allow you to set up "real names" for your reverse DNS, most will charge a small fee for this since they need to issue static (dedicated) ip address to do it. Probably the best route is to relay from your home server to the ISP's mail server to be sent out. SPF will probably help this problem, but it will probably be a long time before it gets to that point. People block IPs based on "dynamic looking reverse DNS" mostly because there really is no better way to tell "legit" from "spammy". If there is a verification scheme that can detect forgery based on DNS and it gets accepted widely, some ISPs might ease up on their other requirements, but that will take time. With your small business servers, you might not need SPF yet... if spammers start forging your name and pretending to be from your domain, it can be added later as needed. It's about as easy as setting up MX records so hopefully people will just start doing it as a matter of course. If SPF gets accepted more widely, spammers will start to move off of the SPF-protected domains and spoof the other unprotected domains, so there will be a lot of shifting around before there is a noticeable impact on spam.
Re:Not helpful to me by Anonymous Coward · 2003-12-16 18:49 · Score: 1, Insightful

> Come on, sell me on this

Assuming SPF is successful, there would be less need for manually-created "residential" IP lists that ban you. It would all be automatically handled by your DNS provider. (However, this put an anti-spam burden on DynDNS etc)

So this is the way out for you.
Re:Not helpful to me by Anonymous Coward · 2003-12-19 11:11 · Score: 0

Use your friggin ISP's mailserver. Everyone has to accept joejob spam because a handful of tech geeks think it's cool they can run their own mailservers on their dsl/cable connection? If you're technically inclined you will find a way to send your mail indirectly.

Adoption Rate by jhunsake · 2003-12-15 18:00 · Score: 4, Informative

I know I'm going to put the SPF records in as soon as I get a chance, but these statistics aren't terribly optimistic so far:

http://www.infinitepenguins.net/SPF/register.php

This system serves to monitor the take-up of SPF. So far, 274 domains with SPF records are known.
As yet, only a count of registered domains is displayed; more analysis tools will appear once the number of domains increases.

Of these:
84 parse cleanly
0 parse with warnings
173 parse with errors
17 are yet to be checked by this system

Re:Adoption Rate by joostje · 2003-12-15 22:36 · Score: 1

84 parse cleanly
173 parse with errors
Although I'm not sure, I assume this is due to changes in the SPF format. I created my records many moons ago, and now I (nor the automated checking tools) don't even recognize the new format.
With the freezing, the errors should be getting less soon.
Re:Adoption Rate by joostje · 2003-12-15 23:04 · Score: 1

173 parse with errors
Ahm, I've just added my domain, was easy to do with the SPF wizzard, you just answer the questions, and it tells you what to enter into you bind config files. (And even explains what it means you're adding). But then, going back to the checking tool you mention, I get this:
Domain: uea.org

Record Found: "v=spf1 a a:co.uea.org include:cistron.nl -all"

Errors: Malformed or truncated domain 'co.uea.org' in 'a' declaration in rule part 3 (a:co.uea.org)
Malformed or truncated domain 'cistron.nl' in 'include' declaration in rule part 4 (include:cistron.nl)
No Warnings
No Notes

So I guess the checking tool is somewhat too strict (or the wizzard is sloppy.).
explains the reported errors, though.
Re:Adoption Rate by wayne · 2003-12-16 08:26 · Score: 1

The registry was created only a few days ago and has doubled in size since I last checked. The SPF specifications were frozen only a week or so ago, and most of the parse errors are due to early adoptors not updating their records. The SPF libraries are now being updated to allow for some backwards compatibility, but I suspect that more of the records will just be fixed in the comming weeks.

--
SPF support for most open source mail servers can be found at libspf2.
Re:Adoption Rate by ahodgson · 2003-12-16 08:26 · Score: 1

That's just people who register in their database.

I've published SPF records for >100K domains for my employer, and that's just one sample.
Re:Adoption Rate by wayne · 2003-12-16 08:56 · Score: 1

So I guess the checking tool is somewhat too strict (or the wizzard is sloppy.). explains the reported errors, though. I just checked your domain name again, and the registry now reports your SPF record as being correct. As I mentioned above, this registry is new and bugs are obviously still being fixed. If you have other problems, please let me know. (I'm not involved in the registery, but I might be able to answer your questions.)

--
SPF support for most open source mail servers can be found at libspf2.
Re:Adoption Rate by WuphonsReach · 2003-12-16 16:03 · Score: 1

Well, I looked into adding the TXT records to my domains, but register.com doesn't provide a method for adding the TXT records if you use the domain manager tool.

Soo... looks like they'll need to be prodded into adding that functionality to their DNS tool.

--
Wolde you bothe eate your cake, and have your cake?

AOL and hotmail don't gain? by jtheory · 2003-12-15 18:15 · Score: 2, Interesting

Imagine how this might increase AOL's or hotmail's network traffic, while they gain nothing from it.

Well, they do gain, actually -- if the plan works, it will blot out quite a lot of spam. AOL and Hotmail spend an astronomical amount of money dealing with spam in the current situation (it doesn't help that lots of spammers forge AOL or hotmail return addresses... I'm sure those bounces crank out the bandwidth required). If they need to pay for more bandwidth and more servers to support SPF, I have to imagine that will be much cheaper than the manpower they have to support to fight the problem now.

Besides, how much extra bandwidth is really involved? Wouldn't it work like other DNS records, and be cached all over the place?

I don't know enough about the technology to properly address your second point... but I think because we're dealing with DNS servers here (instead of needing to contact the mail servers) this may actually work out. Sure, some people run mail servers from home, etc., but DNS is usually provided free by an ISP; there are also free DNS hosts.

Either way, I'm rooting for it. Spam is killing email.

--
There are only 10 types of people: those who understand decimal, those who don't, and, uh, 8 other types I forget.

How does this reduce spam in any shape or form? by onomatomania · 2003-12-15 18:31 · Score: 3, Insightful

Okay, I am not trolling here, I'm serious. This plan will be moderately successful at preventing joe-jobs on unwitting victims. If you control the DNS for a domain, you can say who is allowed to send mail for that domain. Therefore, if a spammer attempts uses your domain in the "From:" header then it will only be delivered to those hosts that are NOT checking the SPF records. That's an important distinction, because getting everybody on the planet to do something is very hard, so this will never completely wipe out the possibility of joe-jobs. And there are the possible negative effects here, for example employees not being able to send company email while on the road without hassle.

But that aside, how does it reduce spam? The spammers will always be able to find a domain to stick in the "From:" header. They can choose to use a domain that they do not control that has not yet added SPF to their DNS or they can choose to use a domain that they control. In either case it's trivial for them to get their mail from their system to yours, and that's all that they really care about anyway -- the "From:" header has always been meaningless to spammers anyway, it's not like they would be forfeiting the ability to receive replies or something.

Note that in the case of using a domain that they don't control, we're back to the issue of "until everyone on the planet does this, there will always be some domain somewhere that can be forged." And even should those run out, spammers can just register anything for $7 a year, or less for bulk registrations. (They already do this when they're playing hosting tricks, to bounce you around from one host to another.)

Now, you might say that at least with this implemented you could discover what those domains are that the spammer is registering for use with his spamming. That is true. But, we've had the concept of a blocklist for ages, that's nothing new. Everyone has ranges of IP addresses that they won't accept mail from, and some very kind organizations have even maintained lists of "bad IP addresses", so you might expect a similar thing to happen with domain names. But all you have to do is look at the current state of blocklists and you'll know this doesn't buy you much. We already have blocklists, and they're riddled with problems. You're back to playing whack-a-mole with the spammers. They make a spam run with example.com, you block example.com; they make another run with example.org, you block example.org. You're always one step behind, while the spam piles up in your inbox. You might make the point that this inconveniences them, but you have to realize how many domains there are out there that are available for forging. The SPF-protected domains will be the vast minority of all domains for the forseeable future.

So, in summary: This might be moderately effective at preventing joe-jobs. It will not make a significant change, however, until everyone on the face of the earth that's not a spammer both updates their DNS and updates their MTA software to check these records. The likelyhood of this happening any time soon is quite small. And even if this were to happen, the spammers would still be able to deliver piles and piles of garbage to your inbox though domains that they control. You're back to blocklisting, which we've had for quite some time now.

So, I ask seriously, what does this do to combat spam that is really all that significant? I applaud any developments on the antispam frontier, but let's not get too carried away with visions of this somehow "plugging the insecure SMTP hole", or anything remotely resembling it.

Re:How does this reduce spam in any shape or form? by Aniquel · 2003-12-15 19:29 · Score: 1

I largely agree with you, but keep in mind that SPF uses DNS; we don't need every host on the planet to adopt it, only those organizations that run mail and DNS servers.

The big ones (yahoo, hotmail) are on board already; that's going to put some fierce pressure on ISPs to use this.
Re:How does this reduce spam in any shape or form? by onomatomania · 2003-12-15 20:13 · Score: 1

Right, I should have said "domain owner and/or DNS server operator and/or mail server administrator" whereever I said "everyone" since that's kind of a mouthful. :-)
Re:How does this reduce spam in any shape or form? by ccarr.com · 2003-12-15 21:57 · Score: 1

You don't need everyone on the planet, just enough so that you can start filtering or bouncing mail from non SPF domains.

--
I don't know half of you half as well as I should like, and I like less than half of you half as well as you deserve. BB
Re:How does this reduce spam in any shape or form? by Piquan · 2003-12-15 22:49 · Score: 1

I use SpamAssassin. The SPF record could be used to alter the score.
Some people use RBLs with a time lag. SPF-passed mail could bypass the lag.
It's another step.
Re:How does this reduce spam in any shape or form? by joostje · 2003-12-15 23:10 · Score: 1

The big ones (yahoo, hotmail) are on board already; that's going to put some fierce pressure on ISPs to use this.
Are they? I don't see any txt records for hotmail nor yahoo, and the checking tool" doesn't see them eighter.
Re:How does this reduce spam in any shape or form? by Godeke · 2003-12-16 02:59 · Score: 1

Ok, so you think this won't work. What do *you* suggest be done in its stead? Although this is a minor step forward, the major steps forward (like, oh I don't know, requiring encrypted passwords (SPA) before a send, which is already available) seem to never take hold because of some downlevel client user whining about lack of access.

Your argument regarding DNS entries that spammers control is valid, but weak in that *any* solution must allow a domain holder to still send mail. What it *does* do, is stuff them back into more easily filtered domains boxes, rather than joe jobbing my domain name. Your complaint about it not slowing spam is also not completely valid: I
don't give a flying fruit loop if someone else implements it as long as it prevents domains *I* am responsible for maintaining being joe jobbed. If others aren't that bright, it doesn't *worsen* my situation. It makes it better. Those who don't implement it will be made to look worse and worse as they become the primary targets of joe jobs, and eventually peer pressure will cause them to implement.

Since it is easy to implement, I don't see why *not* to move forward with this. Your argument is like saying "Well, it's not a good as an armed guard, keycard security and name badges, so why should I lock my front door?" Perhaps so people don't walts in from the street and steal your TV?

--
Sig under construction since 1998.
Re:How does this reduce spam in any shape or form? by Eric+Savage · 2003-12-16 03:36 · Score: 1

This is intended to plug a hole, that's all. It's not designed to solve the whole problem. Its a simply solution that makes existing technology better with little investment. Also I think you are underestimating the indirect effect it has. Look at your spam box, where do probably half of them say they come from? Hotmail and Yahoo right? You can't block Hotmail and Yahoo though because they have millions of legitimate users. With this type of system you are forcing them to use smaller and smaller domains which are easier to block. Also, once this reaches critical mass (which is not 100%), you can start scoring against messages from systems that don't implement it.

I support this system because it is a pretty painless partial solution. It certainly isn't a silver bullet but it's a great step.

--

This is not the greatest sig in the world, this is just a tribute.
Re:How does this reduce spam in any shape or form? by dgp · 2003-12-16 07:55 · Score: 1

If you control the DNS for a domain, you can say who is allowed to send mail for that domain. Therefore, if a spammer attempts uses your domain in the "From:" header then it will only be delivered to those hosts that are NOT checking the SPF records. That's an important distinction, because getting everybody on the planet to do something is very hard
nice summary. SPF becomes useful long before everyone on the planet starts using it. In fact, once you get yahoo.com, aol.com, hotmail.com and a hundred other top domains to add one simple TXT record, you have dealt a major blow to spammers. SPF makes the From: address mean something.

And there are the possible negative effects here, for example employees not being able to send company email while on the road without hassle.
the SPF site addresses this. The answer is client authentication, SMTP AUTH with SASL.

But that aside, how does it reduce spam? The spammers will always be able to find a domain to stick in the "From:" header.
By only allowing mail from domains with DNS records, the spammers are limited to existing domains. So they start picking domains at random. Now thats been working well. Once SPF gets adopted on the top ten domains, many spammers will have to work harder to find a good domain to use. As the system catches on more and abuse@example.com starts getting more email, example.com will add an SPF record. The adoption will cascade and the amount of SPF-free domains will dwindle.

They can choose to use a domain that they do not control that has not yet added SPF to their DNS or they can choose to use a domain that they control.
I am working under the assumption that spam delivery houses have a genuine interest in remaining anonymous. If SPF forces spam delivery houses to use domains which they control, then suddenly the whois database provides a much stronger link back to the original spammer. Now there is much more of a papertrail, and that adds a lot of power in getting spammer shut down.

In either case it's trivial for them to get their mail from their system to yours, and that's all that they really care about anyway -- the "From:" header has always been meaningless to spammers anyway, it's not like they would be forfeiting the ability to receive replies or something.

exactly. SPF makes the From: address mean something. One technique that Ive seen spammers start to use is to spam me from email addresses of my friends! This gets around even whitelists. With SPF, this cannot happen.

Note that in the case of using a domain that they don't control, we're back to the issue of "until everyone on the planet does this, there will always be some domain somewhere that can be forged."

The important day is when your organization can start dropping email on the floor that does not have a From with an SPF record. I assert that this day will come long before everyone on the planet uses SPF. If unix admins can be convinced that SPF is a good idea and the mail delivery agents all have SPF support in their next release, the takeup rate could be amazing.

And even should those run out, spammers can just register anything for $7 a year, or less for bulk registrations.
paper trail.

Now, you might say that at least with this implemented you could discover what those domains are that the spammer is registering for use with his spamming. That is true. But, we've had the concept of a blocklist for ages, that's nothing new. ... You're back to playing whack-a-mole with the spammers. They make a spam run with example.com, you block example.com; they make another run with examp
Re:How does this reduce spam in any shape or form? by ahodgson · 2003-12-16 08:29 · Score: 1

Once any significant percentage of domains use it, we can just not accept mail from the domains that don't.
Re:How does this reduce spam in any shape or form? by wayne · 2003-12-16 08:48 · Score: 1

Okay, I am not trolling here, I'm serious.
You raise some good points, I'll try to address some of them.
SPF is not designed or intended to stop all forms of spam. In fact, I see SPF doing as much to protect people from phishing and joe-jobs as reducing spam. UBE is not the only form of email abuse.
SPF gives domain name owners a voice to communicate which IP addresses, if any, are authorized to send email using the domain name. Anyone who wants to listen to what the domain owner has to say is free to pay attention to this.
Widespread deployment of SPF may well help RHSBLs, but SPF itself has never intended to determine if a sender is good, bad, or ugly. It will also help with whitelists that use email addresses (as opposed to IP address whitelists.)
Not all email is equally important. I think SPF has good potential to be used by organizations such as financial institutions to reduce the chance of being phished and to damage the institution's good name.
While I can't see many major ISPs publishing restrictive SPF records, I can see them checking SPF records in order to help protect their customers. SpamAssassin will most likely have support for SPF in the 2.70 release and there are a couple of other anti-spam filters that will be checking it also.
SPF is a good step forward. It is not the only step that needs to be taken.

--
SPF support for most open source mail servers can be found at libspf2.
Re:How does this reduce spam in any shape or form? by GreyFish · 2003-12-16 08:57 · Score: 1

SPF makes the From: address mean something. No! it dosn't! SPF makes the SMTP Envelope MAIL FROM: mean something. If it checked the From: it would break every mailing list on the planet.
Re:How does this reduce spam in any shape or form? by skookum · 2003-12-16 09:04 · Score: 1

If yahoo and hotmail incorporate this and set it such that only the main/official SMTP servers can send out mail that is from yahoo.com or hotmail.com, then how many people is that going to screw over? (That's an actual question, not a rhetorical one.) I'd imagine there are probably lots of folks out there that set their From name in their mailer to their Yahoo or Hotmail address, but don't use the official SMTP servers. They would all be forced to stop using their email app and start using the webmail interface instead since the free-mail plans don't usually include POP/SMTP. You can get around this with programs like YahooPops, and I think hotmail has that "web access" or whatever it's called that lets Outlook send and receive on the web, so maybe I'm overestimating the effect.
Re:How does this reduce spam in any shape or form? by Eric+Savage · 2003-12-16 10:23 · Score: 1

It's going to affect alot of people. Since spammers use the same mechanisms that normal people use anything that affects spammers is in some way going to affect normal users. The more similar a normal behavior is to a spam behavior, the more it will be affected, like the header "forging" you mention. While putting a legitimate address you own from a different domain is a legitimate thing to do, its indistinguable at the technical level from a spammer forging an address.

This is an important thing however, because alot of ISPs block port 25 to anything but their own mail servers, and nobody wants to use a transient ISP address. The reply-to header is designed to alleviate this, but I'm not sure if that's a good enough solution. I think the ISPs will have to work together to figure this out, but then again, the ISPs are the ones that feel the worst effects of spam.

--

This is not the greatest sig in the world, this is just a tribute.
Re:How does this reduce spam in any shape or form? by gconnor · 2003-12-16 12:52 · Score: 1

It's going to be a long road, and I think SPF is just a first step, but it's an important first step. I would describe it as "necessary but not sufficient" to stopping spam.

Here's how I think things might go if everything works out.
1. Some key large domains publish SPF, and a few large ISPs start checking for it.
2. Some of the mail that is forged using the big names is downgraded or dropped.
3. Spammers start to spoof the addresses of other names instead.
4. Pressure is on for the owners of those other names to publish SPF.
5. Some key organizations stop accepting messages without proper SPF credentials, like private companies or universities, who would rather lose some mail than deal with forged messages.
6. More pressure is on for domain owners to publish SPF to get their messages through everywhere.
7. At some point it becomes a well-known standard like MX, and everyone just does it as a matter of course.
8. At some point, DNS registration sellers get involved and won't register your domain if the SPF records are not right, like they check your NS and SOA info now.

Of course there will be other important things that need to happen to stop spam, and stomping out all forgery will take a long time, but that just means we should get started right away! :)

Exactly by 0x0d0a · 2003-12-15 20:00 · Score: 2, Insightful

From a security standpoint, this is a dumb idea. Really dumb. It's stupid, open to a ton of attacks, and does diddly about spam. However, it's probably going to get some popularity for the following reasons:

* Folks hate spam, and will glom onto anything that claims to reduce it with the gullibility of a cancer victim being scammed by a faith healer.

* It's easy for IT folks to implement. The CIO can say that he "implemented an initiative to reduce the most frequent user complaint, saving the company N dollars". He doesn't give a damn about whether he actually *accomplishes* anything, just whether it looks good.

* SPF is a pet project of someone. Obviously not someone who's a security person, but someone.

* There's money in it for consultants. The firewall craze made many many people very much money. The promise of a *new* even less useful system that can be used to pry money from companies is quite appealing.

Things that *could* reasonably be done to combat spam:

* Limit email amplification.

* Require one of a number of pay or auth systems for email.

* Whitelisting

* Allowing clients to publish rules to mail servers (so that a client whitelist, for instance, would allow a server to avoid soaking up any bandwidth at all...this would be a useful set of IMAP extensions).

Frankly, unless a bunch of engineers get together and put out a *useful* RFC (i.e. not this crap) a la PNG, there's probably going to be an industry consortium that decides what to do. And standards committes have an awfully low rate of getting-it-right.

--
May we never see th

Re:Exactly by bonniot · 2003-12-15 21:09 · Score: 1

From a security standpoint, this is a dumb idea. Really dumb. It's stupid, open to a ton of attacks, and does diddly about spam.
Could you back up those claims with facts?

--
Watch great movie opening scenes!
Re:Exactly by metamatic · 2003-12-16 05:19 · Score: 1

Actually, whitelisting is even dumber than SPF and won't work. The other ideas are good, though.

SPF was shot to ribbons on the IETF ASRG list, but obviously someone decided to go ahead with it anyway.

--
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
Re:Exactly by dozer · 2003-12-16 06:45 · Score: 1

How did you ever get +4 insightful?? Your post has a lot of strong words and zero substance. You spend some time complaining about SPF without backing anything up. Then you offer some rather tired suggestions:

Limit email amplification.

Great. How? People have been trying to do this since the mid-90s with little success. Remember that amplification is a very valuable feature of email ("send this message to everybody in marketing").

Require one of a number of pay or auth systems for email.

Ha. There are 5 companies out there trying to do exactly this (and incidentally become the for-pay email monopoly). Do you want to start company 6? All of them are doomed to failure, of course, because they all involve a central authority.

Whitelisting

Only works in a few very contrived situations (usually involving second email addresses or non-business-critical functions). Whitelisting, if adopted globally, would largely render email worthless.

[a set of IMAP extensions for clients to query whitelists]

Now this is simply insane. It has all of the problems that SPF does (think about it: every single one, plus it's much MUCH more bandwidth intensive). Then it introduces some of its own (like, too much shared information, IMAP's poor security record, etc.). You didn't think about this one very long, did you?

So, smart guy, how about some substance? Exactly what is wrong about the SPF RFC? Be sure to use more descriptive adjectives than "dumb", "useless" and "crap".
Re:Exactly by Zeinfeld · 2003-12-16 08:27 · Score: 1

Actually, whitelisting is even dumber than SPF and won't work. The other ideas are good, though. SPF was shot to ribbons on the IETF ASRG list, but obviously someone decided to go ahead with it anyway.
The people doing the shooting down made no substantive arguments and instead engaged in a series of vitriolic personal attacks against the people making constructive proposals. Basically some folk who had a position in the anti-spam world wanted to make sure nothing challenged it.
IETF is pretty much an irrelevant organization these days (ASRG is actually in IRTF but same difference). The power and influence lies with the large ISPs and the major Internet companies. At this point so does most of the expertise. There are very few of the original Internet pioneers left in the IETF, expertise is a property of individuals, not organizations.
That said the SPF list is not a particularly resonsive or accountable list. Basically there is no mechanism for accountability. Those of us asking for a simpler design have no real say in the process. But at least doing it this way we don't have to wait three years for the IESG to make arbitrary changes to the spec for reasons that defy reality.

--
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
Re:Exactly by 0x0d0a · 2003-12-16 15:59 · Score: 1

Could you back up those claims with facts?

Sure.

First, this is nothing more than an authentication system. It's designed to allow a server to authenticate itself as a trusted source for a domain's email. However, the designers chose to use DNS as a transport mechanism. Not a good idea. DNS is designed to be lightweight and low latency, not to be secure. It's pretty easy to spoof DNS responses. Plus, DNS data tends to get cached. All you need to do is spoof a response, the nameserver's cache is poisoned with false data, and the next N emails (until the cached data expires) are accepted as valid.

Second, this system relies on having everyone implement such functionality. Spammers don't give a damn about return addresses, so they can send email with a from address at any domain. The annoying and ineffective attempts at stopping all open mail relays on the Internet illustrate the failure of this model. A security system that relies on correct implementation over the full Internet to function properly will not work in real life.

Third, this fails to deal with throwaway domains. The authors waffle a bit about them, and finally come out and admit that more mechanisms are required. Dammit, if we had a good PKI trust-ranking system (which is the sort of thing that they are requiring to fix their failings) we wouldn't need this system at *all*, since we could simply sign email and have trust rankings for users.

Enough about the bad design: other reasons I don't like it include:

* The authors have made a decision to make it really annoying to send email from a machine, and have to work with your ISP just to have a mail server. There are plenty of more solid antispam proposed mechanisms that do not place restrictions on who runs what servers (pay-per-email or pay-per-initial-email, PKI systems). This is much more in line with the way the Internet works for most services.

* There is a supposedly trusted authentication system being spread across the entire Internet over an insecure transport protocol.

* DNS caching can make moving an SMTP server or setting up a new one take a significant amount of time.

* IP-based auth isn't a great idea anyway, for a number of reasons. The authors claim that it isn't a huge issue, because IP spoofing is harder (I disagree -- things like Mobile IP have made it harder to *block* IP spoofing).

* Users have no control over what gets blocked. If I *want* to receive email of a particular type, I can't. Two ISPs (sending and receiving) are the ones that determine what mail I can receive). This is perhaps acceptable within a company, but annoying and goes against traditional Internet structure.

* It does nothing to avoid compromised end user machines.

* It does nothing to deal with throwaway accounts.

* It does nothing to deal with misconfigured servers.

--
May we never see th
Re:Exactly by 0x0d0a · 2003-12-16 16:13 · Score: 1

See my comments to an earlier respondent to my post.

--
May we never see th
Re:Exactly by 0x0d0a · 2003-12-16 16:17 · Score: 1

Actually, whitelisting is even dumber than SPF and won't work.

Used alone, it is extreme, though I know a number of people that happily use whitelisting in instant messaging environments.

The biggest use of whitelisting, however, would likely be in conjunction with other solid antispam methods.

--
May we never see th
Re:Exactly by Linux_ho · 2003-12-17 08:24 · Score: 1

How did you ever get +4 insightful?? Your post has a lot of strong words and zero substance. You spend some time complaining about SPF without backing anything up.
It's a troll. Intentionally inflammatory, knows he/she is wrong, just trying to provoke a response.

--
include $sig;
1;

Will make filtering much easier by Micah · 2003-12-15 20:31 · Score: 2

Some people here have complained that this might not work because everyone on the planet would need to use it and even then spammers could use their own domains.

Certainly it's true that nearly everyone will need to get on board for this to work. Fortunately, it should be an easy update on both the MTA and DNS ends.

The real advantage here, I think, is that it will make filtering and blacklisting much easier. Instead of trying to filter on 18 zillion weird rules and scads of IP addresses, some of which may have some valid users, you just need to filter on domain names.

For this to work, we will need one or more trustworthy registries of bad domain names. And it should probably be distributed, with a way to continually update it by automatically propagating the list of bad domains to all clients. There should be a way to get a domain into the blacklist very quickly if anyone receives spam from that domain.

Alternatively, a system could be in place to treat all new domains as bad by default. That has obvious problems though -- how would you get your domain trusted? Would it require a VeriSign like identification process? I would oppose that -- I think people should be able to buy domains and freely run email servers on them without paying some central "authority."

My biggest concern with this idea is that I run a domain where I give out POP email addresses to people. I'm still trying to figure out how that will affect me.

Re:Will make filtering much easier by SpaceLifeForm · 2003-12-16 06:49 · Score: 1

My biggest concern with this idea is that I run a domain where I give out POP email addresses to people. I'm still trying to figure out how that will affect me.
Why is this a problem? Are not those users doing SMTP to *your* MTA running under *your* domain?

--
You are being MICROattacked, from various angles, in a SOFT manner.
Re:Will make filtering much easier by Micah · 2003-12-16 08:44 · Score: 1

No, my users are all over the Net, using their local ISP mail servers.

Guess I could install a POP-before-SMTP type system.

Ok - RFC ? I don't think soon by MerlynEmrys67 · 2003-12-15 21:48 · Score: 2, Interesting

This should become an informational RFC - it could be published within a month. As experimental - I don't see it surviving the IETF Last Call process. Way too many operational people that don't like mucking with DNS records - too man spammers that wouldn't like it.

If they couldn't get consensus inside the IRTF's spam working group, what makes them think they can get it in the IETF community at large (I love the note to the RFC editor - HA)

--
I have mod points and I am not afraid to use them

Re:Ok - RFC ? I don't think soon by WuphonsReach · 2003-12-16 15:32 · Score: 1

More likely, one of the big ISPs (AOL, Yahoo) will require that you configure DNS records in order to send e-mail into their network. Initially (during the adoption period), they'll probably only use the information to score e-mails as being spammy/hammy. After the adoption period, they'll start using the information like a blackhole list and just drop the connections.

The MTAs (postfix, sendmail) will probably implement one of the (4) solutions and admins will start using it by default.

I doubt we'll see any organization (IETF, ASRG, IRTF, random-4-letter-acronym) say "go" on this. Instead, it will be like a snowflake that rolls downhill and turns into an avalanche.

Good chance that end-users (DNS admins) will have to comply with 2 of the proposals in order to meet the demands of the larger ISPs. Fortunately, DNS records are free, just a PITA to setup (grin).

--
Wolde you bothe eate your cake, and have your cake?

how do I get all the server names? by Artifex · 2003-12-15 23:32 · Score: 1

My home ISP is SBC, and while they may let me send mail through mail.swbell.net, chances are that's not the name of the outward-facing ring of mail servers they have (I am assuming they actually built an enterprise-class mail service, of course, which is a lot to assume about SBC).

How do I get all the names of their outward facing servers, which may not leave my original mail.swbell.net in headers as my first stop?

Speaking of editing headers, what's to stop the Asian remailers from having their servers generate fake preceding headers using legit server names and faked user names? Or even real user names, when they find them or want to get someone in trouble? If they claim to be downchain of a legit sender, what help is SPF?

--
Get off my launchpad!

Re:how do I get all the server names? by hattmoward · 2003-12-16 02:43 · Score: 1

After SBC implements SPF (good luck, SBC is a great company to work with;), you can bring up a terminal, and run something like 'dig TXT sbc.com' and copy their 'v=spf ...' record from there. This should provide an accurate description.

I'm not quite sure what you're saying at the end there... A remailer will have to be on the SPF record for the domain they're trying to deliver for, yes, and they'll still get through fine, but if there is also a destributed blacklist system (or responsive DNS registrars, but that's another battle), the original sender domain will be dropped anyway. SPF is a simple authorization system, with extremely weak authentication, but we are taking baby steps, right?
Re:how do I get all the server names? by gconnor · 2003-12-16 13:06 · Score: 1

You can refer to your ISP in terms of "include=" (basically, this tells people checking your record that they should check SBC's record as well). You can add multiple includes for the ISPs you use.

Regarding headers, SPF is able to drop the connection based only on the SMTP commands, before you have sent any headers. Headers are part of the DATA step in the transaction which is like 4th, MAIL FROM step is 2nd.

If you're not adding it directly to the mail server, you can analyze headers after the fact, but you would only want to trust headers from your own server - your own server knows the other guys true IP since they have a two-way conversation.
Re:how do I get all the server names? by Artifex · 2003-12-16 15:38 · Score: 1

Oh, I see, now it makes sense - someone can't really pretend they're just passing on the mail from a legitimate host, because that legitimate host wouldn't relay through someone else without listing it. I guess that was obvious to everyone but me :)

--
Get off my launchpad!
Re:how do I get all the server names? by gconnor · 2003-12-16 16:41 · Score: 1

Not to worry, smtp is naturally confusing -- it's designed that way :)

Spammers definitely take great advantage of how confusing all those extra headers are, and the smtp transaction has a couple of steps to go through before the headers even arrive. Luckily most mail servers will record their part of the transaction, the HELO command is "received from", the Mail From is the Return Path, etc. But it's still kind of tortured.

Summary for mail & network admins by CrystalFalcon · 2003-12-16 00:22 · Score: 3, Informative

If your MX record is also the IP(s) used for outgoing mail, as in my case, all you have to do is add this line to your DNS:

[domainname] IN TXT "v=spf1 +mx -all"

That's it. That's really it, at least for publishing your permissions. So simple I already did it for my domains.

This sounds a lot like RMX by dkt5 · 2003-12-16 05:17 · Score: 1

http://www.ietf.org/internet-drafts/draft-danisch- dns-rr-smtp-03.txt

http://www.danisch.de/software/rmx/

Re:This sounds a lot like RMX by wayne · 2003-12-16 08:31 · Score: 1

The reason why SPF sounds like RMX and DMP is because SPF is a hybrid of RMX and DMP. It is *much* further along with implementations and deployment, however. It is also, IMHO, much better thought out. The latest versions of RMX are actually much more like SPF was 6 months ago than like RMX was 9 months ago.

--
SPF support for most open source mail servers can be found at libspf2.

How to make this work by clenhart · 2003-12-16 06:00 · Score: 1

MTAs should block mail where the FROM does not match the originating server (i.e. act like "v=spf1 a mx ptr -all"). SPF should be used to configure domains to extend this rule. If you have a weird setup, you could add an SPF record to allow additional hosts to send email on your behalf that are outside your domain.

Without this default ("v=spf1 a mx ptr -all"), spammers will be able to find domains that don't use SPF.

ASRG SPF pointers; not shot to ribbons by TimFreeman · 2003-12-16 06:54 · Score: 2, Informative

The parent says

SPF was shot to ribbons on the IETF ASRG list...

but offers no pointers to allegedly valid objections. Here are some pointers into the ASRG discussion. I didn't see any compelling criticisms of SPF there. The criticism that SPF "is not a serious technical effort" is odd, given that an implementation exists.

SPF is too limited by Anonymous Coward · 2003-12-16 08:26 · Score: 1, Insightful

SPF is all well and good if you intend to have all of your users send mail through your short list of approved hosts. Some places can swing this. Good for them.

I have run a vanity domain for myself and a handful of friends from high school since the mid 90s. They use their vanity accounts to get mail, since it forwards to wherever they happen to be now. I want them to be able to send mail as those accounts, but SPF won't let that happen.

The SPF-approved answer is "have them relay through you with something like authenticated SMTP". Yeah, right. I'm not about to open up a bunch of extra crap on my mail server for a handful of people who run around and do stuff all over the world at times.

With SPF, I either cut a huge hole for my entire domain for every network one of my friends happens to inhabit, or I screw all of them by limiting it to my own personal network. There's no middle ground, and it puts me in a bad spot.

I'd much rather say user A is allowed to send mail from network A, user B from network B, and so on. That would let my friends send mail from a few designated places without any trouble. For the true globetrotters, I'd just disable all network restrictions on their address at the expense of possible spoofing.

Everyone seems to be hung up on things like SPF because it uses DNS, so they don't have to write a new protocol, servers, or clients. That makes it very tempting, and that's why this article is here.

Re:SPF is too limited by Anonymous Coward · 2003-12-17 10:54 · Score: 0

The "exists:" mechanism, in combination with the macro syntax, is designed to let you do exactly what you described. You could have found this out in less time than it took you to write that objection.

The wizard was designed to help configure the 99% of simple cases. You fall into the other 1% of people who would benefit from reading the actual specification.

Law of Beta strikes again by Wechsler · 2003-12-16 08:56 · Score: 2, Informative

The registry was only actually completed today; the parser wasn't fully operational before that (it was just online for testing).

Unfortunately some of you caught the parser while it was buggy... it *should* be fine now.

It's also correct that some of the records were produced before the standard was finalised. All these bugs should now be out of the system (I'm going to regret saying that)...

Re:Law of Beta strikes again by Anonymous Coward · 2003-12-16 09:18 · Score: 0

You're a fruity looking piece-of-shit, if I've ever seen one.

these statistics aren't terribly optimistic ? by Fudge.Org · 2003-12-16 08:57 · Score: 1

It's probably not obvious (see also: article on /.) but those stats are voluntary ones based on people that -might- have read the mailing list that -might- have put something into a web form.

It is certainly not exhaustive.

--
http://fudge.org

Combines well with Bayesian filtering by Alderete · 2003-12-16 09:36 · Score: 1

A lot of folks have commented that this won't do anything to reduce spam, because spammers will just register new domains and send from those.

But, those new domains won't have any data in people's Bayesian corpus; they will be neutral in affecting spam probability scores.

On the other hand, domains that publish SPF records will be protecting their domain names, which means that those domain name tokens in people's Bayesian corpuses will count positively towards the message not being spam.

In other words, publishing SPF records for your domain can keep your domain from becoming a spam keyword in Bayesian filtering. I think this will likely improve the effectiveness of Bayesian filters overall.

SPF isn't a silver bullet -- and does not claim to be -- but every little bit helps.

Re:Combines well with Bayesian filtering by Anonymous Coward · 2003-12-21 06:55 · Score: 0

Note:

To register a new domain costs money.

Spam is all about money.

Lucas

spf is moronic by mrsbrisby · 2003-12-16 10:57 · Score: 1

this requires too many levels of parsing and difficulty on the client. it also "tells too much", a better answer involves knowing the IP and the return-path and examining:

1.0.0.10._spf.example.com

for a "valid" record. existing RBL-publishing software can be used, existing RBL-scanning clients can be used, we won't need to set up DNS-over-TCP servers, and we don't give any information we don't have to.

note, and this is very important. what we're looking for is what would better be called a "reverse MX"-- something that describes where mail _comes_from_ so users with separate mail-clusters for outbound and inbound can identify themselves
as such.

RR's are dumb anyway...

Advantages:
* utilizes DNS compression for what it's worth
* no fancy parsing required
* no new levels of indirection
* no sendmail-style configuration barf
* doesn't publish anything special

Disadvantages:
* doesn't require thirty pages to describe :)

Spammer software will just get smarter. by Jason+Pollock · 2003-12-16 11:05 · Score: 1

A lot of spam is supposedly coming from compromised hosts. Those systems will probably have a SMTP server on the local network that can be used. It is probably even already configured with the configuration easily found (outlook/mozilla/etc?). So, the software, instead of running it's own SMTP server, looks up the configured one and uses that, with a made-up account name on that server.

You still get the spam, it even looks more authentic!

Jason

Re:Spammer software will just get smarter. by Linux_ho · 2003-12-17 08:38 · Score: 1

Well, at least with SPF, joe-job bounces only go to the people who have compromised systems or non-SPF compliant domains. Maybe that will serve as incentive to get their systems/domains fixed.

--
include $sig;
1;

SPF flaw in a nutshell by metamatic · 2003-12-16 12:47 · Score: 1

Like whitelisting, SPF works for a while.

Then the spammers build up a database of SPF information, and whenever they get access to a server to spam through, they make their software use one of the correct SPF-allowed domains for that server, to generate the fake From: addresses.

e.g. they 0wn 1000 Windows boxes of Comcast users, so they look up the SPF info from Comcast's DNS records, and use those domains to generate the fake e-mail addresses when spamming using those 0wn3d boxes. When spamming via their collection of open relays in China, they use SPF-valid From: addresses for those open relays.

In short, SPF provides zero protection once spammers work out how it works and build the necessary database into their spamware, which I reckon will take them maybe a week. That's assuming enough people even deploy it to make it worth hacking around in the first place.

Believe me, I wish something as simple as SPF could stop spam. I have no position in the anti-spam world to protect; the stuff is nothing but a nuisance and time sink to me.

--
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak

Re:SPF flaw in a nutshell by NZKiwi · 2003-12-16 13:37 · Score: 1

Yes, but in the world with SPF, under your scenario, Comsat will only publish it OWN smtp servers as allowed (the customer boxes will NOT be published as allowed); thus the owned boxes won't be authorised, and people like me will reject them outright; unless they relay via Comsats own servers (which hopefully causes Comsat to wake up and take notice that their smtp servers have a giant outbound spam issue from certain customers, who they can then block until they are fixed)
Re:SPF flaw in a nutshell by metamatic · 2003-12-16 14:06 · Score: 1

But if we could rely on ISPs to block spam going through their systems, we wouldn't need SPF to start with.

--
GCHQ Quantum Insert installed. If only our tongues were made of glass, how much more careful we would be when we speak
Re:SPF flaw in a nutshell by WuphonsReach · 2003-12-16 15:24 · Score: 1

e.g. they 0wn 1000 Windows boxes of Comcast users, so they look up the SPF info from Comcast's DNS records, and use those domains to generate the fake e-mail addresses when spamming using those 0wn3d boxes. When spamming via their collection of open relays in China, they use SPF-valid From: addresses for those open relays.

Uh, no... in a world with a high adoption rate of any reverse-MX solution (DRIP, DMP, SMTP+SPF, RMX), the spammer can only use a FROM: domain that is valid from the IP address of the 0wn3d box. Reverse MX solutions are merely designed to stop "joe jobs".

Reverse MX solutions answer the following 2 questions:

1. Does domain X have reverse-MX information?
2. Is IP address of server Y that is trying to send me e-mail on behalf of domain X listed?

If #1's answer is "NO", then a spammer can forge that domain's address onto their spam, send it from any machine on the planet, and the destination server can't tell anything useful about whether the FROM: domain is good or not.

If #1's answer is YES, but the server that is attempting to send the e-mail is NOT listed, the destination SMTP server can flag the message as suspect or drop the message into a quarantine or just drop the connection. Entirely up to the destination mail admin as to how they use that bit of information.

If #1's answer is YES, and the IP address matches one of the reverse MX entries, then the destination server has some assurance that the e-mail is not spoofed. Possibly, the spammer has hijacked the companies outbound SMTP server, or broken the DNS record to list their zombie machines. The SMTP server hijack is fixable at a local level (just as you would fix a hacked rootkit'd server). The DNS hijack is something that the BIND/DJB/DNS folks need to fix by improving DNS (DNSSEC?).

Right now, there is *no* way to stop "joe jobs", which means that an admin that takes due care can get tarred and feathered while being perfectly innocent. This closes *one* of the loopholes in the SMTP mail system, but is no more of a cure-all then any other anti-spam solution currently out there. I'd rather see this loophole closed sooner then to just stand around wringing my hands while someone invents the perfect solution.

A nice side-effect of reverse-MX's impact on domain spoofing is that it makes "whitelisting" work *better*, because the destination SMTP servers can have greater confidence in the FROM: domain. (Right now, spammers can just spoof domains to get around domain whitelists.)

--
Wolde you bothe eate your cake, and have your cake?
Re:SPF flaw in a nutshell by sorlov · 2003-12-16 21:49 · Score: 1

You're missing the point. ISPs will have to decide what to do: protect users from joe jobs or control outgoing smap.

Tradeoff between "simple" and "useful" by gconnor · 2003-12-16 13:26 · Score: 1

Early versions of SPF actually had reverse lookups like 1.0.0.10._spf.example.com. I actually like the new TXT format better since it is more flexible and easier to write for most domains.

There is a careful tradeoff between "too simple" and "too complex". I have seen other similar proposals doomed by getting too complex, because they bow to what the users want and then nobody wants to write the code. At the same time I have seen proposals doomed because they were not flexible enough to handle most cases.

If you're looking for "something that describes where mail _comes_from_ so users with separate mail-clusters for outbound and inbound can identify themselves" - SPF is it, exactly. With SPF you CAN do reverse-IP style lookups like DNSBLs do, but for most domains it will be easier to just say "v=spf1 +mx +ptr +include:rr.com -all" which basically means "Allow from my MX, and anything with a verified PTR-DNS name in my domain, plus also allow from rr.com"

Anyway, I think SPF is a good compromise between simple and useful -- it is not too simple to be useless and not too complicated to get adopted. I think so far the plug-ins for SpamAssassin, Sendmail, qmail etc. have taken on the average "a few hours" to write.

Yes, it is similar to RMX, DMP, and MS. For my money, I don't care that much which proposal wins as long as we do SOMETHING. Cleaning up all forged mail is going to take a long time, so we should get started NOW :)

Re:Tradeoff between "simple" and "useful" by mrsbrisby · 2003-12-18 04:22 · Score: 1

statements like that, "v=spf1 +mx +ptr +include:rr.com -all" are dangerous- they add extra complexity to the client which is EXACTLY where we don't need complexity- and they're difficult to be correct. How do you know you didn't accidentally type "... +include:. rr.com -all" -- that's as good as nothing, and isn't that easy to spot.

Such Old News by Slur · 2003-12-17 02:53 · Score: 1

But didn't Coppertone standardized SPF over 20 years ago?!

--
-- thinkyhead software and media

Re:Such Old News by jamehec · 2003-12-29 02:10 · Score: 1

No, SPF has a different meaning in that context. SPF = Sun Protection Factor, iirc.

This SPF is different. Since spam and sunblock are NOT related, no one is gonna confuse the two SPFs.

--
This post made with the Dvorak layout.
"Friends don't let friends use QWERTY"

105 comments