SPF To Be Integrated With MS 'Caller ID' System

An anonymous reader submits "CNET's news.com is reporting 'An ongoing effort to consolidate antispam authentication schemes took a big step forward with the merging of Sender Policy Framework (SPF) and Microsoft's Caller ID for E-mail.' This is potentially good news." For more background, here are three previous mentions of Microsoft's proposed Caller ID-style system.

Comments

would be nice

[mpost4]

if the use can then set up their client to auto reject email that does not pass.

Re:would be nice

[YetAnotherDave]

??

the user can set up their system to reject anything they damn well feel like rejecting.

the hard part is the testing, not the actions taken after testing...

Re:would be nice

[GregChant]

There are similar things: HBO has a system that does this (albeit at the server level); if the return-path is spoofed, the e-mail gets dropped at the MTA. The users never even know they were e-mailed.

Re:would be nice

[Rick the Red]

It would be nice if we didn't all have to switch to Outlook to use this, but since this is from Microsoft I'm afraid the sole purpose of the proposal is to get Outlook mandated as the only legal email program. Send email without this "ID" and you go to jail. Oh, you're welcome to use any email program that supports "caller ID", but due to patent restrictions only Outlook will qualify.

Re:would be nice

[danheskett]

I bet you are wrong.

1. MS wants to be folks regarded as "solving" SPAM.

2. E-mail is declining in value as a business tool thanks to SPAM, and as result MS's products have reduced usefullness. This hurts their core business.

3. SPAM and Trojans/Viruses etc go hand in hand. This is yet another way for MS to prevent their software from being cracked all to shreds.

Contact me by e-mail if you want to seriously wager on the topic. My position is that MS will not in fact make this "outlook" only - but instead end up giving it away as widely as possible for themselves.

Re:would be nice

[ichimunki]

Since it is Microsoft's products that present the biggest risk to the rest of the email-using world when it comes to viruses and worms, it's only fair that they do some serious work on making email more usable somehow. Not that I'll put their software anywhere near my computer, but I'm just saying.

Re:would be nice

[Spruce Moose]

Perhaps the poster is trying to make a point about Microsoft's deal with Ironport to allow "marketing messages" through their spam filters.

See the recent slashdot story, Microsoft Will Sell Whitelist Services For Hotmail

The only good anti-spam solution

[FictionPimp]

Stop using email. Require all communication to come on 11X17 inch plastic sheets sent via fedex. Thats what I do.

Re:The only good anti-spam solution

[NineNine]

I actually don't use email for my business. It's just not reliable. Vendors will ask if they can email me PDF's and other nonsense and I just say "no". We have email, of course, but I don't use it. It's a real hassle. Fax machines and the US Mail work just fine for us.

Re:The only good anti-spam solution

[Da Fokka]

Hail, brother!

We don't use cars at my business. They're too cumbersome. I still prefer the good ol' horse-and-carriage. They can make a mess but doing so in front of the offices of the guys who keep sending us directmail solves that problem as well.

But what is this 'fax' you are talking about?!

Re:The only good anti-spam solution

[I confirm I'm not a]

11x17, eh? I was tempted to point out that "I live in Europe, you insensitive clod!" until I thought... that'll make it even harder! What's the US equivalent of A0, Really f*ing big? And does it have to be plastic - would a precious metal do? ;)

Slightly less off-topic: the Singapore government is proposing to fine spammers up to a $1 (Singaporean) per spam. Granted, it won't stem the tide from outside Singapore, but it's a sight better than CAN-SPAM or anything the EU's proposed.

<thinks>Can't we just exile our spammers to Singapore?</thinks>

Re:The only good anti-spam solution

[Anonymous Coward]

I hope that you at least have an FTP site or WWW submission form? Fax machines are terribly lossy and the US mail is not only slow, but it is also not exactly reliable. I've lost items to the postal service and I've known two long-time postal employees who were not at all surprised to hear it.

Re:The only good anti-spam solution

[BlackHawk-666]

I've also simplified my communications system by using two tins and a piece of string. If vendors want to call they have to buy their own tin and some string and tie it into the string-ter-net. They need to replace the string whenever local cats play with it or after six months, because it rots quickly but I sure as hell don't get spam anymore :-) Time to move in with the Amish communities.

Re:The only good anti-spam solution

[NineNine]

My point is that email is by no means a necessity to do most business. It's often a *huge* time waster for employees, too. Plus, with no paper trail, it's not really good for important communications.

Re:The only good anti-spam solution

[Anonymous Coward]

You racist! I'm calling the ADL!

Re:The only good anti-spam solution

[Anonymous Coward]

Well, you are slightly less annoying than the guys who email me saying, "I have a question--call me," instead of just asking the question. But then you probably send the same sort of crap Priority Mail.

Re:The only good anti-spam solution

[Anonymous Coward]

If vendors want to call they have to buy their own tin and some string and tie it into the string-ter-net.

The vendors that talk out their ass use "sphincter-net".

Good they've merged. Why XML ?

[Space cowboy]

The combined SPF and Caller ID, which has yet to be named, will use XML (Extensible Markup Language) to let Net service providers post IP addresses in the Domain Name System, the giant database that translates alphanumeric domain names like "news.com" into numerical IP addresses for Web servers.

I have yet to see a good reason why XML is the choice for the payload. I'm not really buying the argument that it's easier to shoehorn XML into TXT fields rather than have another tag. Either way, in order to implement the proposal the MTA authors will have to do some work, and I don't think there's much to choose between the two...

I still can't really rid myself of the nagging suspicion that the extensibility of an XML-driven anti-spam system plays into the hands of 'embrace and extend' that MS has used successfully since time began...

On the other hand, getting some authentication that it really came from where it says it came from will be very useful. The corollory is that 'owning' a mail server will become a higher priority for the hacker/spammer coalitions. Look for more attacks on MX machines if this becomes widespread...

Next on the agenda - get everyone to use digitally-signed certificates :-)

Simon

Re:Good they've merged. Why XML ?

[Anonymous Coward]

not only that but xml will slow down the dns lookup since the answer will exceed the udp limit and will have to switch over to tcp.

Re:Good they've merged. Why XML ?

[Allen Zadr]

That's a good point, but I see the eXtensability of XML as the power here. It would be relatively simple to extend the Email-Caller-ID XML specification to include an <spf:details/> tag. Which, would naturally allow for other extensions as well.

Remember, too, that XML is not a Microsoft technology. It's a W3C technology that Microsoft also uses. That's a big difference. If this proposal included a .NET extension to my Mail server, then I'd be suspicious.

My question is: How will SPF or Email-Caller-ID take into account mailing lists? Will this block Emails from my address sent through sourceforge.net's many fine list servers?

Re:Good they've merged. Why XML ?

[mydigitalself]

"I still can't really rid myself of the nagging suspicion that the extensibility of an XML-driven anti-spam system plays into the hands of 'embrace and extend' that MS has used successfully since time began..."

isn't that what the X in XML stands for? you seem to be under the impression that XML is evil because MS embrace it so heavily. ok, i'm putting words into your mouth, but that's what I read nontheless. personally i don't see the difference between RCPT-TO: blah@blah.com and blah@blah.com. both of them require string parsing, perhaps a tad (and i'm talking nanoseconds here) more for the XML. there would be nothing stopping MS from embracing and extending current SMTP fields, in fact i'm sure they do it with Exchange.

so what i'm saying is that XML should not be seen as an issue here. i do take your point about ms's history of extension, but they can do that with or without XML.

Re:Good they've merged. Why XML ?

[Space cowboy]

you seem to be under the impression that XML is evil because MS embrace it so heavily

I'd like to think I'm a *bit* more analytical than that! If you read the link in my post, I do say I'm a fan of XML, I just didn't think it was worth repeating...

My point is that if you establish an open standard which everyone starts to use, and people then start to add things, because they can, because XML parsers ignore what they don't understand, the whole system can become far more complicated (and therefore vulnerable) than it needs to be. I don't think this is a good thing in a system that is (a) critical to the use of the internet as a whole, and (b) responsible for helping prevent me getting spam.

Adding an 'MO' tag to complement the MX tag would nail down what you can do with it. Period. Perhaps I *am* being paranoid, but I think this would be preferable.

Simon.

Re:Good they've merged. Why XML ?

[chaffed]

...Either way, in order to implement the proposal the MTA authors will have to do some work, and I don't think there's much to choose between the two...

Very true. Davaide, the maintainer of Xmail Server still has not decided on any of the new methods.

Rightly so. Everyone is heading in 5 different directions. Of course it will take a power house to set the standard. It just so happens the power house will be MS.

Unless opensource can come together. Mod me as a troll but I do think opensource and come together are words that have not and will not ever be used again in a sentence.

Re:Good they've merged. Why XML ?

[thogard]

Cracker will love the xml format. It turns out that the record size will exceed the UDP packet size for DNS records so they get upsized to use TCP packets.

The thing is how many people allow TCP packets on port 53 on their firewall? There is no reason execpt to talk to your second-dns records. All other cases should be turned off but this requires that it be turned on.

Re:Good they've merged. Why XML ?

[geoffspear]

So crackers are going to somehow use port 53 of my machine to connect to a nonexistant server and root my box?

Open ports are not inherently insecure.

Re:Good they've merged. Why XML ?

[Allen Zadr]

Actually - nevermind. I found a reason why I can't impliment this technology, ever.

Use of this technology requires submitting to a Microsoft license. This license allows distribution (but not re-distribution), and is not compatible with the GPL. That is to say, no GPL mail server will ever be able to directly impliment checks for this.

From the license (forgive typos, I typed this from the PDF):

2.2. Source Code Distribution You also have a nontransferable, non-sublicenseable, personal, license to distribute or otherwise disclose source code copies of such Licensed Implementation licensed in Section 2.1 only if You (i) prominently display the following notice in all copies of such source code, and (ii) distribute or disclose the source code only under a license agreement that includes the following notice as a term of such license agreement and does not include any other terms that are inconsistent with, or would prohibit, the following notice:

"This source code may incorporate intellectual property owned by Microsoft Corporation. Our porvision of this source code does not include any licenses or any other rights to you under any Microsoft intellectual property. If you would like a license from Microsoft (e.h. rebrand, redistribute), you need to contact Microsoft directly."

That, my friend, is embrace, extend and assimilate. Nothing under strict GPL can impliment this natively. IIRC, SPF (Sender Permitted From) did not have source restrictive terms.

Re:Good they've merged. Why XML ?

[Russ Nelson]

Dude, there's a gazillion formats which can be extensible and self-documenting. XML is a religion, and you obviously drank the kool-aid.
-russ

Re:Good they've merged. Why XML ?

[Allen Zadr]

Wow, a gazillion? Cool!

Do you have a better alternative? Preferably something that is standardized, or at least based on something that most developers would already have some familiarity with.

Troll me if you like, but please, contribute an alternative idea along with your troll.

Re:Good they've merged. Why XML ?

[Malc]

In theory you should be allowing TCP 53 anyway as it is part of the DNS spec. If you don't, you're misconfigured even though it will work most of the time.

Anyway, it's a matter of only allowing incoming packets for established connections, which can be further confined to just your outside name servers.

Re:Good they've merged. Why XML ?

[ceswiedler]

Doesn't this apply only if you include Microsoft's implementation in your GPL application? A license like this can't prevent you from implementing it from scratch. Patents can (and I wouldn't be surprised if there were a few dozen of those) but I don't think the license is an issue.

Re:Good they've merged. Why XML ?

[Electrum]

In theory you should be allowing TCP 53 anyway as it is part of the DNS spec. If you don't, you're misconfigured even though it will work most of the time.

That's not correct. See RFC 1035 section 4.2.

Re:Good they've merged. Why XML ?

[Allen Zadr]

The problem here, is that the license is for use of the XML Schema definition for the purpose of filtering Email.

Schemas are licensable on their own.

Historically, this is because of ERP (Enterprise Resource Planning) database products like SAP, Daly.Commerce, Oracle Financials, etc. In the case of an ERP, if a schema can be used and replicated without license, then the data benefits of the product could be culled across and enterprise with only 1 user license (or enough licenses for the uses that enter the data through the product).

In the case of Microsoft's license, the use of such a schema by entering it into DNS is (technically) non-portable. Thus does not need to be protected by distribution licenses. My ability to give you my Email-Caller-ID object is useless to you, because it will have my data in it, not yours.

However, the ability to use this XML schema to filter out Emails (the filter implimentation) is subject to, and likely to be, distributed and re-distributed. MS doesn't want this to occur without your 'checking in' directly with Microsoft. In this way, if they choose to stop distribution of new implimenations, they have the ability to do so. That, makes a strong statement about the openness of this license.

Re:Good they've merged. Why XML ?

[Feyr]

imagine the pain for all those losers using tinydns in a default setup (eg, no tcp listening deamon by default)

Re:Good they've merged. Why XML ?

[pjrc]

The plan appears to be to support both XML and v=spf1 formats (according to the summary of the meeting between the SPF and Microsoft folkw). The expectation is that the vast majority of sites will publish the simple and compact v=spf1 format for the forseeable future. XML allows extensibility for the unforseen future that follows afterwards.

Re:Good they've merged. Why XML ?

[7x7]

Because Microsoft has a patent on "Embedding XML in an XML File." That means, if you put and XML file inside another XML file, you've violated the patent. With mail systems gaining complexity and the flexibility of XML, you have a situation ripe for extending a basic standard by embeding further information in the XML. Microsoft's ideas so far have all incorporated some sort of XML implementation, and none of those ideas were rolled out until just after their patent was granted. I personally think they want to corner the e-mail market.

Re:Good they've merged. Why XML ?

[DonGar]

Um... this license should only apply to their implementation, so we could create a new version from scratch that is licensed any way we want.

That is, unless there are there patents or other issues that get in the way. If my hopes/dreams/vague memories are correct, then the w3c has a policy to refuse to accept standards that are patent encumbered.

However, I haven't RTFA, so maybe I'm missing something here.

Re:Good they've merged. Why XML ?

[Allen Zadr]

Look to the previous reply for the bulk of your question.

Second, two more points.

1. The W3C has revised this policy
2. While this technology uses W3Cs XML, the patent has to do with XML use within DNS combined with SMTP, and neither is a W3C controlled standard.

Re:Good they've merged. Why XML ?

[Anonymous Coward]

"Remember that XML is not a Microsoft technology"

Insightful...

Re:Good they've merged. Why XML ?

[Malc]

Errr, I don't see anything there that contradicts me or differs from what I remembered. All it says is that UDP is the recommended and preferred method for DNS answers. It is limited to 512 bytes after which TCP should be used. Furthermore, only simple DNS queries are even attempted over UDP - zone transfers and the like are always TCP.

Re:Good they've merged. Why XML ?

[bozojoe]

>>>Next on the agenda - get everyone to use digitally-signed certificates :-)

If only I could get my grandmother to get a cert (let alone get her to use her fax machine)

Re:Good they've merged. Why XML ?

[John Starks]

Yeah. We'd have to... run axfrdns-conf. And then add one line to /etc/axfrdns/tcp. Then run make in /etc/axfrdns. And then do a ln -s /etc/axfrdns /service. And that's it. Wow. What a pain. I should switch back to BIND.

Re:Good they've merged. Why XML ?

[thogard]

Some of us don't use port 53 at all for zone transfers so thats not an issue and no sane request to my DNS server will ever produce a packet bigger than 512 bytes therefore its only misconfigured for people playing with my DNS server which the firewall blocks.

Re:Good they've merged. Why XML ?

[rew]

It's worse.

You're allowed to download the MS-Caller ID spec (free-of-charge) to develop software. Then your users require an MS-licence to use the software, wether or not you want your code to be GPL or not.

At least that was the state when I last looked.

we have this...

[Mz6]

.. it's called the mail.

In the year 2000....

[ArmenTanzarian]

You will be truly disturbed to find out that your Grandmother is apparently the one who's been pushing for you to use Cialis.

SPF 30

[FireBug]

Why on earth are they integrating SPF into technology? I mean, it's not like Slashdotians ever go out into the sun or anything...

Laugh! It was a joke!

Re:SPF 30

[Pxtl]

Cute. Still, I do wonder why they even advertise this as "caller ID" - after all, everyone knows caller id can be spoofed. Really, the existing e-mail header is the same as caller ID. In theory it says who's calling, but anyone dedicated can get around it.

Why don't they just call it like it is? A secure substitute for the "source" field of the e-mail header.

Re:SPF 30

[Short Circuit]

Laugh! It was a joke!

A joke?!

The last time I got a sunburn, I moved my computer away from the window!

Re:SPF 30

[_w00d_]

It's probably worded the way so PHBs can understand it.

Re:SPF 30

[curtoid]

I thought SPF meant Single Point Failure at first glance!
You know, like putting all your eggs in one basket so when you trip, you're back to Zero.

Re:SPF 30

[pjrc]

Really, the existing e-mail header is the same as caller ID.

Really? How, exactly, are existing email headers anything like a DNS server that answers a query with a list of IP numbers that are the only ones authorized to transmit email for that domain?

In theory it says who's calling, but anyone dedicated can get around it.

Microsoft's Caller ID (and SPF) are about providing a DNS-based query that can be used to check if the IP number of the system sending the message is among the servers that are supposed to be sending that domain's email. Well, that's a bit over-simplified, but still the basic idea.

Caller ID and SPF aren't about adding an easily faked extra bit of info to the email headers. The IP number of the sending MTA is not easily faked, because SMTP uses TCP/IP which requires round-trip packet exchange, and the receiving MTA adds the "Received" line to the header with the verified IP number of the other MTA that sent the message.

To "get around it", a spammer who's forging someone else's domain name would have to be able to intercept or interfere with the receiver's DNS query. Very unlikely... especially when using open proxies, hacked home-user machines, free trial accounts, and anything else that's in no position to interfere with the receipient's DNS.

Of course, a spammer can also "get around it" by using a domain name that is under their control, and simply return a Caller ID or SPF result that says the IP number they are using is authorized (or that all IP numbers are authorized). This is what spammers will do (and reportedly some are already doing). But forcing spammers to use their own domain names is the goal of domain-name authentication.

Why don't they just call it like it is?

"Caller ID" in voice phone systems refers to the transmission of caller identify between the first and second ring. It seems you have assumed Microsoft's email Caller ID works roughly the same way, but it does not. The sender's IP number is always known (and virtually impossible to forge) because of the mechanics of TCP/IP, and Microsoft's Caller ID proposal is a query to the claimed source for a confirmation that the IP number is among the ones that are authorized to be sending.

A secure substitute for the "source" field of the e-mail header.

You completely misunderstood how it works. Hopefully you now understand.

Re:SPF 30

[Pxtl]

No, you completely misunderstood my message. The point was that caller-ID is a bad analogy for their new system. True "caller ID" behaves like the old e-mail header. The new system works, as you said, completely differently.

IP addresses?

[Anonymous Coward]

Can't the TCP/IP packets be formed to include these originator server IP's?

Re:IP addresses?

[Allen Zadr]

No that won't work.

Because the SMTP protocol requires two-way communications, the packet has to have a valid IP address, so that the TCP/IP traffic can go between the two mail servers (sender, reciever). Because of this, you are guaranteed that the IP address is correct to within a given sub-net. Within that sub-net, yes, spoofing is possible (convince the router that you are the real 131.107.3.124). This is definately "close enough" to usually be accurate.

Why not XML?

[Mz6]

Why build a new format when there is one already available that would suit their needs?

Re:Why not XML?

[lpontiac]

XML is not a format. It's a metaformat.

Re:Why not XML?

[DrPizza]

Because, since XML is not a format (but rather a standardized way of creating one's own formats) the issue of "creating a format" is not solved by the decision to use XML.

What XML "wins" is off-the-shelf parsers; one still needs to write some amount of code to convert dumb XML (elements and attributes and all that crud) into something with semantic meaning to your application.

For a simple application like this it's not clear that the overheads of XML (both in terms of size, computational complexity, and programmer overhead to make the aforementioned conversion) are at all worthwhile.

Re:Why not XML?

[silas_moeckel]

How about because XML is pretty bloated and an embrase and extend sort of language thats designed to let people insert new things without breaking old things. Meaning MS or anytbody else can introduce new tags that arent compatable with somebody elses tags, they shouldent collide but the bloat gets pretty bad when you have to support 2 3 4 20 differnet version of the same tags to be compabible with everybody. XML is good for programs that should be embraced and extended. Internet related thigns should come from RFC and be fixed one set of rules, at least I think so. One set of rules is what has gotten us email that works around the world in less than 20 years.

Re:Why not XML?

[BlackHawk-666]

Plain old text files might be better I reckon. e.g.

MX:MyFatServer:212.169.24.12,212.169.24.13
MX:MyOtherServer:212.169.24.16

Not taxing to parse, simple, grepable, fast, lightweight, human readable. Even if you work for MS you should be able to write the code to parse this ;->

Re:Why not XML?

[taybin]

But SMTP already allows anyone and their grandmother to add new header fields. Keeping that feature is good for change and innovation.

Re:Why not XML?

[Russ Nelson]

Uhhhhhhhh, because a DNS packet is limited to 512 octets??
-russ

Re:Why not XML?

[silas_moeckel]

Yes they allow new headers to the mail thats all fine and dandy (I would hate for spam assisin not to work) it's about allowing people to change things in DNS most importantly insert big ugly XML that needs to be retreived in a single packet or your overheard goes through the roof setting up a tcp session etc etc etc. Basicaly adding an extensible standard to something of a fixed size does not make sence. The packet simply can only be so big so you cant let anybody do whatever they want in it.

Re:Why not XML?

[amRadioHed]

Nice try dude. Metaformat means a format for creating formats. In other words, a metaformat is a format. Therefore you're a karma whore. QED.

Re:Why not XML?

[Anonymous Coward]

I'm waiting til they make a Mega Format before I adapt!

Re:Why not XML?

[GregChant]

Oooh, so close. What you're describing would be more analogous to "Metaformat qua format." A metaformat does not need to be a format; it can be a specification, an idea, or a shoe as long as it deals with those fundamentals required for making formats.

Mmm... metaformat qua shoe...

Re:Why not XML?

[psoriac]

Someone should tell Yahoo that. Theirs is 520 bytes last time I checked (about 5 months ago).

LOL, this is soooo easy to bypass!

[numbski]

*67 && mail -s "enlarge your elbows" mpost4@mikeoconnor.net cat enlarge.txt ;)

Re:LOL, this is soooo easy to bypass!

[mpost4]

that mail would never get to me, since all email going to mikeoconnor.net is not accepted there are no address tehre, so spam mpost4@mikeoconnor.net all you want.

Sounds like a truly awful idea

[isdnip]

Now it sounds like a bad idea for both semantic (what it does) and syntactic (how it is coded) reasons!

The syntactic bit is easy -- XML is hardly appropriate for a DNS function. Mickeysoft is running around patenting XML schemas, and it adds a new layer of complexity to DNS. But then bad syntax is usually dealt with by code.

The semantic bit is worse -- SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted. But wait -- SPF doesn't block spam! It just blocks spam where the From: is not right. Spammers can still create new domains on a hit-and-run basis, and they'll pass SPF. So it's another blast-proof vault door stuck onto a grass hut, a silly waste of time. The only potential real benefit, I suspect, would be to make phishing harder. The address will have to be slightly different from the spoofed domain. But that leaves plenty of opportunity to create deceptively-close hit-and-run domains (like, say, pay-pa1-approva1.com).

Worse, of course, is the collateral damage. How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server? My "from" address is an alias, not a real sender, and I use it to send via more than one ISP, depending on where I am. SPF seems to make this a lot harder, thereby forcing more people to put their ISPs' name in the From: field, rather than their own. Since email is not portable, a user's address is lost when they change ISPs, or when their ISP changes names (mediaone->attbi->comcast). Personal domains (forwarded via a service like mydomain) solve this. Will SPF kill mydomain?

I repeat what I've said before. The only way to kill spam is to stop having all email be totally, absolutely, "free" of charge in any quantity. This is not the topic to discuss solutions, but they are certainly possible, and they aren't SPF.

Re:Sounds like a truly awful idea

[lukewarmfusion]

As for email not being portable, I'd expect more personal domains (or permanent email addresses) as a result. Google's GMail (and other long-term, high-storage services) would let you keep your email address even if you switched ISPs. I bought a domain just so I would have an email address that I wouldn't have to worry about losing.

Re:Sounds like a truly awful idea

[jolajolajola]

0.02-per-email (or US$0.05) is probably better. Like SMS messages, and they're inclusive, so if you send 1000 a month, then you're fine. If you send any more than 1000 a month (i.e. 1000 separate recipients) then you're probably doing some marketing (legitimate or otherwise), in which case you shouldn't be using a consumer mailserver, and they'll bill you bigtime or ask you to move over to another "tarriff".

There's more to the idea, but you get what I mean.

Re:Sounds like a truly awful idea

[andycal]

It's not a great solution, but you can list your ISP (s) in the SPF records. This way you can still send mail "from" your domain name, and have it delivered by your ISP.

The real problem is compliance, until 99% of mail servers provide this data, I can't reject mail from non SPF listed domains.

Re:Sounds like a truly awful idea

[arr28]

Spammers can still create new domains on a hit-and-run basis

Fortunately the whois data contains that date and time that the domain was created. Therefore, I can choose to refuse mail from recently created domains. Alternatively, this could be a springboard to domain-name reputation systems.

At the moment the domain name can't be trusted so I can't use these otherwise handy techniques.

Re:Sounds like a truly awful idea

[jafomatic]

It just blocks spam where the From: is not right.

Correct, but what this means is that there should now be some level of accountability to the originator. One of the biggest complaints I would have, looking into email-spam as a problem, would be that there's no way to hold a sender accountable when the true origination of the message is unknown. If I understand the proposal correctly, that accountability will at least be marginally present.

The ability to spoof this system is another issue entirely. The system's intention would appear to add value.

How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

A valid concern; as I read it, this wouldn't be possible. Either that or we're overlooking exactly what makes up a valid original sender.

Re:Sounds like a truly awful idea

[Anonymous Coward]

But wait -- SPF doesn't block spam! It just blocks spam where the From: is not right.

...thus protecting domain owners from getting millions of bounces every time a spammer decides to spoof their domain.

Worse, of course, is the collateral damage. How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

Your business wants you to send email but doesn't provide a mail server?

Re:Sounds like a truly awful idea

[Albanach]

This is not the topic to discuss solutions, but they are certainly possible, and they aren't SPF.

If spammers have to buy new domains for every couple of thousand spams they face a big problem.

• Firstly it all adds to the cost - with tiny response rates you'd have to imagine the margins are tight.
• Secondly if they have to buy domains they need to pay for them - that leaves a physical paper trail to spammers, now legislation can help.
• Thirdly we have plenty of existing technology such as black hole lists that will be a lot more effective if lots of spam comes from one newly registered domain.
• Fourthly we don't need the entire web to be using SPF for it to become effective. If you receive spam from an AOL account it's now possible to easily check if it in fact came from an AOL mailserver. That other people haven't yet implimented SPF is irrelevant - we can use the technology to our advantage today. Once it's widely implimented we can even start to apply a small spamassassin score to not SPF confirmed mail. As adoption grows we can increase that score still further.

Re:Sounds like a truly awful idea

[thogard]

I get spam from people with valid SPF records. SPF has nothing to do with anti-spam.

As far as its protecting aginst bounces, thats only going to happen if a few million systems start using SPF which is not going to happen for at least a decade. SPF has too many problems to use in the real world.

Re:Sounds like a truly awful idea

[AlienFactor]

As for email not being portable, I'd expect more personal domains (or permanent email addresses) as a result.

<Conspiracy>Funny, that's exactly the business that SPF's author (pobox.com) is in.</Consipiracy>

Re:Sounds like a truly awful idea

[Chang]

> SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted

This is false. There is no requirement for every domain on the internet to adopt SPF before it becomes useful.

Instead each domain owner decides when to flip the switch on for SPF enforcement for their individual domains. Since 14,000 domain already have valid SPF records and many of them have enabled enforcement, SPF is useful for not accepting worthless spoofed emails TODAY. Not in some far off future.

Re:Sounds like a truly awful idea

[Short Circuit]

One issue I'm concerned with, that I haven't seen answered yet, is mailing lists. I usually see something like this:

From: Joe User &lt;joe.user@grnet.com&gt;
To: T Ward &lt;t.ward@scinet.edu&gt;
Reply-To: bbs-devel@lists.sourceforge.net

The message is being sent by SourceForge, but has a "From" address as at grnet.com.

Re:Sounds like a truly awful idea

[djmurdoch]

The real problem is compliance, until 99% of mail servers provide this data, I can't reject mail from non SPF listed domains.

There aren't many tests which are perfect spam identifiers with no false positives. You should use the SPF compliance as part of a scoring scheme. Messages that fail SPF are more likely to be spam than messages that pass, so they get a higher spam score. If the score exceeds a threshold, mark it as possible spam. If it exceeds a higher one, delete it unread.

This is the strategy Spamassassin allows, and it works really well, especially if you let the scoring be adaptive (i.e. use "Bayesian tests").

Re:Sounds like a truly awful idea

[thogard]

who gets the money? Billy Gates?

Re:Sounds like a truly awful idea

[FattMattP]

But wait -- SPF doesn't block spam!

Correct. It's not meant to. SPF's goal is to prevent domain name forgery. Blocking spam, if it does that, is a side effect. Authenticating the sender is the primary goal.

It just blocks spam where the From: is not right.

No, that's what MS "Caller-ID" does. SPF checks the MAIL FROM in the SMTP transaction. Think of it this way, SPF does its checks on the envelope and caller-id does its checks on the header.

The only potential real benefit, I suspect, would be to make phishing harder.

That's the point.

Re:Sounds like a truly awful idea

[leviramsey]

An SPF record is a record stating what hosts are registered senders of mail from a given domain. There's nothing stopping you from adding, say, smtp.verizon.net or whatnot to the record.

Of course, there's also SMTP AUTH...

Re:Sounds like a truly awful idea

[Nugget]

SPF enforces the envelope sender (from the smtp "mail from:" comman", not the header "From:" line. I believe if you look, you'll see that your mailing list mails have the envelope sender set to the mailing list's sender address.

It works just fine.

Re:Sounds like a truly awful idea

[leviramsey]

Actually that would cause a problem, for precisely the reason you outlined.

• mail.sourceforge.net sends mail with an envelope sender of joeblow@comcast.net.
• receiving server looks up comcast.net's SPF record.
• mail.sourceforge.net is not approved (assuming that comcast has an SPF record).
• mail fails SPF test.

The solution, as I understand it, is an SMTP extension, RFROM, though I've forgotten whether it's the relaying sender that goes in RFROM or the original sender that goes in RFROM.

Re:Sounds like a truly awful idea

[Smallpond]

You don't need to have 99%. You can just delete mail from spoofed SPF-compliant domains, from non-resolving domains, and from domains in blocklists. This will force the spammers to send "from" real, unblocked, non-SPF domains. These will quickly get lots of bounce messages from spam they didn't send and get added to block lists. This will encourage the holdout domain owners to use SPF. Critical mass ensues. All domains will end up using SPF if they want to send mail. If not, they end up in block lists.

Re:Sounds like a truly awful idea

[Thagg]

isdnip says, among other things:

Worse, of course, is the collateral damage. How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server? My "from" address is an alias, not a real sender, and I use it to send via more than one ISP, depending on where I am. SPF seems to make this a lot harder, thereby forcing more people to put their ISPs' name in the From: field, rather than their own.

I would think that it would not be hard, at all, to invent a system that would actually send mail from your business's domain. You could compose the mail at home, or on the road, and then forward it through your business's mail server. Isn't that what you really want to do, in any case? Don't you want your business clients to have confidence that the mail actually originated from the business?

It seems to me that your apparent 'flaw' would quickly become an actual benefit.

Thad Beier

Re:Sounds like a truly awful idea

[Bob Ince]

> XML is hardly appropriate for a DNS function.

I have to agree that *full* XML is inappropriate for DNS. Requiring an XML parser for all mail servers is undesirable (unless you're MS and have MSXML available as part of the platform).

On the other hand, some kind of structured format is needed for the more complicated cases, which are currently badly served by SPF's somewhat clunky macro language.

As someone who's written a complete XML parser I wouldn't wish such a thing on MTA authors. A reduced profile of XML with no declaration, DOCTYPE, entity references (argh - imagine external parameter entity references in a DNS record!) or CDATA sections would seem preferable. Even comments and character references are pointless for DNS.

> SPF doesn't block spam

SPF doesn't block spam, but it makes blocking spam feasible. Currently thanks to spoofing, zombies etc. it's not.

> Spammers can still create new domains on a hit-and-run basis

Of course. But we can now block the new domains, so it'll cost them $10 to send a message to servers using 'the New Email' or whatever Caller-ID+SPF will be called. This will drastically alter the economics of spam.

> It just blocks spam where the From: is not right.

Nope, it goes on the envelope sender (SMTP MAIL FROM), the machine sending the mail.

This will also help in cutting down the insane number of misdirected bounces.

> How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

Same as you do now. You can still use any From address you like; the bit that will be checked is that the ISP's server really does represent the ISP.

Re:Sounds like a truly awful idea

[CustomDesigned]

Worse, of course, is the collateral damage. How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server? My "from" address is an alias, not a real sender, and I use it to send via more than one ISP, depending on where I am. SPF seems to make this a lot harder, thereby forcing more people to put their ISPs' name in the From: field, rather than their own. Since email is not portable, a user's address is lost when they change ISPs, or when their ISP changes names (mediaone->attbi->comcast). Personal domains (forwarded via a service like mydomain) solve this. Will SPF kill mydomain?

Your objection mentions a multitude of configuration scenarios. I will address a few:

• A home user with a personal domain simply lists their ISP as a valid sender in SPF.
• A travelling salesman with a laptop should use a VPN or SMTP AUTH or webmail.
• A travelling salesman using anonymous PCs wherever he finds them is crazy. But if he insists, he should use webmail. That is probably less of a risk than entering an SMTP AUTH user and password.
• SPF does not prevent you from putting whatever you want into the From: header. (Yahoo domain keys addresses that.) SPF only authenticates the envelope sender. The envelope sender is used for bounces, return receipts, and the like.
• If you can't authenticate from the field in any way, you could always publish SPF that lists your home system and leaves everything else neutral. Since anyone, including me or a spammer can currently send email claiming to be from you, that accurately reflects the situation.

Re:Sounds like a truly awful idea

[jaseuk]

The envelope sender would be mailinglist-owner@mail.sourceforge.net (or similar).

Re:Sounds like a truly awful idea

[isdnip]

> forward it through your business's mail server.

You're not reading the OP correctly. My business, like many small businesses, doesn't own a mail server. It owns a domain name, which works via a mail-forwarding service on mydomain. Outgoing mail goes via whatever ISP I happen to be on at the time; i.e., the one at home when I'm home, the one at the office when I'm at the office, or the one at the hotel when I'm at a hotel.

My clients don't care where my mail originated. They know it's from me. I'm not Paypal or BankOne.

If I had a mail server of my own, then another popular (but dumb) antispam measure, port 25 blocking, would cause it to be inaccessible from at least some locations.

Re:Sounds like a truly awful idea

[not-folly]

Yeah, but he'll send it back to you in a check marked "Paid In Full"

Re:Sounds like a truly awful idea

[mr_mischief]

So the people who break into Windows boxes with weak security and send through the proper server for that box but advertise biggerpenis.com in the email still get their mail delivered, and have it "verifiably" traced to their victim.

This is worse, not better.

Re:Sounds like a truly awful idea

[Just Some Guy]

True, but you can immediately start rejecting email with invalid SPF results.

One of my domains gets joe-jobbed all the time. I've started publishing SPF records for all of my domains, and from the moment I started doing so you can verify whether a particular email did or did not originate from my system. You win, in that you get to reject a bunch of email with zero false positives. I win, in that you know that the email didn't come from me, so you won't be bouncing the messages back to my domain or calling me to complain about "my" spam. The spammer loses, in that it cost resources to attempt to deliver those dead-on-arrival messages (not much, granted, but multiplied by the 14,000+ domains currently serving SPF records...).

SPF may stop spammers, but it sure goes a long way toward protecting the reputations of some of their victims.

Re:Sounds like a truly awful idea

[Anonymous Coward]

Can someone post an example of an SPF record that would loop and/or take a long time to parse, please?

Thanks...

Re:Sounds like a truly awful idea

[WoodstockJeff]

How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

Under SPF, that's simple - you list all servers that can handle your domain's mail in the SPF record. Take a look at the SPF home page for more details, including a link to a wizard to "calculate" the proper SPF DNS entry for situations such as yours. You should be using some form of certification of who you are when sending mail via any server, so that you can tell people "that wasn't me!" when someone forges a message from you. SPF is simply one method to aid in that.

That said, I don't like the MS scheme - it's yet another demand on bandwidth that doesn't provide the benefits claimed in any way better than less intensive methods (like SPF).

Re:Sounds like a truly awful idea

[myov]

If spammers have to buy new domains for every couple of thousand spams they face a big problem.


I should show you my inbox. All the spam on one of my accounts comes from the same source, and with a different .biz every day.

A domain is, what, $8 US at GoDaddy? Less in bulk?

Re:Sounds like a truly awful idea

[randomencounter]

SPF has much to do with anti-spam. It isn't a magic bullet, but there are no magic bullets.
TXT records are no different to your firewall than any other DNS request.
Normal SPF records fit in UDP DNS requests.
Looping could be a problem, but it is a resolvable one.
sendmail.cf is amazingly flexible, I'm rather certain that it could handle at least one level of SPF parsing.

Re:Sounds like a truly awful idea

[mla_anderson]

The mobile user is one I had concerns about as I work both in the office and from home.

The article covers that with a "Sender" header as well as a "From" header.

e.g.
From: mla_anderson@work.com
Sender: mla_anderson@home.net

This means a rewrite of many email clients, but I suspect it's easy enough to do, and open source ones would be able to do it sooner than MS ones.

Re:Sounds like a truly awful idea

[miquels]

So you connect to port 587 (submission protocol - it's just authenticated SMTP) of the mailserver of your mail-forwarding service, and send your mail.

Re:Sounds like a truly awful idea

[Eraser_]

There is new technology available today which allows you to connect to a corporate (or home, etc) network in a secure way which gives you a presence on that network, it's called VPN, or SSH, or pcAnywhere, or Timbuktu, or SMTP-AUTH, or oh never mind.

Some of those options require another computer to be there answering to your beck and call, others just a computer with powerful multi-user virtualization (FreeBSD? Linux? OS X?). VPN gives you local presence on the network, allowing you to do all sorts of cool things. SMTP-AUTH would let you send email from anywhere into your corporate network, where it would then validate SPF checks.

Your IT department should easily be able to accomodate this on a laptop for a roaming user. I work for the public school system and we have both VPN, SSH, and dialup access to anyone who cares to ask for it (VPN is restricted to administrators, though, sorry students).

These security precautions should have been in place before you left the office, otherwise I could just send email as your company anyways.

Re:Sounds like a truly awful idea

[randomencounter]

I run a company mailserver with remote clients.
We use SMTP AUTH and they send through our work servers wherever they might be, so we are able to use a strict SPF record.

Re:Sounds like a truly awful idea

[williamhooper]

And then there are E-mail providers that don't provide SMTP. Am I supposed to talk to Freeshell and get them to add every ISP I send from?

Re:Sounds like a truly awful idea

[pjrc]

This statement is simply wrong:

SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted.

I'll type out an example, to show you exactly how it is already working and rejecting forgeries TODAY, without being mandatory, and long before widespread implementation.

Suppose your MTA receives a message from a spammer who is impersonating me. The message claims to be from "paul@pjrc.com", but it isn't. Your SPF-aware MTA (or spam filter) does a TXT query to my domain, and because my domain is one of the 14500-some returning SPF, you'll get this:

paul@preston ~ > host -t txt pjrc.com pjrc.com
Using domain server:
Name: pjrc.com
Address: 168.143.173.65#53
Aliases:

pjrc.com text "v=spf1 a:outgoing.pjrc.com mx -all"

That means there's only two sources that are authorized to send messages for pjrc.com. So your software will first do a lookup on "outgoing.pjrc.com", sorta like this:

paul@preston ~ > host outgoing.pjrc.com pjrc.com
Using domain server:
Name: pjrc.com
Address: 168.143.173.65#53
Aliases:

outgoing.pjrc.com has address 168.143.160.91

If the IP number of the MTA sending the message is 168.143.160.91, then you know the message is coming from my server. If not, then the next authorized source is checked. Your software will do a MX lookup, like this:

paul@preston ~ > host -t mx pjrc.com pjrc.com
Using domain server:
Name: pjrc.com
Address: 168.143.173.65#53
Aliases:

pjrc.com mail is handled by 10 mail.pjrc.com.

Then, the name "main.pjrc.com" is looked up, such as:

paul@preston ~ > host mail.pjrc.com pjrc.com
Using domain server:
Name: pjrc.com
Address: 168.143.173.65#53
Aliases:

mail.pjrc.com has address 168.143.173.65

If the MTA sending the message has the IP number 168.143.173.65, then you also know the message came from an authorized server for my domain, and thus the message is not a forgery by a spammer.

However, the last part of the SPF record was "-all", which means NO OTHER sources are authorized to send pjrc.com's email. So if the MTA sending the message is any IP number other than 168.143.160.91 or 168.143.173.65, then you know it's being sent from an authorized source, and thus it's a forgery (or I've messed up my SPF record).

Please take notice that for this to work, only you and I need to have SPF implemented. All the communication was between your receiving MTA (or SPF-aware spam filter), and my the DNS server for my domain. No other servers or other hosts on the internet needed to implement SPF in order for you to discover that a forged message isn't really from me.

Of course, only about 14500-some domains are currently publishing SPF records... so you can only detect forgeries if the spammer is foolish enough to forge one of our domain names. However, "aol.com", a very commonly forged name, is on the list. Likewise, "hotmal.com" is publishing caller ID (which works more or less the same as SPF, but with different syntax and with the minor distinction of sender vs from).

The bottom line is that your statement is incorrect. SPF does not have to be mandatory to function. It already works for those cases where the receiver bothers to check and the (claimed) sender publishes a SPF record. You can already reject all forgeries from 14500-some domains, including little ones like mine and big ones like AOL and Hotmail... and this is fully functional TODAY, long before 100% compliance is reached.

Re:Sounds like a truly awful idea

[ChaosDiscord]

But wait -- SPF doesn't block spam! It just blocks spam where the From: is not right.

SPF can help in three areas, even without 100% compliance. First, it can protect you from joe jobs where a spammer uses your domain as the return address, flooding you with bounce messages and angry complaints. As more and more people restrict mail based on SPF the damage from a joe jobs decreases. Second, with similar gradual pay-off, it can seriously limit attempted identity theft spams that fraudulently appear to come from eBay, BankAmerica, or whoever the target it. Third, again with increasing benefit as more people respect SPF, it will limit the amount of damage done by email worms that spoof the sender based on random addresses found on the infected source machine.

No solution is ever perfect, fortunately no solution ever needs to be.

Re:Sounds like a truly awful idea

[Oloryn]

How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

The current implementation of SPF handles this. It's possible to say 'the valid mail servers for <domain> are also valid mail senders for me'. As long as your ISP has SPF entries made, you can thus incorporate them in your own SPF entry.

Re:Sounds like a truly awful idea

[FattMattP]

How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

It's your domain. You control the SPF record. You can add a record to allow anything if you wish. You can also add include:myisp.com to include your ISPs SPF record if they have one.

Re:Sounds like a truly awful idea

[phayes]

You need merely add the domains of the mail servers you use at work & home to the TXT record for yourdomain.com. This is how I setup SPF for my private domain.

Re:Sounds like a truly awful idea

[Anonymous Coward]

SPF checks the MAIL FROM in the SMTP transaction. Think of it this way, SPF does its checks on the envelope

So not only does this not stop spam -- it doesn't stop joe-jobs either. Who cares what MAIL FROM says? It's what in the From:-line that the clients use and the users see.

Some people think they will be able to blacklist on the envelope adress, which is inferior to blacklisting on IP. This has been done for some 8 years at least and is one of the two methods that has been proven effective against spam (the other being statistical filtering) and we should not replace it.

Re:Sounds like a truly awful idea

[JuggleGeek]

SPF doesn't block spam unless the mail system makes it mandatory, after all, so until 100% compliance is reached, non-SPF mail will still have to be accepted.

However, SPF compliant mail can still be scored as much more likely to be legitimate.

Spammers can still create new domains on a hit-and-run basis, and they'll pass SPF.

Spammers using SPF comliant mail are going to quickly be blacklisted, at which point mail from those domains can be auto-dumped. Spammers who continue to forge domains are going to find their mail auto-dumped when they forge a domain that uses SPF authentication. Domains which don't support SPF can still be forged - but that will simply increase pressure on those domains to get their shit together.

Spammers can register tons of new domains - which greatly increases their costs. And each new domain will help them for only a short period of time before the blacklists pick up on them.

So it's another blast-proof vault door stuck onto a grass hut, a silly waste of time.

Your theory seems to be that since this will effect only 90-95% of spam, and that spammers can get around it if they spend enough money, we shouldn't bother. Your theory aids the spammers, not the rest of us. I'll pass.

How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

You haven't even read how SPF works, have you? All you have to do is register your ISP's mail server as a valid sender for your domain. This is going to be quite common. Most people do not run their own mail servers. Many who have their own domains, like myself, do not run our own mail servers. But I know which mail servers have legitimate reason to send mail using my domain. The others that stick my domain in the "From" field are just forgeries. SPF will allow others to see the forgeries and dump the mail.

SPF seems to make this a lot harder, thereby forcing more people to put their ISPs' name in the From: field, rather than their own.

It's sad that you believe this, and worse that you got modded up for spouting completely false information. It's utter bullshit.

Will SPF kill mydomain?

Maybe - are you a spammer? If not, it's going to help protect your domain from being spoofed by a spammer. That happens to my domain daily.

I repeat what I've said before. The only way to kill spam is to stop having all email be totally, absolutely, "free" of charge in any quantity. This is not the topic to discuss solutions, but they are certainly possible, and they aren't SPF.

And I'll repeat what I've said before. Pay-to-Email isn't a good solution. It has a lot of bad side effects, and it simply isn't necessary. In order to charge per-email, the first step is to figure out who is sending the mail. You can't charge for email if you can't figure out who is sending it. And if you try to charge for email, you can damn well bet the spammers will try to figure out ways to send tons of email and make someone else pay for it. They haven't shown any interest in honesty in the past, and that isn't going to change.

You're arguing against a system that makes it easy to tell who actually sent the mail, yet arguing in favor of charging for email, despite the fact that you *can't* charge for email if you don't know who to charge. You seem to think that the spammers will stop forging other peoples domains, or that people will pay for spam sent that forged their domains. And it ain't gonna happen. The very idea is lunacy.

Once you can verify who is sending the email, then blacklists, whitelists, trustlists, all become much more useful (and less prone to abuse) than they are currently. And therefore pay-to-email is likely to be unnecessary.

Pay-to-email also has a lot of drawbacks. Slashdot sends me several emails a day. I have no idea how many they average a day, but it's bound to be a lot. Under pay-to-ema

Re:Sounds like a truly awful idea

[jolajolajola]

No, the ISPs charge you, on the basis that they know your MAC address, so they know who the IP belongs to, etc, etc.

Obviously there are security concerns regarding MAC spoofing, etc, but then you only let registered MAC addresses send outbound email.

Re:Sounds like a truly awful idea

[grahamm]

So not only does this not stop spam -- it doesn't stop joe-jobs either. Who cares what MAIL FROM says? It's what in the From:-line that the clients use and the users see.

But bounces are sent to the MAIL FROM line. From the mail administrator's point of view, one of the worst aspects of being Joe Jobbed is the receipt of bounces for mail which did not come from your site.

i still don't trust it...

[m2bord]

anything from ms...i don't trust. there has got to be something in it for them.
they are not going to spend their dollars and create something like this for the good of the net.
it's just not the ms way.

Re:i still don't trust it...

[WormholeFiend]

I think their main motivation is to stop the spread of virus attachments... anytime there's a MS-targetting worm going around, using similar distribution processes as spam, it creates an additional workload, not to mention that it tars Microsoft's image.

From my point of view, the spam cleanup would just be collateral.

Re:i still don't trust it...

[jellomizer]

Yes more people who will still use their email clients vs. switching to ones that block spam mail an other way. Plus they can say. We Stopped spam. Plus sience they have made the protocal they will be the first to implement it in their email servers and their clients giving a month lead over the rest, and obtaining market share and lockin.

Re:i still don't trust it...

[millahtime]

I guess the big question is then, would that really stop M$ viruses. Email isn't the only way to infect them. And, how many holes would there be in that scheme for virus and worm hackers to take advantage of.

ALso, what happens if a M$ Caller ID server gets a virus. Image the damage from that payload.

Re:i still don't trust it...

[kunudo]

I think their main motivation is to stop the spread of virus attachments

Maybe they should start distributing their security updates that way?

Stupid /., ruining my joke

[numbski]

[erwin: ~] root# *67 && mail -s "enlarge your elbows" mpost4@mikeoconnor.net << cat enlarge.txt ;)

joke. Helllo, McFly? :P j/k

[numbski]

You DO know what *67 does to caller id, right? :)

Lets hope they ditch the patent then.

[patthoyts]

having looked into doing an implementation of CallerID to add to an SPF parser -- I have come away with crossed eyes trying to work out if I'm allowed to release a BSD licensed implementation of this. I think maybe I can -- I'm pretty sure I can't release a GPL implementation though.

Damn, now where did I put that lawyer....

Re:Lets hope they ditch the patent then.

[Anonymous Coward]

If you can release a BSD version, you can release that version as GPL.

Re:Lets hope they ditch the patent then.

[JuggleGeek]

That's my biggest worry. Anything done under a "MS owns the patent, you can only use it with their permission" leaves everyone working under the good will of MS.

Software patents suck. Software patents on something as basic as "Email works like this" are even worse.

Money

[Frit Mock]

1. Create an virtual "Authority"
2. ???
3. Profit

OF course, it is an absolute must, that M$ is part of that game.

Blocking e-mail

[entropy123]

Currently I just block all the spamming domains and only allow e-mail from my friends and workplace into my inbox. At this point, a spammer needs to hijack @northwestern.edu to send me a penis ad. ent

Re:joke. Helllo, McFly? :P j/k

[mpost4]

yes I do, it displays your ID, but I was talking about the fact the email address you used does not exist.

You could've read all about it last week...

[jgardn]

...in a comment I made here.

Basically, this is a simply classic way to "embrace and extend" Microsoft's Caller ID. Before the flag day, SPF will work the way it is now. After the flag day, which will probably occur later rather than sooner, SPF will have all the functionality of Caller ID. The idea of allowing both XML and text descriptors is simply brilliant. Microsoft wanted to force everyone to use XML, but now you have a choice. I believe most (like 99.9%) will use the text descriptors, both because it is easier and because it is sufficient for 99.9% of the cases.

The net result is Microsoft can't claim ownership anymore. Caller ID will be a footnote in the history of email authentication.

Re:You could've read all about it last week...

[Russ Nelson]

Uhhhhhhhh, so now everybody has to have both an SPF and an XML parser in their MTA? And that's an improvement ... how?
-russ

Re:You could've read all about it last week...

[Anonymous Coward]

slower MTA = less spam over a given amount of time...maybe the goal is to slow the MTA so much, no mail gets through at all?

Re:You could've read all about it last week...

[pjrc]

The plan is to support both types of records as soon as possible. The "flag day" is not about...

Before the flag day, SPF will work the way it is now. After the flag day, which will probably occur later rather than sooner, SPF will have all the functionality of Caller ID. Here's a line to the post with the real info. Feel free to re-read it. Here's the sections about the "flag day". It's easy to see the "flag day" is about requiring forwarding MTAs to implement an extension to ESMTP, not the dual syntax of accepting either v=spf1 or xml (which will presumably occur very soon, since it doesn't impact anybody negatively, except people with a religious believe that one syntax ought to be universal.)

THE BIG NEW IDEA: let's define a new ESMTP field, "RFROM".

[snip]

If a MAIL RFROM is not present, checks run against the PRD from the headers ...

... Until the Flag Day, when you can go back to checking MAIL FROM if the RFROM is not present.

[snip]

THE NEED FOR A FLAG DAY

* The promise made to the SPF camp was: you will one day be able to reject forged MAIL FROM based on SPF lookups. I intend to keep that promise.

* Under the Old SPF, until the forwarders were all SRS compliant, we used a hacky workaround with trusted-forwarder.org.

* Under the New SPF, until the forwarders all use RFROM, we are in the same boat.

* Either way we need the industry to announce a flag day: we need to strike a balance between giving forwarders lots of notice, and being able to block stuff before DATA.

* Changing the burden to RFROM instead of SRS makes things a lot easier for forwarders. That reduces their adoption threshhold. Everybody wins.

* We can't decide today when the flag day will be, but one day soon we
probably will be able to.

* Before the Flag Day:

- Campaign for MTAs to start supporting RFROM.

- In the absence of RFROM, receivers SHOULD NOT reject spoofs until after seeing data, and SHOULD use responsible addr from 2822 header.

- (Receivers who know exactly who's forwarding to them may choose to reject anyway. Local policy, etc.)

- Fight Joe Jobs with SES.

* After the Flag Day:

- MAIL FROM RFROM now expected from forwarders.

- MAIL FROM spoofs without RFROM MAY be rejected.

- Flag day requires industry coordination to ensure MAIL FROM / RFROM support is out there.

- RFROM harder to spoof because only a PASS result is acceptable.

Re:You could've read all about it last week...

[Anonymous Coward]

Turn off the MTA. Same effect at lower cost.

Boycott of Microsoft's Caller ID for E-mail

[Anonymous Coward]

Make sure you let them know that patents on email technology are unacceptable. Merging is okay, let's just keep the SPF license, not the Microsoft one.

Re:Boycott of Microsoft's Caller ID for E-mail

[Rayban]

I heartily agree! It's good to see them cooperating, but I hope that the final license has a royalty-free patent grant with no attribution clauses.

If the two camps agree, this will speed up adoption of SPF records enormously.

Re:Boycott of Microsoft's Caller ID for E-mail

[Rayban]

Since the parent of my last comment was rated as a troll (I don't agree with that) here is the text:

---
Make sure you let them know that patents on email technology are unacceptable. Merging is okay, let's just keep the SPF license, not the Microsoft one.
---

It's just saying that patents on email tech are unreasonable. That's pretty reasonable to me.

Looks like there's some phun to be had!

[Xoder]

Does anyone know how to ANI fail with Outlook? Can you email the operator, claim you are blind and tell them "your" email?

SPF is harmful. Adopt it.

[Anonymous Coward]

Check this article for an interesting commentary on SPF.

Already got a mail client that handles spam well..

[millahtime]

I already have a mail client that handles spam well. It's Mail on my mac. It is amazingly accurate at filtering.

Now, if we could integrate that at the server level we would be all set. Maybe it's time for Apple to write a mail server with this technology.

it won't help there

[Anonymous Coward]

A worm can easyly check "who you are" and send as that entity to all your contacts of whom only one needs to be "doable" as you were and so on and so on ad nauseum... :/

Spam solution already exists

[Rashkae]

Spam would very quickly cease being a problem if mail clients were configured to start using PGP (GPG) keys and signatures by default. There is no need to re-invent or even change the e-mail RFC's.

Very simply, people can choose whether they want to receive unsigned e-mail, or accept sinatures from unkown keys. We'll eventually start building a web of turst (mistrust), such as, being able to automatically accept a key signed by some people or orgs, and similarly, blacklisting keys.

I could very easily, for example, instruct unknown senders (people who aren't in my contact list yet) to download my public key from a specified location to encryp a message that would bypass my filtes. Only a person who followed the instructions would be able to send me an unsolicited message.

Re:Spam solution already exists

[gclef]

You know, people have been saying that for almost a decade now. Face it: digitally signed email isn't working. Key management is a pain in the ass, the bootstrapping necessary to check user's keys is a mess, and it doesn't really gain you that much in the end. We've had 10 years to get signed email working, and it didn't happen. Time to find another way (whether it's this SPF or something else is a point for argument).

Re:Spam solution already exists

[_Sprocket_]

We've had 10 years to get signed email working, and it didn't happen.

Huh. How long did it take to get internet email accepted as a commonly used service? How long for the telephone?

Re:Spam solution already exists

[gclef]

How long did it take to get internet email accepted as a commonly used service?

Once people had access to it, signifigantly less than 10 years. Sure, email was around for a long time before it took off, but that's a function of access to the Internet, not a function of the usability or functionality of email.

Email signing has been around for just as long easy access to email, if not longer. Once there was easy access to email, it took off in popularity. Email signing, which has been freely available that whole time, did not. There's a reason for that.

Re:Spam solution already exists

[idesofmarch]

That works fine for your limited group of tech-savvy buddies, but the real world is different. I run my own business, and I am never going to ask a client to do this, and neither would anyone I know who runs a business. If a client cannot communicate with you - that could likely result in lost income. Downloading a key isburdonsome to do, and think about what message is sent to those clients who somehow get your email address (without key downloading instructions), try to send you an email, and are rejected. The message is "we do not trust you."

One aspect of a workable email system must not change: you should be able to give out your address and someone can use that single piece of information to contact you.

Re:Spam solution already exists

[Rashkae]

Besides, how can you say it's failed for 10 years, when it was never *even* tried. Thunderbird is the first mainstream e-mail client I've installed that supports Encryption out of the box. But calling it mainstream is a stretch, and it doens't really include key management. If MS is serious about fighting spam and improving security, I think they should start with *Implementing* technology and standards that have been around for 10 years.

Re:Spam solution already exists

[Rashkae]

I agree.. really, I do.

But then, I dont gripe about the 50 or so people a day who contact me.

Re:Spam solution already exists

[Lumpy]

it would stop even faster if all email servers simply had a rule that ignored any transfer requests from any machine that DID NOT have a Fully Qualified Domain entry and the top lever is not mail. or mail2. or mail3.

users sending email send to the email server from INSIDE the provider's network and if you are a roaming road warrior then you are forced to use webmail or VPN into the network.

this would solve a GOB of the problems and certianly stop all the emailing viruses.

But we cant get most email providers to force the use of encrypted authentication let alone set the damned servers up properly...

Re:Spam solution already exists

[Rashkae]

Well, I have to disagree with your proposal. I'm one of those stubborn bastards who sends mail through my own e-mail server.

1) I really hate webmail.

2) As the administrator for our small network, I prefer to have direct access to the e-mail server logs for security and verification purposes.

3)Your suggestion would still not solve the spam problem.

As annoying as the constant Viagra and porn ads are, not to mention offensive to some people, 3 years from now, those will be pleasant memories compared to what's going to happen once MS and friends convice the powers that be (FCC et al) that the real problem is porn and scams, and therefore the solution is to create a mechanism for legit businesses to send legit messages. I forget the statistics, but I remember a post somewhere about how many spams we'll be getting per day if every business in the US sent 1 unsolicted message per year.... it was frigthening.

((Besides, the Viagra ads are starting to get artful and entertaining.))

Re:Spam solution already exists

[Anonymous Coward]

I remember using digital signatures and encryption with IE 4 and Outlook Express long before I heard of Mozilla...

Re:Spam solution already exists

[_Sprocket_]

Once people had access to it, signifigantly less than 10 years. Sure, email was around for a long time before it took off, but that's a function of access to the Internet, not a function of the usability or functionality of email.

It's also an issue of people realizing the need for something. Back in the 80's, I hadn't even heard of the Internet. But I did have access to FidoNet. I used email. I thought it was amazingly cool - even profound. Although my family and non-BBS friends had no idea what email was or why anybody would want to use it.

You're right - email as we know it soared as ISPs became commonplace. However, to many people, "the Internet" was as much about email as the Web. Email being the killer app and all that. Many people sought access to the Internet because they wanted to use email. Even if Internet access required learning new technology and buying new hardware and software.

These days, people don't understand why they need to encrypt email. So they don't invest any effort in to getting the capability to do so. Ease of use or not.

Re:Spam solution already exists

[Anonymous Coward]

so you are too lazy to register a domain name for yourself and name your email server mail.guywhohatesbmail.com??

sorry but I really believe that if you dont have a FQDN for your email server you need to be blocked.

Ip address only email servers are not a proper email server.

a modification of lumpy's solution would work. get a FQDN and reject all email not from a FQD...

simple as that. that way the offending server can be tracked down and blackholed easily.

Re:Spam solution already exists

[Rashkae]

Would be happy too.. I would even pay Rogers a premium to have a Static IP to go with my "Business Class" broadband connection....

Oh, except, Rogers doensn't offer Static IP.. Bell offers it, for an insane premium, but they block port 25. *Ha*. And why would a registered e-mail server by any easier to block than an IP only server??

If the solution were that simple, it would have been implemented years ago.

Re:Spam solution already exists

[rthille]

Are you willing to accept email from anyone on the internet? Perhaps someone who read a post of yours on slashdot, or a web page you put up? If so, then only accepting signed email will not protect you from spam. There's nothing to keep spammers from generating new keys often and signing their spam. Once the key is blacklisted, they create a new identity (or identity web if necessary) and send more spams. Sure, it can up the cost incrementally, but it doesn't scale well.
SPF is designed to authenticate the sending domain, SMTP AUTH can authenticate the sending user in the domain (both protecting whitelisting), and HashCash can up the cost to send an email such that for non-whitelisted sender the cost is reasonable, but for bulk email it's too costly.

The only trouble with the combination of SPF & HashCash is where spammers sign onto mailing lists and use the fact that the list itself would be whitelisted by the recipients to deliver the email. That could be handled by filters and moderators.

Re:Spam solution already exists

[Ageless]

The old Netscape mail client way back in the day had S/MIME support. Outlook and Outlook Express have had it for years as well.

My sent mail only stretches back to 1997, but I have S/MIME encrypted and signed messages from that year using:
X-Mailer: Mozilla 4.03 [en] (WinNT; I)

This technology has been readily available for a long time. The reason that people don't use it is because managing certificates and their infrastructure is a pain in the ass.

Re:Spam solution already exists

[Rashkae]

I think the whole point of filtering by this technique would be to filter out messages that were sent from unknonw senders. Someone sending me a message would have to either: have their key signed by a authority I trust. B), Encrypt the message with my public key. Otherwise, the message should land in my "Unknown Senders" pile, which I can scan through if I really feel like it.

Re:Spam solution already exists

[Rashkae]

I just opened outlook and tried to use this feature. Right off the bat, there is no facility for me to create my ID or key, only a cryptic "Import" function....

So, allright, I stand corrected, it was implemented in Outlook.. If I had to come up with a less user friendly design, I would be very hard pressed to do so. (Well, ok, mutt is slightly more obscure.)

Re:Spam solution already exists

[Anonymous Coward]

That's because the S/MIME implementations in netscape and outlook were designed to have the keypair both generated and signed by a CA like verisign.

Re:Spam solution already exists

[Ageless]

This is the fatal flaw with any kind of mail encryption or signing system. It's not easy to use, and it's not easy to make it easy to use. Someone has to vouch for the fact that you are who you say you are. Current systems do this by having you apply for and receive an X.509 certificate from the likes of Verisign. Everyone trusts Verisign so if Verisign says that I am Jason von Nieda, you can be guaranteed that I am Jason von Nieda. The problem with this, of course, is that before I can use the system I have to go through that process.

If there were a real demand for this, support could be added to mail clients to quickly and easily request a certificate from a certifying authority, but you still have the problems of losing your certificate, forgetting your password, needing to send mail from another computer and so on.

This is why client/personal based authentication fails. Users just generally can't be bothered. Making it the sysadmin's burden, or the ISP's makes it possible to quickly secure and authenticate a large group of people instead of just one person here and there that's willing to spend some time on it.

Re:Spam solution already exists

[OrenWolf]

It would stop even faster if all email servers simply had a rule that ignored any transfer requests from any machine that DID NOT have a Fully Qualified Domain entry and the top lever is not mail. or mail2. or mail3.

By extension, then, you figure if only ISP email servers could send mail, spam would be greatly reduced.

Congratulations! You just explained why SPF is a good idea! The whole point of SPF is to point out which servers for a given domain are allowed to send email.

if you are a roaming road warrior then you are forced to use webmail or VPN into the network.

Not heard of SMTP AUTH? Have your ISP setup AUTH on both ports 25 and 587 (MSA, to get around port 25 filters) and you can now relay your mail, via your ISP server, from anywhere.

Re:Spam solution already exists

[pjrc]

if mail clients were configured to start using PGP (GPG) keys and signatures by default.

If you reject messages without a PGP signature, spammers will simply sign their messages.

If you reject based on the signing author being a known spammer, spammers will simply generate a new key for each message. This isn't a computational burden (as it is in PGP) if the keys aren't generated in a secure manner.

If you reject all unknown senders, people unknown in your "web of trust" will be rejected.

instruct unknown senders (people who aren't in my contact list yet) to download my public key from a specified location to encryp a message that would bypass my filtes.

These challenge-response systems are already in use today, by people willing to be rude to previously unknown senders. They suffer from all sorts of problems (out-of-office autoresponders, automatically generated shipping notices for online purchases, and other legitimate automated senders). How is adding the complexity of PGP an improvement?

The original statement is missing one important word "if [all] mail clients were configured".

This is an idea that just won't work if any significant number of senders still send out plain, old email as they currently do.

Contract with SPF and MS's Caller-ID, which are effective at detecting forgey when only the (claimed) sender publishes a SPF and/or Caller-ID DNS record and the receipient checks it.

To be successful, an change needs to provide value for those who have implemented it, even before widespread adoption occurs. SPF and MS Caller ID meet this goal, as people checking SPF and Caller ID can already reject forgeries claiming to be among 14000-some domain names (including AOL and Hotmail), and domain name admins who publish their SPF and/or Caller ID records are already receiving some protection from having their names forged.

The scheme "let's all switch to PGP" doesn't allow receivers to filter out messages until almost all legit senders sign their messages, and senders who sign their messages don't get much value from the signature until almost all receivers reject unsigned messages. Even then, PGP still suffers from the problem of unknown signatures, requiring a massive worldwide trust database or all the problems of callenge-response negotiation for new contacts.

PGP is an excellent answer to protecting privacy and proving strong authentication. Saddly, crypto only answers the question "is this message certainly authentic", but the question that needs answering is instead "is this message almost certainly a forgery".

Mailserver IPs listed in DNS Record

[psyclone]

IF the mailserver IPs have to be listed in the dns record of the from domain, then perhaps some records will contain hundreds of zombie hosts. Various parties would be interested in this information...

breaks forwarding

[close_wait]

I dislike SPF because it breaks forwarding. There is a "workaround" but that's required on every MTA in the world that allows forwarding, and is intensely ugly - it requires adding a bunch of garbage to the sender address, and also requires the MTA to main a cache of forwarded addresses so that bounces can be passed back down the chain.

The problem is this. Suppose AOL start adding SPF records to their DNS, saying effectively 'only the following IP addresses are authorized to send @aol.com emails. Suppose also that Hotmail start rejecting emails from SPF domains where the IP addresses don't match. Now suppose that joe@small.biz is going to be away from the office for a couple of weeks, so he gets the small.biz mail server to forward his emails to his hotmail account. At this point anyone from AOL who emails him will find the emails bouncing (although if they're from AOL, this may not be such a bad thing...)

Re:breaks forwarding

[Anonymous Coward]

No it doesn't - SPF works on the _envelope_ headers and the fact that the forwarding machine is declared OK to send. Rewrite the envelope header (the one you never get to see in most mail clients) and make sure forwarding happens on a machine SPF-permitted for the rewritten address (or is routed via a mail gateway permitted by SPF).

Re:breaks forwarding

[close_wait]

If you rewrite the sender envelope address you break error returning - which is why in the presence of SPF all MTAs capable of forwarding now have to maintain a 5-day cache of all sender address rewritings.

And in my original example, how is small.biz supposed to get AOL to add small.biz's MTA to the list of valid sender IPs - just on the off-chance that one day an AOL user may send an email to small.biz that gets forwarded?

Re:breaks forwarding

[randomencounter]

AOL _does_ publish SPF records. They default to "unknown" currently so they won't be dropped.

Of course, if I have to choose between having useful e-mail and having e-mail forwarding, I'll lose forwarding.

Re:breaks forwarding

[WoodstockJeff]

True, SPF would break this cludged-up solution to the wrong problem. The problem would seem to be that joe@small.biz can't properly access his mail server while on the road, and fixing that would be the "proper" solution.

Mobile access to a mail server isn't the big deal it was 5 years ago. If you can tolerate a web-mail interface like HOTMAIL, you're 90% of the way there; I have found over a dozen web interfaces that will work with arbitrary mail server, and the resulting mail comes FROM joe@small.biz, rather than joe12342233123@hotmail.com, confusing his customers.

Chosing the right mail server software is another method - our postfix+dbmail system will accept outbound mail from our customers from any routable IP in the world (even ones on our block lists), so long as they authenticate themselves, which is as simple as access their POP3 account prior to sending. That starts a timer for approved access. And there are more sophisticated ways of doing it, such as TLS, that have wide client support.

SPF isn't a perfect solution. It is a piece of an imperfect puzzle, and it doesn't hurt to publish the records, unless you fall into one of the narrowly-defined exceptions that others have pointed out... And, in my reading of one long piece on why it is better to replace SMTP than to implement SPF, I saw a long list of "SPF breaks this" problems that were also imperfect solutions to the problem they addressed...

Re:breaks forwarding

[Anonymous Coward]

There are two ways Joe could solve this. First, he could add:

small.biz. IN TXT "v=spf1 +all"

to the dns for small.biz. This says that any computer on the internet can has the authority to send mail as if it came from someone @small.biz .

To add significantly more security, Joe could add

include:hotmail.com

to small.biz's existing spf record. This says that hotmail's mail servers have the authority to send mail for small.biz users.

However, the proper and most secure way would be for Joe to find a way to use his work computers to handle his email. This could be by installing a webmail server, by setting up a VPN to his office, using remote desktop, etc.

The way I see it...

[RAMMS EIN]

The way I see it is that SPF and caller ID provide one service: they ensure that the From: header is "correct" in the sense that it really represents that user at that domain.

My question is why we don't do this the same way we do it everywhere else: through authentication. If the would-be sender can't authenticate to the server (through PKI or password), they don't get to send mail. Now you know that when you get a mail from j.r.hacker@2600.com it was really someone who could authenticate to 2600.com as j.r.hacker. It works for your INBOX, why not for sending mail?

Re:The way I see it...

[randomencounter]

What you are describing is SMTP AUTH. It works nicely for sending e-mail.

SPF is used so that the receiver can verify that the host it is receiving the e-mail from is authorized to do so for the domain, thusly:
SMTP server gets connection from zombie.bigISP.com
zombie claims to be sending mail FROM example.com
example.com's SPF record says that zombie.bigISP.com is not authorized to send mail for example.com.
You get to refuse to accept the mail, mark it as spam, or whatever you please with it.

Simple, eh?
Most importantly, SMTP AUTH makes SPF easier because it lets you have your remote users use your authorized mail servers without making them open relays.

Re:joke. Helllo, McFly? :P j/k

[Anonymous Coward]

Look, McFly. *slaps* It STOPS your identity from being displayed. Hence it's a .:***FUCKING JOKE***:.

Sorry if I'm not overjoyed...

[Sebby]

...since MS is part of it.

I can only assume they'll eventually charge some sort of licensing of other kind of fee, and they'll force their way to approval to displace Yahoo's proposed system.

Caller ID

[supe]

WTF, I thought I owned the patent on that?

Re:Caller ID

[spectro]

Seriously, if somebody thinks about patenting SPF, I proposed the concept on 2001:

A flag in DNS MX record to indicate IP addreses of servers allowed to send email from their domain, SMTP servers will reject mail from IPs not in that list, this will get rid of all these spammers spoofing hotmail, yahoo, etc. accounts.

Perhaps an RFC is in order

[Sebby]

Good idea. Someone want to write up an RFC and submit it to the IETF?

Pobox.com antispam working like gangbusters for me

[DeadVulcan]

Well, I'm a pobox.com customer, and my own experience of their new antispam measures is absolutely nothing but fantastic. They recently overhauled their spam filters, and the result (again, this is just my experience) has been stunning.

Of course, this says little about SPF itself, but at the very least, for what it's worth, the company that invented it comes with my recommendation.

...until 100% compliance is reached, non-SPF mail will still have to be accepted.

Well, the way pobox.com has done it, you can choose to have your E-mail "flagged." SPF is one of those possible flags. If an E-mail gets X (a user-definable number) or more flags, it can be rejected as spam. This makes SPF useful even when there isn't 100% compliance.

How will I be able to send mail using my own business' domain, as I do today, when it is going out via an ISP server?

I would think that if your ISP is interested in doing honest business, they would make the effort to list their own mail server.

If you're running your own mail server, then, yes, this is a valid concern.

The only way to kill spam is to stop having all email be totally, absolutely, "free" of charge in any quantity.

I don't deny that that would be a very effective way, but I don't agree that it is the only way.

Legislation hasn't helped yet

[swb]

Secondly if they have to buy domains they need to pay for them - that leaves a physical paper trail to spammers, now legislation can help.

Sorry, but I have to hit the bullshit button. Legislation hasn't helped yet, and I'm not talking about CAN-SPAM or any of the other anti-bulk mail bills, but the existing laws dealing with all manner of fraud, FDA regulations and any of the other various and sundry state and federal laws regulating the almost-universally fraudulent commercial content of spam.

You're suffering from the same delusion that many people, myself included, often suffer from -- "Can't we pass a *law*"? -- when there are many good laws already on the books that better deal with the problem in general.

I'd suggest a RICO investigation into some of the top-level spammers, their clients, and the people involved in the payments, the network access, and find out how dirty they really are. I don't know, but I suspect, that most of these people know they're involved in deeply fraudulent activity. A few racketeering convictions involving major ISPs, banks, spammers, and their business clients with some noisy investigation of other spammers could have a *real* impact -- squeezing the spammers out of ISP suppliers and banking services they literally can't do business without.

Re:Legislation hasn't helped yet

[Steve B]

If nobody has gotten the government's attention with the obvious fact that the filter-cracking junk appended to spam is a perfect terrorist comm channel (you can't do traffic analysis on a signal that is going to practically everybody), what would it take to get their attention? Another 9-11 in which such a communication is one of the dots that were not connected?

Too late !

[bpcz]

Most of the spam I received today came from valid addresses, controled by some virus. Neither SPF or Called ID methods can prevent a virus from sending spam.

Re:Too late !

[ahodgson]

Yes they can. The viruses for the most part aren't sending from the real address of the person whose computer they're on. They're forging other addresses found on the machine as the sender address.

dynamic dns users

[tacocat]

How will this effect dynamic DNS users who send email? I'm not talking about some rogue spammer, but the people who have legitimate servers running on real IP addresses with domain names that are managed by the likes of dyndns.org

In the past, these DHCP hosted addresses have been under a lot of grief with people erroneously RBLing them simply because they are DHCP (like it ever really expires!!!) managed IP addresses.

Much of the workaround for this has been to RELAY all the email up to the ISP for delivery from a non DHCP hosted IP address. But some people block these because they show evidence of being relayed by anyone and hence must be evil.

So what will have to do in order to get my mail server considered acceptable for sending email under this SPF/CallerID scheme?

I'm also really curious to see how this can be a good thing at the same time that it involved Microsoft, but I'm trying to keep an open mind on this one...

Re:dynamic dns users

[ahodgson]

In theory, SPF should make it easier for these people to send E-mail. They can publish a valid SPF record for their domain, which should make mail from their system more trustworthy than mail from dynamic IP space is generally. Ie, the reason people block mail from dynamic IP space is because of the incredible amount of crud coming from trojanned Windows machines in that space.

If a real sender can somehow distinguish themselves via a valid SPF record, they might actually have better luck sending mail than they do now.

Re:dynamic dns users

[Rich0]

I guess as long as an SPF record can be published at below the top domain level (ie for joe.homedns.org, for example), and as long as dyndns supports it.

I'd hate to see this become a system for extracting domain/certificate/whatever income - a la ssl certificates...

XML ?

[defsdoor]

Wasn't XML last century's buzz word ? The old saying that "when the only tool you have is a hammer - everything looks like a nail" seems totally appropriate.

Except for Microsoft's embraced and extended email

[kyz]

where all you get is (if you're lucky) a text version of the email message, then a WINMAIL.DAT uuencoded or MIMEd attachment, which contains all the useful data in a proprietary binary format.

Rather than simply create compliant MIME mails, Microsoft uses this secret format to say "yeah, we'll try and send email, but if you really want to communicate with companies that use Exchange Mail Server, you need to buy a copy of Exchange Mail Server".

And this is different from the GPL???

[Anonymous Coward]

Use of this technology requires submitting to a Microsoft license. This license allows distribution (but not re-distribution), and is not compatible with the GPL. That is to say, no GPL mail server will ever be able to directly impliment checks for this.

And this differs from the GPL how?

Use of GNU technology requires submitting to a GPL license. This license allows distribution (but not re-distribution), and is not compatible with Microsoft EULAs. That is to say, no Microsoft mail server will ever be able to directly implement checks for this.

Touché Microsoft methinks

Re:And this is different from the GPL???

[Anonymous Coward]

Use of GNu technology does not require submitting to the GPL. Look at GCC for instance, you don't have to agree to the GPL to GCC as your compiler - but if you want to redistribute GCC then you need to comply.

Re:And this is different from the GPL???

[Allen Zadr]

The GPL explicitly allows distribution and redistribution, based on the GPL alone. You do not have to directly inquire to the FSF to re-distribute GCC, or any other GPL application.

However, the Microsoft redistribution does require a 'check-in' directly with Microsoft (a single entity), and not just an agreement to a portable license.

Further, because the license is tied directly to Microsoft's control (as opposed to the "change, and re-distribute" permissiveness of the GPL), the technology and specification can not be extended and manipulated except by Microsoft.

IANAL, and I immediately found those two differences that makes this different (and incompatbile) from the GPL.

Re:And this is different from the GPL???

[Allen Zadr]

Technically, use of the Email-Caller-ID technology does not require submitting to the license either. However, developing an Email server that will implement the functionality of Email-Caller-ID will require submitting to the license agreement.

For Sendmail, someone will come up with an external "milter" that complies to the MS license. For Exim and PostFix, similar external filters will be developed. However, these filters will not be able to ship WITH these mail servers, because of the incompatability of the implimentation license.

This will, of course, only make it harder to set up your Mail server. My current checklist includes - latest patches, ClamAV hooks, SpamAssassin hooks (all from separate projects). In the scheme of things, this would become just one more thing to track down. But, it would be so much better if it was merely a built-in. (Advantage, MS Exchange).

Regardless, the MS agreement is still different from the GPL. See my reply to the grandparent post.

What if.

[man_ls]

What if, say, businesses started showing up promising "unrestricted email" to get around SPF.

They set their SPF to everyone/everyone...or something.

Then it's an open relay with an SPF signature that matches.

and we're back to square 1.

Re:What if.

[mooingyak]

...and then they get blacklisted. Problem solved.

Re:What if.

[WoodstockJeff]

If any domain owner out there specifies "+all" in their SPF record, they deserve the reputation they'll get as a spam-friendly outfit. It would be better to NOT publish SPF records at all than to specifically authorize anyone in the world to claim to be from your domain.

What SPF does is say, "For mail claiming to be from our domain, if it came from one of the following locations, we have vetted it's authenticity to the extent that we feel appropriate." The domain owner can then figure out how they want to enforce their authentication - we make sure that people sending under our domains have logged into the server if they want to send, even if they're sending from within our network. No log-in, no SMTP...

Publishing a wild-card SPF record would be saying "We don't give a damn about mail claiming to be from us." Personally, if I were writing the SPF verification code, I'd tag any domain with "+all" in their SPF record for rejection...

Re:What if.

[taustin]

More likely is that spammers will take the doznes, or hundreds of domain names they register at a time now, and that they run their own DNS for now, and simply automate adding SPF records for, based on their current IP address.

No "email for this domain is allowed to come from these machines" program that is under control of the domain owner has any hope of reducing spam, because spammers will just use it, and keep cycling through disposable domain names at $5-10 each. Only .mail solves that, and that's at the expense of giving someone else control over some of your email settings (and paying them $2000 for the privilige, per domain).

In the short term, if whitelisting based on SPF becomes popular, it will increase the amount of spam coming through. In the long run, it will have zero effect.

Re:What if.

[williamhooper]

And this works better than the currect blacklisting crap how?

This is just adding one more meaningless hoop to jump through. It's all well and good for the people that want to play by the system, but just a very mild annoyance for the people that don't.

Oh, and how to turn it off.

[kyz]

How to actually send compliant, fully functional email instead of encumbered, lock-in Microsoft crap.

Why don't Microsoft set this by default? Email is email. People have got to learn that Microsoft are responsible for this abomination, and the hassle required is Microsoft's fault for not complying to the standard.

Re:Oh, and how to turn it off.

[Christopher_G_Lewis]

The information in this article applies to:
Microsoft Exchange Windows 3.x client 4.0
Microsoft Exchange Windows 95/98 client 4.0
Microsoft Exchange Windows NT client 4.0

Aren't Windows 3.1 & NT 4.0 at least 10 years old?

Come on, MS screws up a lot of stuff, but don't troll out on the fact that MS didn't get the internet in the 90's.

Re:Oh, and how to turn it off.

[kyz]

It has nothing to do with the client. It is the Microsoft Exchange SERVER that sends these fucklumps out. We still get WINMAIL.DAT files to this day. People are still sending them.

Nothing annoys me more than having to save off the WINMAIL.DAT attachment and run it through a TNEF unpacker just to get the REAL attachments of a mail. OK, nothing except people who send me MS Word documents with nothing but ASCII text inside them. And, of course, people who send me ASCII text inside a MS Word document inside a TNEF attachment.

I knew it...

[Daneurysm]

Knowing microsoft, they're gunna toss this thing in there as an afterthought...much like real caller-id.

In fact, how surprised would you be if it was just a 1200-baud half duplex signal leading every email?

how to get rid of spam

[sk8king]

Get rid of HTML tags in message bodies and preview panes in Outlook Express. Previewing messages with HTML just leads to confirmations that the email has been delivered to a certain address. Address confirmed, put it on a list somewhere.

I mean, how many legitimate emails from friends/business partners do you get that have big red letters.

Plain text is all you need. Readers can then be written to identify URL's and allow the user to CLICK them.

AOL and MS say: publish SPF records

[wayne]

AOL just added a webpage saying that you should publish SPF records if you want to be whitelisted with AOL.

The MicroSoft Caller-ID/SPF merger proposals say that SPF records will be honored, so you can publish them without fear of losing support.

So, go ahead and publish SPF records.

MicroSoft supporting SPF records is a really smart move. Last week, I posted results of a survey of 1.3 million email domain names to the IETF MARID mailing list. Now that I'm back from the MARID meeting, I just finished a survey of Caller-ID records. There appears to be about a factor of 500-1000 more domains that have published SPF exclusively than Caller-ID exclusively and only a tiny fraction of the 1.3 million domains have published Caller-ID records. In short, MicroSoft isn't changing to support SPF records because they are better (I think they are), but because it is an acknowledgement that MicroSoft's Caller-ID hasn't caught on.

Meng Weng Wong (the SPF author) and MicroSoft are still discussing how exactly this merger will work on. I personally don't see any reason to support XML right away. MicroSoft has not come out with a single concrete extention that can't be done with SPF already.

I also think that there are alternatives to the complex Caller-ID algorithm and that doesn't require every Ezmlm and other mailing lists to upgrade their software. From the research that I've done (and yes, this is something I have really researched), there appears to be far more mailing lists broken by MS's Caller-ID system than email forwarders broken by SPF.

(I'm the author of libspf-alt and the maintainer of the trusted-forwarder global whitelist. So, now you know why I have researched this stuff so much.)

SPF To Be Integrated With MS 'Caller ID' System?

[fox8118]

SPF To Be Integrated With MS 'Caller ID' System

For some reason when I first read the title I thought they were integrating sun block with the email tracking system. I must be tired or just stupid this morning.

Why XML is bad

[tacocat]

It's simple really. DNS is one of the highest areas of traffic and hits out there. Every web page generates multiple DNS hits and so does email and P2P and everything else.

XML, is a bunch of text that wraps around a bunch of data and is called meta data. It's not the data you need, but data about the data you need. In DNS, you already know what you need, so the "meta" is silly.

Point being, you add a lot of extra characters to the data transmissions. UDP won't support it anymore so we have to to with TCP, which has even more overhead being added to the process.

Compound this with MSFT's tendency to send shitloads of data across every network they touch just because they can, and you've DDOSed the Internet.

XML may have a place, but DNS sure as hell isn't it.

Re:Why XML is bad

[Anonymous Coward]

Any benchmarks to back that up? There's all kind of validation done around a DNS records already. A simple XML parser wouldn't add much to it, I think.

Re:Why XML is bad

[Anonymous Coward]

From Microsofts point of view it is a perfect fit. They have several patents on XML and more coming. The extra bandwidth and processing required is well worth it.

Re:Why XML is bad

[tacocat]

Benchmarks aren't required. Just think about it.

If I ask for a data element that is of the form "123.234.345.567" in response to a DNS lookup, and you go to XML, the response will be along the lines of:
<IP:123.234.345.456> or <IP Address>123.234.345.456</IP Address>
Both of which require more data to be transmitted. If you consider the HTML code generated by MSFT applications, this will probably turn into a 1K data transmittal.

It's the same as asking for someone's phone number and they have two responses:
"My telephone number is 1-123-555-4567"
or
"1235554567"

You've already admitted the problem when you say wouldn't add much to it. You are looking at a HUGE resource load application and anything that's not much when multiplied by the number of DNS queiries going on every day turns into a much.

Don't forget to consider the added network overhead of the additional bits in additon to the added overhead of not being able to use UDP and forced to use TCP instead. Now you've compounded the problem even further.

How big does not much have to be before we are all forced to upgrade to gigabit ethernet from our current PPP connections?

Moderation Comment

[Anonymous Coward]

I am wondering how a first post can be rated "Redundant", and while this post is clueless, it doesn't seem like a troll.

Microsoft employing open source ideals?

[PHP Wolf]

From the Caller ID for Spam page:
"Send your comments. We are circulating this initial technical specification for comment because we believe that your feedback can help make it stronger."

Is it just me, or does that process sound dangerously close to open source software development? ;-)

How will the spammers fight back?

[WoodstockJeff]

I take it you either don't operate a mail server, or you don't monitor it. At the present time, over 90% of the spam attempts against our servers come from open proxies around the world, most likely home machines that have been infected by SoBig, Netsky, or whatever worm-of-the-day is installing proxy software at the moment. The spam is coming with envelope senders not associated with spam, in general, because it's forged - heck, lately, nearly 5% of it is coming in with the envelope sender forged to match the mail to address!

The percentage of spam that comes through a server that is associated with the domain cited in the envelope sender field is very low, even those "throwaway" domains you mention. Granted, if SPF catches on, some of the more savvy spammers will use throwaways to associate some of those proxies with, but that increases their cost in time quite a bit, and you can block proxies by other means (dial-up and open proxy lists are available from several sources).

Re:Sounds like a truly awful idea -- Filter spam!

[iamcf13]

I repeat what I've said before. The only way to kill spam is to stop having all email be totally, absolutely, "free" of charge in any quantity. This is not the topic to discuss solutions, but they are certainly possible, and they aren't SPF.

Filter spam out and preserve the current email infrastructure.

Wow! Caller-ID!

[WheelDweller]

Should someone tell him that there are headers on email?

The problem is that everyone CAN send email...wether they know they're doing it, or not- because the spammers have bought their congressmen, and they spend a lot of time learning about computers that congressmen don't.

EVEN IF we went to a regulatory system by which you were licensed to send mail (and some key kept you from it) the spammers would still be able to bribe a key.

It's time for alternatives. Not for the entire world, for the rest of us. Leave SMTP open on the entire world, and antispam the hell out of it. But make another mechanism, running, let's say, under Linux, that we can count on. Who cares if it doesn't work in Tangique or Morocco; we'll make sendmail plugins for special cases.

Let's stop waiting for Microsoft to write their own standard, 'cause it's gonna be Linux-unfriendly. And you can COUNT on it involving the personal data that Microsoft "doesn't" have access to. (See also: tracking Word Documents to their source)

Re:AOL and MS say: publish SPF records

[nexus987]

It looks like microsoft isn't even publishing records for their own domains (microsoft.com, msn.com, hotmail.com)..?

Huh?

[ThisIsFred]

For the most part, I'd say the solution is simple. Both the sender and recipient need to authenticate to send/receive messages. It's a very logical conclusion, considering how people currently avoid unwanted messages. Consider how most of us deal with telemarketers: People read their caller ID. "Is it someone we know? Nope." Why can't we do the same thing with e-mail, but with a service that makes the determination (where a passcode replaces a phone number as the ID)? Does the sender have the proper sender auth? No? No delivery for you!

This is the best possible solution for the people (ex: me) that really rely on e-mail to do their jobs. It sucks for those that wish to fill your inbox with junk. It doesn't really matter to me at this point, we buy one product from a vendor and they're sending ads to my inbox on a daily basis. The marketers can have their own separate protocol from e-mail, but no one can force us to use it.

Mod parent up and read the article

[John Starks]

The article makes some great points, and I'm disappointed that the parent is currently only a 2.

Back to your Chair TIMMY

[sPaKr]

JEBUS timmy you need to learn a new word other then your name, thats DUPE. Now back to your chair you short bus rider!

Re:AOL and MS say: publish SPF records

[swmccracken]

They're not as easy to find as traditional SPF records. Do a TXT query for _ep.hotmail.com.

"<ep xmlns='http://ms.net/1'testing='true'><out><m><ind irect>list1._ep.hotmail.com</indirect> <indirect>list2._ep.hotmail.com</indirect><indirec t>list3._ep.hotmail.com</indirect></m></out></ep>"