Microsoft Releases Patent on SenderID

wayne writes "Microsoft has now put the SenderID patents under the OSP. The Open Specification Promise was discussed on slashdot before in conjunction with web services and it is good to see that they are opening up even more. There are still technical problems with SenderID compared with SPF and, of course, SPF isn't problem free. Still, over the last year, the number of SPF records has more than doubled from around 1.7 million to 4.1 million, with rate of growth increased in the last 6 months."

Comments

Why the hell did Microsoft have to go and...

[Avillia]

...Make a new grading scale for suntan lotion? I mean, honestly, we've already got Sun Protection Factor, we don't need some retarded system like SenderID... Hell, we don't even need SPF, idiotic parents just can't think of their children and get the thick blue paste that WORKS instead of this new THE PURPLE FADES IN crap.

Honestly.

Re:Why the hell did Microsoft have to go and...

[Fred_A]

Maybe children like it better because it comes from Barney ?

Have they released a SenderID SDK?

[Bucaro]

Although I may not be a fan of M$, I am a fan of anything anti-spam. How much coding does it take to make ones own email client, Mail, Thunderbird, or whatever, work with a senderID tag?

Re:Have they released a SenderID SDK?

[grcumb]

Although I may not be a fan of M$, I am a fan of anything anti-spam.

I'm not. Not a fan of anything at all, that is. I'm a fan of open systems (preferably officially endorsed standards) that are well understood and secured for use many years into the future. SMTP, for all its baggage, is one standard that has actually aged fairly well over the years.

There are fundamental flaws, of course, and now these flaws are costing us a lot of money, time and effort trying to stop people from preying on the system and on human naïveté.

Microsoft's approach to this can be summarised as, "Hey gang let's all get together and fight spam my way!" This is okay, but in the opinion of this hoary old curmudgeon, I'd rather people said, "Hey gang, let's all get together and figure out how to fight spam!" There's a small but integral difference between those two statements. It lies in the potential for Microsoft to stop in mid-fight, take its ball and go home.

What Microsoft is trying to do with this latest move is to convince the world that it will not do this. I'd like to believe that's true, but their track record gives us every reason to believe the opposite. Even if they're perfectly sincere about this right now, people will still be suspicious that at some time in the future they might try to lock things down again.

It's unfortunate that we have been led to feel this way, and I suppose it's never to late for a leopard to change his spots. I doubt this one will, though.

Re:Have they released a SenderID SDK?

[Keeper]

Email clients are not what SenderID is for: it's for mail servers, to reject the spam before it even gets into the user's cue.

SenderID can be implemented on both mail servers and clients.

Unfortunately SenderID is not only patented, the Microsoft license prevents other people from modifying it for other uses. This means it should not and cannot be used in Sendmail, Postfix, or other open source MTA's due to license restrictions.

Wrong: http://www.microsoft.com/interop/osp/default.mspx

SenderID is also cryptographic. This prevents software with it integrated from being exported to "restricted" companies, due to the strange rules about encryption being a material of war.

SenderID has no cryptography. You're thinking DomainKeys.

SenderID is also fundamentally broken: SPF rejects spam messages in a way that is very lightweight and free to implement (publish a TXT record in your domain's DNS), and rejects the message before its contents are even sent, based on the "FROM" line used for email bounces.

Incorrect. Both SenderID and SPF are based off of DNS TXT records. The primary difference between the two is that SenderID validates that the FROM field has not been forged, while SPF validates that the return path has not been forged.

SenderID requires purchased keys from Microsoft, and requires the MTA to accept the email message to process the SenderID key, which seriously burdens the server.

SenderID basically has nothing to do with SPF or anti-spam: it has to do with selling keys for bulk emailers, legitimate or not, to send bulk email while avoiding anti-spam messages. Its presence in a message is actually a very powerful sign that the message is spam, just as those "Haiku" messages in email headers used to be.


SenderID has no cryptography. You purchase nothing from Microsoft. You're thinking DomainKeys.

Unfortunately, the creators of SPF accepted Microsoft sponsorship and involvement with SenderID to get Microsoft support, integrating SPF-like features into Hotmail and other Microsoft tools in order to get a larger user base, but unfortunately accepting a corrupt influence that has actively hindered the acceptance of SPF.

Blah blah blah, insert Microsoft is teh big evil rant here. You should learn what you're talking about before complaining about something it doesn't do.

Re:Have they released a SenderID SDK?

[sgtrock]

I recommend that you spend a little time studying the history of the Internet before making these kinds of statements. Spend even more time studying how electronic mail was originally designed, and how it has evolved. I think you'll find that far from being 'cruft', a client/server model was exactly the model that the original designers wanted for a variety of very good reasons.

The Internet didn't start with the Web, after all.

Brr...

[Mikachu]

It's a little cold in hell for this time of year, don't you think?

Re:Brr...

[gbobeck]

Well, lets see. Hell, MI has the zip code 48169 and is located at 422605N, 835906W. Using Yahoo weather, we can lookup weather for that zipcode.

from http://weather.yahoo.com/forecast/USMI0672.html , the weather on 23 October 2006 at 12:15 am EDT is:

Fair, 34F.Barometer is 29.93 in and steady, and 87% humidity. There is a 7 mph west wind.

Of course, conditions will change, so keep on watching for Hell to freeze over.

Re:Can we get the FUD tag now?

[nmb3000]

More MS FUD about being open

What? Do you even know what FUD is? Fear Uncertainty and Doubt. It's usually meant to mean the kind of news Microsoft might release saying "OMG Linux is insecure!!!~" or SCO saying "WTF Linux newbs must pay money or we'll sue!!!". Microsoft trying to show some interest in open standards certainly does not qualify as FUD, especially since this isn't the first open stuff they've done.

• (no smoke without fire)
  
• Not to be the boy who cried wolf
  
• count the brass tacks
  

I think we have a finalist for the category 'Most Useless Cliches in a Slashdot Post'. Congratulations, however I've never heard of actually counting the brass tacks (though it appears I'm not alone) :)

Sender ID, SPF, DomainKeys

[bcrowell]

So now we have Sender ID, SPF, and DomainKeys.

AFAICT, they all aim to accomplish similar things. Unfortunately, there's no consensus on which to use, and that means that they're all basically useless. One of these mechanisms would only become useful if virtually everybody used it, because then people could refuse to accept e-mail that didn't use it. Gmail and yahoo both use DomainKeys, which suggests that it's something that can really be implemented successfully in the real world. Looking at the Wikipedia articles, Sender ID seems to have problems because it breaks preexisting standards (see "Standardization issues"). My impression is that a lot of people looked at DomainKeys and said, "oooh, scary, it uses crypto." But hey, this is 2006, not 1992. Strong crypto is everywhere. Is there any reason not to go ahead and standardize on DomainKeys?

Re:Sender ID, SPF, DomainKeys

[ldspartan]

DomainKeys doesn't break forwarding and... you know, SMTP. DomainKeys doesn't require mail servers to rewrite the headers on every message ever.

In short, DomainKeys wasn't designed by idiots, while the other two apparently were.

I'm unbiased! pffffft :P

--
phil

Re:Sender ID, SPF, DomainKeys

[Zeinfeld]

Actually Sender-ID and SPF are the exact same thing. Both allow the sender to describe their email sending configuration in identical terms. The only difference is what receivers decide to do with the information. That part is outside the scope of any sane spec since the whole point of spam control is that the recipients not the senders get to decide.

There is a big difference between Sender-ID and Domain Keys, Sender-ID uses the IP address of the outgoing email server. DKIM uses public key cryptography. We knew at the start that it would take about four years to agree a cryptographic standard hence the decision to adopt a two track approach.

This is not a VHS vs Betamax competition. There are genuine differences in the specs. If you are going to deploy one you have to do much of the work required for the other.

One of the core problems in MARID was that most of the people involved had little experience of the standards process and no inclination to accept reasonable compromises. Another problem is that the IETF rushed the formation of the group in order to prevent a rival standards body moving in on their turf. This pre-empted the negotiations I was moderating in an attempt to agree on a common proposal before the working group was chartered. As soon as the WG was chartered with an open charter the way was open for third party groups to introduce additional proposals even though they had no support from any constituency.

The original patent license terms were not unusual or unreasonable. It was just that a number of persons decided to make an objection in this case to a practice that nobody had objected to for over a decade. As a result of the SCO case the patent lawyers at several large companies (not just Microsoft) had determined that the reciprocation clause in the traditional open patent license was probably not enforceable if there was an open sublicense clause.

Some people decided to make SPF the place to fight this particular battle and started making unjustified accusations of bad faith on Microsoft's part. Then a splinter group decided to exploit the situation and propose a completely unrelated specification that had no commercial support whatsoever.

The point that was lost on many participants was that the only reason to go to a standards body is to get buy in for a proposal. If you want the best technical proposal you should not involve more than five people in the design.

Sender-ID is not incompatible with SPF as alleged. The only difference is at the recipient side and the recipient cannot be forced to interpret SPF or Sender-ID in any particular way. We had agreement in the WG to proceed on a common spec and nobody found any problems until the patent issue was raised.

Re:Sender ID, SPF, DomainKeys

[RyanAXP]

SPF most certainly was not written by idiots, although MS's wacky SenderID carries the distinctive odor of cretinism. What you should perhaps understand is that SPF and DomainKeys/DKIM are complementary to each other, while SenderID bears all the hallmarks of yet another "just incompatible enough" Microsoft "extension" to SPF.

Re:Sender ID, SPF, DomainKeys

[DA-MAN]

DomainKeys doesn't break forwarding and... you know, SMTP.

DomainKeys breaks a lot of things. As one of the maintainers of the QmailToaster project, I've run across a lot of people where DomainKeys breaks their entire setup.

1) If you forward your mail to an upstream server (sendmail smarthost, Exchange SMTP Connector, etc), DomainKeys will always be void.
2) If you have a backup mail server or a scanning mail server that receives and then transfers to your primary mail server un-modified (IE doesn't remove the DomainKeys) then your main mail server will reject it.

DomainKeys sucks. SPF sucks, SRS is a hack.

Re:Sender ID, SPF, DomainKeys

[DA-MAN]

For the record Google also checks the SPF, though I'm not sure if they actually do anything with it (as I've seen messages that fail still get through)

The following is from one of my emails:

Received-SPF: pass (gmail.com: domain of ***@yahoo.com designates 68.142.206.106 as permitted sender)

That's peculiar because Yahoo! doesn't publish SPF records.

Typical SPF Record:

$ host -t txt gmail.com
gmail.com text "v=spf1 redirect=_spf.google.com"
$

Yahoo!

$ host -t txt yahoo.com
$

Re:Sender ID, SPF, DomainKeys

[statemachine]

Actually Sender-ID and SPF are the exact same thing. Both allow the sender to describe their email sending configuration in identical terms. The only difference is what receivers decide to do with the information.

But it's a huge difference for the receiver, whether or not you feel it's sane.

1) SPF regulates the envelope sender. Sender-ID regulates the TO: field.

2) SPF can be used to reject incoming messages before any data is sent. Sender-ID has to (at least) wait for the TO: field to be sent along with the rest of the DATA part -- which doesn't limit bandwidth consumption very much.

3) If the MTA isn't going to reject messages and only add to the score, then Sender-ID will be fine for you. If you want to reject messages to avoid tying up your MTA (and lower your bandwidth consumption), SPF is the way to go.

And for the parting shot (not against the parent), DomainKeys is just too much of a load on a busy server, IMHO, because it requires computing a hash for every single message. It just doesn't scale. It has other severe problems too, but I saw them adequately discussed earlier.

Re:Sender ID, SPF, DomainKeys

[Just Some Guy]

That's peculiar because Yahoo! doesn't publish SPF records.

The default is to guess at permitted relays for a domain, so smtp.example.com would be allowed to forward @example.com email. Perhaps it should have read:

Received-SPF: pass (gmail.com: domain of ***@yahoo.com designates 68.142.206.106 as permitted sender, or doesn't designate anything at all but got lucky this time)

but that's a bit more verbose.

Re:Huh?

[Shados]

Office probably has crazy R&D and coding costs(even if you find the quality of Office to be lacking, it doesn't change that, outsourcing aside, the programmers coding it for MS don't work for peanuts...they probably make 2-3 times what I make, and I don't work for peanuts myself!), so recoding the whole damn thing (even if you port the Mac version, I'm sure it uses a lot of Mac-only API), plus support, etc, it most likely would come down to a loss, or a very tiny profit margin: not worth the customers they'd -lose- from people who would move from Windows to *nix when they see Microsoft's alternate products available there.

You disgust me

[suv4x4]

This is Slashdot, and there's not even ONE anti-Microsoft post modded up!

Re:Why not just fix Windows?

[ATinyMouse]

I realize this is Slashdot and Microsoft is a convenient scapegoat, but I have to disagree with your statement. Compromised Windows systems may play a large roll in SPAM delivery today, but they aren't the root of the problem. If you want the root, look at any ISP that allows unauthorized hosts to send mail. They deserve far more blame then Microsoft does. You'd think with the cost of bandwidth, the tools available for detecting, and the problem with SPAM today, ISP's would be doing everything they could to tighten up their network. It doesn't really cost anything to put in blocks on port 25 and only allow traffic from authorized hosts, like their own email servers and customers paying for that capability.

SPF/Sender-ID is great in theory

[jonwil]

However, until people start saying "these are the only mailservers permitted to send mail for my domain, anything else should be rejected outright", mailservers wont reject mail from support@paypal.com sent from paypalscam.ru.

nice, but lacking teeth

[oohshiny]

The terms of the OSP "promises" seem fine: irrevocability, general applicability, etc.

The trouble is that it's a "promise". A "promise" on a web page is not the same thing as a legally binding commitment.

The proper thing to handle this would be for Microsoft to submit the specification to a standards body with a legally binding contract and steep penalties should Microsoft break the contract and take legal action against anybody implementing the specification.

I can't tell why they aren't doing this. It could be

* arrogance ("we're too big to have to make a binding commitment to anybody"),

* it could be ignorance ("if we promise, it ought to be good enough"),

* or it could be nefarious ("the OSP will be good enough for commercial implementors, but it's not FOSS compliant", "they think it's open and binding, but we have hidden this pitfalls in the fine print").

Any guesses?

Note that Microsoft's spec is not needed, since there are already alternatives.

When to use SPF instead of DomainKeys

[billstewart]

Look, crypto is a fine thing if it's doing what you want, and DKIM may be useful *after* a message has passed an SPF check, but they're doing different things, even though both of them are ostensibly about preventing joe jobs and other forgeries.

SPF lets a domain administrator specify that all mail from that domain will come from one of the specific servers, so you can trash crude forgeries quickly at the cost of a couple of DNS lookups, and incidentally trash a lot of phishing spam without burning up lots of CPU.

DKIM lets an administrator specify crypto keys that will be used to sign real email from that domain, so you can validate it at the cost of a lot of CPU. That's useful for checking mail that purports to be from *your* bank but might be a *good* forgery. But it's a waste of CPU for checking mail from banks you don't care about, or the 99.44% of purported PayPal/eBay messages that are fake, since you can use SPF to discard the ones sent by zombies, Chinese spammers, address-space hijackers, etc.

So maybe you want both, or maybe you'll use other methods to deal with the good forgeries. But SPF lets you trash a lot of the crude phishing spam before you do any heavy lifting. (Of course, it won't protect you from mail purporting to be from PayPalSecurityLtd.co.uk or Paypal.aq, and spammers will fight back by polluting the namespace, but it's at least some help.)

Mod Parent Up, Please

[billstewart]

Paypal and EBay are the two sites I most want to have using SPF or equivalent, because I get huge amounts of spam from them but also occasionally get real email from them if I've bought something. There are also a couple of banks that are big scammer targets, and it'd be nice to have their phish get trashed.

On the other hand, I've never had a problem with forged mail from the Bank of Nigeria, so maybe they don't need to use it :-)

Breaking SMTP not a solution

[fruey]

All of these solutions have flaws. I'm with deBoynePollard on this:

• SPF is harmful
• Anti UBM don't work

An interesting take is to make the sender responsible for storing mail: suggested by Dan Bernstein (DJB), the qmail guy.

There's always politics in it. Some people don't like DJB's attitude and they're anti-qmail and go for Postfix or sendmail.

Wietse Venema, the postfix guy, isn't too happy about SPF either: but he does provide plugins for Postfix.

SPAM needs a solution, but breaking SMTP isn't the way to go IMHO. I think a well configured email server, RBLs, requiring reasonable RFC compliance and such will eliminate much SPAM. Spending energy on evangelising good mail server configuration is still the best way to go.

Re:Breaking SMTP not a solution

[Just Some Guy]

An interesting take is to make the sender responsible for storing mail: suggested by Dan Bernstein (DJB), the qmail guy.

As per typical DJB ideas, it's broken and only implements half the functionality of what it intends to replace. I've used this example before, so skip this if it sounds familiar:

A friend of mine hosts a customer that sends weekly newsletters to about 25,000 subscribers. With SMTP, my friend can spool the whole set and then watch as the mail queue flushes over time (measured in a small number of hours). It takes advantage of the fact that if 10,000 of those newsletters are going to @example.com addresses, it can deliver all 10,000 of them at once. In any case, his system delivers mail at the pace it can handle.

Enter DJB's scheme. Now, my friend delivers 25,000 "you've got mail!" notifications. Then, he watches in horror as 9AM EDT rolls around and 5,000 of his customer's customers simultaneously try to fetch unique copies of the newsletter to read with their morning coffee. Repeat at 9AM CDT, MDT, and PDT. His choice is to get out of the newsletter delivery business, or spend $$$$ on vastly increasing his bandwidth.

Basically, it's fundamentally broken. SMTP is more or less optimized for throughput. DJB's plan is more or less pessimized for latency.

Re:Breaking SMTP not a solution

[fruey]

That's a fair point. I suggested the DJB link as a talking point, and I'm glad it brought out an intelligent response. I just said it was an "interesting take" and has some ideas which are worthy of discussion.

For major businesses RSS & such would be a good way to deliver "subscriber" content. Bloggers can do the same. They can also take advantage of proxy hierarchies. Bandwidth is getting cheaper anyway. Newsletters sent massively are exploiting the same weak link that SPAMmers exploit, so it's a tough call!

The line between SPAM and opt-in newsletters is something that makes the process difficult. Most Internet protocols are based on trust, store & forward, and good network configuration. Where you can catch SPAM is to axe everything in your policy to fight it around bad config.

That is what I retain as my key point: better DNS config, better SMTP/MTA config... if marketing people have to send better formatted newsletters and run well configured DNS servers... Hotmail / GMail / Yahoo can already fix some rules for that and begin to oblige the newsletter sender's SMTP/MTA/DNS chain to be better configured.

Re:Breaking SMTP not a solution

[wfberg]

Enter DJB's scheme. Now, my friend delivers 25,000 "you've got mail!" notifications. Then, he watches in horror as 9AM EDT rolls around and 5,000 of his customer's customers simultaneously try to fetch unique copies of the newsletter to read with their morning coffee. Repeat at 9AM CDT, MDT, and PDT. His choice is to get out of the newsletter delivery business, or spend $$$$ on vastly increasing his bandwidth.

Of course, you say this as if his newsletter doesn't contain any hyperlinks to http://yourfriendswebsite.com/ - most newsletters contain such references, and deal with the onslaught on the main website just fine.

That's not to say your example is wrong, but even so, the number of times people would be hit with problems from the reverse model would be limited. In fact, many subscribers will also neglect to fetch the actual newsletter's contents, saving your friend bandwidth as well.

The reverse model is already being used (even more inefficiently since many clients disregard ETAGs etc.) in RSS, and hasn't led to the end of the world.

In the end DJBs model doesn't solve much, because you'll still have to wade through an inbox filled with spammy mail headers, which is the biggest waste of time. In fact, spamfilters will be less effective because they won't have access to the message contents, and if they do, when they retrieve the contents, maybe they're getting a different version from when you retrieve the contents (or should the spamfilter fetch the message, and if not found to be spam, store it in your local mailbox? but then you're fetching the contents without human interaction again!)

Wraught with problems, it is. But the slashdot effect would be the least of its problems.

Re:Why not just fix Windows?

[RidiculousPie]

Given that all currently supported versions of Windows, from a technical perspective, have security capabilities that exceed those of most unixes, how do you propose they do more than they already have ?

It doesn't matter if these security measures are there if noone uses them. Windows still ships with new user accounts being administrator by default. The default group policy is very permissive, and acls do nothing versus the administrator user. If windows had decent sudo capabilities (yes I have used runas and credentials storing in shortcuts), which make it painful for the average user to run as anything other than Administrator.

Poor security by default is the real issue. Corporate entities can afford to create group policy and run users as non admins and have things like standard images if systems do get infected. A home user does not have the resources. Security needs to be on by default.

The original MS patent license & v=spf1 vs. PR

[Julian Mehnle]

Some of what you write is debatable, but some isn't.

The original patent license terms were not unusual or unreasonable. It was just that a number of persons decided to make an objection in this case to a practice that nobody had objected to for over a decade.

Saying that is ignoring the facts. Both the ASF and the Debian project classified the Microsoft's license for their patent as inherently incompatible with free software. And patents on e-mail standards, unlike patents on many other IT technologies, are a very particular problem because a very large (if not the larger) part of the e-mail server world runs on free software. Go read the ASF's and Debian's explanations, they certainly did do their homework.

Sender-ID is not incompatible with SPF as alleged. The only difference is at the recipient side and the recipient cannot be forced to interpret SPF or Sender-ID in any particular way.

(To be explicit about my motives: I am the one who appealed to the IESG/IAB on behalf of the SPF project about the reuse of "v=spf1" records for the PRA algorithm in the Sender ID specification.)

You correctly point out that a communication standard is little more than a silent agreement between senders and receivers that only works if the receiving party tries their best not to misinterpret what the sending party meant. But then you simply quit the subject, assuming that communication standards will work even with everyone interpreting stuff their way, because, after all, there is no protocol police, thank you. Sorry, but "compatible" means something else to me.

We had agreement in the WG to proceed on a common spec and nobody found any problems until the patent issue was raised.

Again you are missing the facts. Quoting from my appeal to the IESG:

It is also worth noting that at the time the MARID WG was closed [in September 2004], the then-current Sender ID specification draft-ietf-marid-protocol-03 did not include the re-use of "v=spf1" records for PRA checking. This was only introduced in [Microsoft's] individual submission draft-lyon-senderid-core-00 in October 2004. Also did Microsoft's record generation wizards generate only "v=spf2.0/pra" records until the end of October, when they began generating only "v=spf1" records.

Read the appeal. It connects a lot of dots that many do not like to remember.

Keeping separate issues separate

[Julian Mehnle]

The problem with doing this in the IETF is that the IETF does not have an institution wide patent policy. Every WG creates its own policy.

Given the mostly-free-software nature of the e-mail server world, I consider that a feature. But of course I'd prefer the IETF to adopt an institution wide free-software-compatible IPR policy.

The ASF position was that there was no alternative but for Microsoft to change.

I doubt that's true. I think the ASF (and Debian) position was that there was no alternative but to have a free-software-compatible patent license. Nobody really cared why this requirement postulated by some wasn't met (i.e. whether it was due to Microsoft's marketing strategy or whatever), just that it wasn't met. There would have been no point in ceding that position because, assuming MARID would have successfully produced a protocol, then it couldn't have been implemented by all the free MTAs and other free e-mail software out there. It seems that even Microsoft has now come to recognizing this fact.

The obsession with blocking the PRA check only began when a few zealots started to worry about the fact that people might dare to check their email with Microsoft's patented PRA check.

There are two issues that you need to keep separate. A: people objected to the inclusion of a technology in the upcoming e-mail authentication standard that was covered by a free-software-incompatible patent license. B: people objected to their "v=spf1" records being deliberately misinterpreted, causing technical problems for them. IMO, both objections were (and are) justified, but still these are separate issues.

Now that a technology covered by a free-software-incompatible patent license (=PRA) has not become part of the upcoming e-mail authentication standard (now there is not "one"), nobody really cares much anymore what happens to the PRA license. Only the technical issues of the "v=spf1"/PRA reuse (B) remain.

The PRA has always been offered on royalty free terms. Why is that particular filter the one you make such a song and dance over rather than the royalty bearing ones?

Because "royalty free" isn't the same as "free software compatible". The former as in "free beer", the latter as in "free speech", yadda yadda. As a developer of free e-mail software, I did not wish to be excluded from implementing the upcoming e-mail authentication standard, so I shared the objection.