Slashdot Mirror
Microsoft Releases 'Caller-ID For Email' Specs
gfilion writes "Microsoft has released a draft specification for Caller-ID for email, 'to address the widespread problem of domain spoofing' - the concept is similar to SPF, but is using XML. There's already an Caller-ID to SPF converter in the works. A few weeks ago, Microsoft discussed compatibility between the projects with Meng Weng Wong (SPF's project leader), but most SPF users are against using XML, so nothing has come of it thus far." We recently covered a brief article mentioning Microsoft's anti-spam work, though this is a clearer indication of their intentions. Update: 02/26 21:36 GMT by T : NewsForge is carrying a brief article with FSF counsel Eben Moglen's take on the draft; Moglen says it is "encumbered with unclear and unnecessary patent license claims."
While I acknowledge that XML is great for some things, why is it that it gets used for almost everything nowadays? Damn buzzword-dominated market...
Ok, I'll be quiet now :)
Why are SPF's developers against using XML? - because it's not more than a buzzword.
Why is Microsoft using XML? - because they're a business, they need buzzwords.
Microsoft is one big player in the email world through their Hotmail service. They probably serve more spam to more places than any other single mail service. As such it makes sense that they would want to be at the forefront of spam-elimination technologies. They ought to be applauded for their initiative here, as well as their cooperation with SPF and Sendmail.
However, it disconcerts me that they are also applying for a patent in this area instead of engaging the community through a consortium-like committee that could share the technology across the board unencumbered by licensing fees. The specter of Hotmail becoming a proprietary mail system requiring foreign mail servers to run Microsoft-licensed "Caller-ID" to interact with Hotmail is a very legitimate concern.
I have been pwned because my
Caller-ID for email will help prevent spoofing, but will only increase spammers use of zombies. I wonder if increased exploitation of Microsoft OS weaknesses (to create spammer platforms) will have a long-term detrimental effect on Windows or whether it will hasten adoption of Trusted Computing? I wonder if Microsoft wants ISPs to become so sick of zombie boxen that the ISPs will prohibit all but a few chosen OS options (read the lastest version of Windows) for connection to their networks.
For a very well-entrenched provider, making everyone sick of you old product is a good way to force them to buy your new product.
Two wrongs don't make a right, but three lefts do.
if it will mean I have to pay fees to Microsoft to get my domain signed, I'd rather continue filtering out spoofed-bounces, thank you.
Interesting how instead of supporting a perfectly sound project that has been going for a year, everybody seems to have to come up with their own little *patented* scheme.
Then why not just call you in the first place, and do away with the email?
Stop corporate
I do believe this is one area we have to really keep an on eye on M$ in. Do they really want to stop the spam or is it just PR. They have the browser that doesn't block pop ups and on a default install of windows Ad-Aware will find things it considers an issue right after the default install.
This may just be a PR issue to show people they are pushing for it. When they implement something like this will they put their own hooks in it to allow what they want???
M$ really needs to be kept an eye on if they do this.
Evolution or ID?
They would have allowed a user to disable a the javascript popup function in the browser. Instead we have to rely on bandaids like googles toolbar to block popups from websites.
(From Microsoft's license.)
So by building support for "Caller ID for Email" into your software, you suddenly give Microsoft an unlimited license to use and sell it. And, in fact, not only Microsoft, but everyone else who writes software that supports "Caller ID for Email."
There is a word for this: Insane.
No thanks. I'll stick with SPF--especially since the two are essentially identical, just a slightly different parsing format.
It's hard for thee to kick against the pricks.
They already have systems that do this [challenge-response], you know. This doesn't require any changes to standards; but it does require that the sending user be clueful - and given how quickly Netsky.C spread, I think that's a hopeless cause.
In the US at least, caller-ID is not a challenge response system, it simply displays the originating phone number - and ONLY if you haven't requested that your number be hidden, and only if you live in an area that supports it.
So, what lessons can we carry from this fact to MS's suggestion of "caller ID" for email? 1. We'll still get emails that are unauthenticated, because it will take a long time for folks to upgrade MTAs to manage this - after all, there are still open relays - and 2. someone will figure out some way to sell a solution to get past the authentication system so blocked spam senders can still get through (can you say "sales@viagra.hotmail.com"???).
That still has the same problem as every other C/R system. In order to GET that phone number, presumably every email is responded to by a notice to call that phone number. It still bombards the poor shmuck whose email was forged with C/R requests.
... because the performance is crap. This is true on my pc (with any parser you care to name - i've tried it) so what it'd be like on a mail server handling x thousand messages a minute I have no idea.
XML is great, but only when the underlying data is sufficiently variable within a pre-defined schema and where throughput is not an issue. It's not necessary here.
sean.
On a first reading, I thought the ideas seemed quite sensible. One problem they did address in an interesting way was that of people with several email identities. One of their suggestions is that whoever is hosting the incoming email provides outgoing smtp services too, which would be a change from the (outdated?) idea that one should always use the "nearest" smtp server for all email. Though ISPs who currently block outgoing port 25 (such as my University!) would have to think again.
N.
XML is awesome when you are looking for interoperability between different applications/systems. I would think that when the Internet community agrees upon whatever protocol, it should be a common standard and will not need the benefits of XML. Indeed, XML would actually be a bad choice, as the extra market will just use more bandwidth. Sounds like MS should just bow its head, say thanks to SPF, and adopt it. If they want XML on their side, then let them right an internal API/converter so those developing with Exchange or Outlook will have access to an XML version. By leave it off the pipe! And I say this as a guy who works with XML everyday and enjoys the benefits it provides my company.
Great ideas often receive violent opposition from mediocre minds. - Albert Einstein
because the Sendmail sender verification proposal (mentioned here) relies only on already existing tech (Domain Keys, mx records, and smtp auth) thaty is already incorporated into the vast majority of MTAs, it does not really make much sense (from a users, or a non-microsoft, point of veiw) to create a seprate and more complicated solution (even if the license is rather innocuous).
I cannot help but think that continuing to allow senders that do not have a mx record for the sending machine to bypass smtp-auth for sending messages will fail to curb the spam problem, as it fails to tie the sent mails to an actual domain, and it allows (encourages) ISPs to restrict mailing through their email services only. With smtp-auth, it is still possible to send using an smtp server connected anywhere on the net, which allows accountability, but also makes it more possible to identify those providers who are allowing their users to send spam.
Read, L
No, it is not insane. It is called cross-licensing. They are saying if you want to use this technology, then you agree that you are not going to come back and sue Microsoft (or any other licensee too!) for patent violations relating to this implementation. This is a good thing!! They are protecting themselves.
So by building support for "Caller ID for Email" into your software, you suddenly give Microsoft an unlimited license to use and sell it. And, in fact, not only Microsoft, but everyone else who writes software that supports "Caller ID for Email."
Absolutely not. There is something called copyright law. Microsoft or any other company cannot just go and resell your software on their own terms. The license just means you cannot sue them for patent violations when they choose to build software that implements technology similar to yours in this area (provided you had obtained additional patents relating to this 'Caller-ID').
XML has become what ASCII files were used for back in the 80s and 90s. From that perspective, we've come a long way. At least now we can make a rough guess at what the data inside the XML file represents (unless of course it was designed by a moron).
And as for speed, it's really a non-issue for email. Who gives a rats-ass if it takes your email server an extra 100ms to process an email. More then likely you have a email server that serves 1000 users or less. The amount of time you'll save processing XML headers vs. the shit your server process now will be infinitesimal compared to the amount of time it currently spends processing spam.
Say what you will about XML, but at least it's better then the custom format binary crap files that proliferate tons of legacy systems.
just to escape-out special characters requires instantiating a new "entity" element in the middle of the text string element.
Maybe that's the "right" way to do it, but I highly doubt that you cannot set the value of a text node to a string that contains an entity (i.e., "this is an ampersand: &"). That would be the more direct approach.
And I still haven't figured out how to make the thing give me a CRLF at the end of each element. No, XML doesn't require the whitespace, but it would have sure made it easier for my clients to read the file!
First, you could have them read the file with Wordpad or just about any text editor other than notepad. And BTW, why are you complaining about MSXML not generating CRLF? You DO realize CRLF is a Microsoft-ism and not "standard", right? So you're complaining about MSXML generating text files in a manner more in line with the way every other system does it. Baffling...
But the worst part is that I *succeeded* in using MSXML. Now, if I wanted to go back to just writing a text file (which I do!), I can't -- my code is tangled up in the objects to the point that it would take a complete rewrite.
I've got news for you -- every decent XML parser library requires you to manipulate the XML tree in an object-oriented manner! It's called the Document Object Model for a reason -- you're not manipulating raw text! You can go ahead and do that if you like, and we'll see how much "easier" that is for any project requiring more than the most basic use of XML.
Mods, get a clue. The way the MSXML library handles XML is not unique in some "Microsoft always makes crap" kind of way. Every decent XML library handles XML the same way.
I say ignore them.
Microsoft has never been interested in helping the community but rather wants only to further its own dominance of the market. When did they start being philanthropic?
What's to say in a few years time when everyone is relying on this that they don't pull some stunt and start charging people? Do you know enough about the law to say they couldn't?
Anyway their record on enhancing email is not good. I knew the first time I saw the ability to embed HTML and * SCRIPTS * into email that the virus writers would have a field day. I mean, what complete arseholes to allow code to be executed when someone just *reads* and email. It beggars belief!
If they are serious they could assign their patents over to the FSF and then we'll consider it. I bet they won't.
Not only would it be difficult, it would be POINTLESS because spam prevention only works if EVERYBODY DOES IT.
It's 10 PM. Do you know if you're un-American?
So don't comply and risk getting your mail dropped. You can have your privacy, but you can't FORCE others to read mail from suspicious and unknown sources. Your call. There are plenty of non-email alternatives to be anonymous. Post in a random newsgroup from a web cafe. Or use a secure IM protocol, or secure IRC.
It's 10 PM. Do you know if you're un-American?
Have you ever tried to emit those types of compound documents without using any Microsoft controls? I.e., on another platform? A non-trivial task.
Isn't this likely Microsofts attempt to get everyone using passport of something similar?
Once they authenticate everyone using their anti-spam system, they'll be able to authenticate for financial transactions, etc...
I use a locally running postfix SMTP server on my laptop to send pretty much all of my email. Microsoft's proposal doesn't address this: of course, my laptop gets various IPs. I cannot use the SMTP server provided by my organization, as they firewalled it... With the MS proposal, I will have to go for VPN or talk to my sysadmins about smtp-auth -- and lose my independence...
Like caller ID worked for the phone system. About 90 percent of my calls were either "Unknown" or "Private Line", and some action was still requried on my part to respond to the ringing phone.
I don't have facts readily available to back this up but I'll assume somebody made money off caller-ID, as will Microsoft will attempt to do with their new "standards".
-Phil
Shoot questions, first ask later...
It doesn't even take a free account.
The major problem with ALL these systems is critical mass.
Corporations are not going to be blocking mail based on a lack of SPF, Caller-ID, or anything. Too many companies are going to be slow to implement, or apathetic about it. No larger business is going to block mail and potentially lose contact with potential customers, or existing clients.
90% of the current crop of spam would stop if all ISP's would block outbound port 25 from dynamic IP clients by default (unblock if the client agrees to keep their system patched and secure and face penalties if found spamming.)
For the most part, open relays have been closed due to RBL like activity, as enough sites use RBL's to make life very difficult for admins that leave their systems open. So spammers have moved to dynamic's, which there is a virtually unlimited supply due to the piss poor security of Windows and clueless users. RBL's are helping with that too, but it's hard to keep up. Again, many corporations won't use RBL's due to problems noted above.
While I have not read the detail on MS's solution, SPF has the "roving user", "mail forwading" problem that there is no solution for that has been discussed to death. Anyone know if MS's solution has the same problem?
Yes - and then we'd know exactly who's machine has been trojaned with much less effort. The ISP can then disconnect them until they have patched their OS/removed the trojan.
Oolite: Elite-like game. For Mac, Linux and Windows
This isn't true. The SAX API is event-oriented, and though it may be a little bit more difficult to wield than DOM it has the advantage of giving you complete control over memory allocation. That is, you can allocate as little as you need, and only when you need it, whereas DOM libraries allocate all that is required to completely represent the entire document in memory up-front.
Every decent XML library handles XML the same way.
Also not true; the same example suffices.
The "cue the foo posts in 3, 2, 1..." posts will commence with no subsequent foo posts in 3, 2, 1...
I can assure you that when Satan sends messages, he sends them ASN.1-encoded. Especially the BER encoding, which doesn't even have one canonical means of encoding.
It's so much fun that it causes buffer overflows all over the place (Microsoft OSes, OpenSSL...)
I think "do we want XML" vs. "do we want a series of header fields" is asking the wrong question. It's the schema that's wrapped up in the XML or fields that's important.
XML is great for expressing tree-like data structures, where as the "field-name: field-body" approach is probably better for expressing linear data. If you look at a schema it is usually obvious if XML is being used just for the sake of it, and parsing SPF as it stands is trivial.
Companies with an "embrace, extend and extinguish" mentality towards standards can leverage XML by using it without any formal machine-processable schema (DTD, XSD or RNG), whilst all the while insisting it is "standard" because it uses XML. Look no further than WordML for an example of Microsoft doing this.
ISPs can already see exactly whose machine has been trojaned from the time and IP. Checking their logs to find that info is trivial - the tricky part is getting the user to patch/clean their computer. Knowing the email address of the person whose machine is trojaned doesn't really help the recipient.
Having correct sender addresses would be nice, and would force spammers and virus writers to adapt somewhat. The question is whether the effort of implementing it is worth it for the gains available.
Previous comments have been for or against XML being used to deliver this information. I don't have a strong opinion either way on that; it seems reasonable enough. What does seem silly is that this information is being stuffed into a TXT record, and limited to 2k. A goal of using XML should be to easily add information and to make the information hierarchical. But that goal will likely never be realized in a 2k string. The XML tags will eat away at the number of allowed characters pretty quickly. And the zone file examples in the document are pretty ugly.
SPF is better in that it keeps the information simpler. If XML is should be used, perhaps the TXT record should simply include an HTTP URL to the XML file. Alternatively, a simple URL standard could be used, such that one could reliably get Caller ID information regarding mydomain.com from http://mydomain.com/callerid or http://callerid.mydomain.com/.
Personally, I think in this case MS is actually, honestly trying to do the Right Thing. And it's easy to see why. What is one of the three biggest reasons the average user would even consider moving away from MS and Windows? Exactly. (The other two are spyware and virii. Popups don't get a seperate category, as they're just another form of spam.)
Microsoft realizes this, and are trying to fix it, in their own very good interest. See also: SP2 contains antivirus, an upgraded firewall, a popup blocker integrated into IE, buffer overflow protection for processors that support it (Athlon 64 and Opteron currently), and I assume there's more.
So you can safely expect for it to be That Much Harder convincing people to move to *nix, once SP2 is released. Do it while you still can. (Note again that I am not saying *nix will lose any advantage it has/had over Windows. Merely that in the eyes of the average user, it will.)
Work is punishment for failing to procrastinate effectively.
By including the above notice in a Licensed Implementation, you will be deemed to have accepted the terms and conditions of this license. You are not licensed to distribute a Licensed Implementation under license terms and conditions that prohibit the terms and conditions of this license.
I guess this means no GPL apps, but I will now head to Groklaw and refresh the page until some legal info comes up
For example, it allows me to tell SpamAssassin that IF a domain has SPF-records, and the email doesn't come from one of the ips that send mail for that domain, then in the spam-bucket it goes.
Thus, for example, all the spam that claims to be from hotmail is gone.
Secondly, I can, by publishing spf-records on my own domain eliminate the problem of spam bouncing back to me because it *claims* to be sent from me.
Third, once a sufficient part of the people I communicate with email from domains that *have* spf-records, I'm free to, for example, implement a challenge-response system for email coming from other domains. Yes, this will mean people using those domains gets some challenges based on spam that only *claimed* to be from their domain, but actually isn't. That migth serve as a good incentive to get them to also publish spf-records. It's not as if it's a huge deal to stick 2-3 extra records in your dns-info.
I wish you would learn something about existing mail standards-- like their colossal drawbacks. SMTP is entirely "a simple text format", and that's one of its biggest problems. We have all kinds of lame hacks for mailing binaries around and handling attachments. Nearly everyone who writes a mail client writes a mail parser and a composer. Not just a formatter, or presentation-level stuff-- basic goddamn parsing and composition.
You don't seriously believe that any format that is newline-dot-newline-delimited is a good one, do you? SMTP is a relic, all the way down to the message format. I hope to god someone eventually succeeds in dislodging it.
I wish this wasn't true, but if Microsoft implements some sort of spam-blocking in Exchange, that's all the critical mass you'll need. Especially if they turn it on by default.
Many here have pointed out flaws, problems or complications with the proposal, I have a fundamental problem with it. .doc file. Who releases a proposed "standard" in a proprietary format? Shouldn't this have been plain text, RTF, HTML or even PDF so that everyone could read it properly?
.doc formatting in TextEdit yet.
They released it as a
Using a Mac without Office installed I get lots of document formatting commands interspersed with the text. Apparently Apple hasn't figured out all of the
Article X: The powers not delegated... by the Constitution...are reserved...to the people
Colossal drawbacks to text? LOL! It is a feature. You could say the same for most internet services. There are no standard client API's for FTP or Telnet or most other services either. Has that stopped their widespread adoption? Has it made them any less useful? No.
I am not concerned at all of people like you who make the internet groan under the weight of 20MB excel files wrapped in proprietary XML formats. MIME has done enough damage. Maybe the Standard should be a Microsoft (C, TM) paperclip icon that does a dance while he speaks your message in one of a hundred supported languages.
an ill wind that blows no good
This is because you don't understand SMTP.
The Sender entry in the headers is often added by MTAs as the value in the SMTP envelope's MAIL field. This is the same value that SPF validates against.
Just because you don't understand SMTP and SPF is written in RFC language does not mean that Caller-ID is better. The XML in DNS TXT records is a big deal. The fact that with Caller-ID you have to validate after DATA is a big deal. But you won't understand these issues if you don't understand SMTP.
Matt. Want XML + Apache + Stylesheets? Get AxKit.
You said it! I'm sure we'll all regret using a standard format for hierarchically arranged tuples of name-value pairs. I only have to use this type of data in maybe 99% of my projects.
Nothing wrong with agreeing. Agreeing on a standard that's cruddy will bite you in the ass. There are many, many standards, and most of them are cruddy.
And "name-value pairs"? How do attributes figure into that? Well.. Cruddily, that's how!
Perhaps you're thinking of RDF (which has issues of it's own.. A lot..).
And the output files sure are difficult to understand if you've never seen any markup language before and don't have a file viewer that understands ASCII text.
XML allows for a lot more than ASCII.. Which is the reason a fully compliant XML parser is enormously bloated.
Instead why doesn't everyone just make up their own format that is uniquely tailored for the individual application? You can leave off the attribute names since the recipient of the data should just know what they are anyway. And you can use a binary encoding to really add efficiency to the process. And developers love the challenge of trying to figure out new data formats on top of interpreting the data itself.
Slippery slope? Or straw man? The latter. I never said no standard should be agreed upon. I would have preferred if it had not been something as complex and cruddy as XML. I even specifically gave S-expressions as an example that would be much simpler; you might note how that's not a binary format.
One day, ASN.1 was what XML is now (well, it still holds telecommunications and cryptography in its stranglehold). Do you propose we use ASN.1 because it's so well accepted and standardized and there are so many tools? Or do you recoil in shock at how bloated the featureset is, how convoluted the encoding, how shockingly incomprehensible the parsing process? XML is simpler than ASN.1, and XML is better than ASN.1 (except that ASN.1 has a cute way of compiling parsers from its syntax/schema language, which is a nice feature); but that does not mean XML is the best general purpose meta-syntactic language imaginable. It's not.
SCO employee? Check out the bounty
"You don't seriously believe that any format that is newline-dot-newline-delimited is a good one, do you?"
Ask that again when you've got your x million messages-per-hour email gateway parsing an XML file each time...
Email is so simple you could probably parse it with a circuit board and a few NAND gates, and that's very good indeed when you want people to start using it.