Slashdot Mirror


IETF Decides On SPF / Sender-ID issue

Zocalo writes "The MARID working group at the IETF responsible for deciding on which extensions to SMTP will be used to try and prevent spoofing of the sender has made their decision. At issue was whether Microsoft's patent encumbered Sender-ID would be eligable for inclusion in an Internet standard. An initial analysis of the text of their decision, available here with a brief analysis, would suggest not. Unless Microsoft is going to make any dramatic concessions out of desperation, that pretty much clears the way for Meng Wong's Classic SPF to become the standard and hopefully make Joe-Jobs at thing of the past."

31 of 269 comments (clear)

  1. I saw spammers are ready for this by Anonymous Coward · · Score: 4, Interesting

    Why is that the spammers actually supporting this ? Does this give them more targeted email addresses ?

    1. Re:I saw spammers are ready for this by DrZaius · · Score: 4, Interesting

      SPF isn't meant to stop spam. It is meant to stop spam that spoofs the from address.

      This means all the spam that comes from AOL and Hotmail accounts that don't actually leave from there servers would be bounced at your mail servers. At this point in time, if everyone used SPF, my guess is that at least 50% of spam would be blocked.

      Of course, spammers are going to register domains to use for spamming and set SPF records so that their mail appears legit to the SPF filters.

      You're probably thinking, "What's the point?" Well, it's easier to understand if you have ever hosted a domain that has been either blacklisted or had an increase in bandwidth charges because of millions of bouncebacks due to spammers using a FROM address in your domain.

      --
      -- DrZaius - Minister of Sciences and Protector of the Faith
    2. Re:I saw spammers are ready for this by Zocalo · · Score: 2, Interesting
      Spammers are supporting this ("using" would be more accurate) because they hope that systems such as SpamAssassin will assume that this indicates the email is more likely to be legit. Like many people, they've missed the point of SPF et al; SMTP is flawed in many ways and there is no single magic bullet, this bullet is designed to prevent address spoofing, not combat spam. However, if it encourages spammers to spend a few extra dollars on throwaway domains, that's fine by me. ;)

      There are several possible outcomes from an SPF query, and given the adoption by spammers I've taken to doing a reject on a hard failure and not scoring the rest at all. Once I've got a few hundred emails in each contingency I'll calculate the ratios and assign SpamAssassin some apropriate scores. At the moment, I see four possibilities for this;

      A hard fail

      A hard success ("-all" present)

      A soft success ("-all" absent, or some other open ended SPF record)

      SPF is absent

      Depending on how spammers continue to implement SPF, I think it's very likely that only the first one is going to be a serious indicator, but that's fine because that's enough to kill Joe-Jobs.

      --
      UNIX? They're not even circumcised! Savages!
    3. Re:I saw spammers are ready for this by theNote · · Score: 3, Interesting

      The fact that SPF can verify the server a message is sent from doesn't go far enough, and will only increase the demand for zombied machines.

      Let me explain:
      Most major ISPs here in the US have already shut down outgoing 25. This means even if you have a hosted domain that allows you to use the host's smtp server, you can't (without jumping through hoops). You have to send through your ISPs smtp server.

      Most large ISPs run only a few smtp domains, for example east.smtp.ISP.com, west.smtp.ISP.com.

      With that being said, even if SPF was 100% rolled out, how many domains would have east.smtp.ISP.com SPF records?
      I'm guessing thousands.

      Anyone with access to these few servers (100s of thousands I'm guessing) would be "authorzied" to send mail for any one of these domains.

      The problem will only increase as the number of major home providers decreases.
      SPF relies on a low smtp/domain ratio, which just can't be guaranteed.

  2. Standards == Monopoly?? by bunburyist · · Score: 5, Interesting

    Question: Is the IETF allowed to adopt patent-encumbered standards? I mean, wouldn't that grant some sort of monopoly license in effect for MS, seeing as if you want to adopt a standard, you need to pay somebody? Shouldn't standards be free, and people can make money off the implementation of said standards? I don't know how these things work, nor am I a lawyer of any capacity.

  3. Worried... by renelicious · · Score: 4, Interesting

    This worries me more than it makes me excited. I have several email addresses that I send mail from home through ISP. I don't believe they are going to put those domains in thier DNS list.

    In the case I guess the only option will to be use webmail for any addresses not provided by my ISP. That's a pain...

    --
    "Luke, I am your node.parent();"
  4. Not over until FAT lady sings by tobybuk · · Score: 3, Interesting

    MS will not take this without fighting back. I suspect they will hint that they may have issues with SPF as well, maybe in a years time or so.

    Fucking S/W patents. If these had been available 20 years ago the NET would never have been born.

    These people are just selfish. They build their bisinesses on the NET backbone, given to them for free and then do everything possible to destroy the vehicle that built it.

    Human nature?

  5. Reason #1 That I don't like SPF by Anonymous Coward · · Score: 4, Interesting

    SPF Breaks Forwarding.

    Yes I know about SRS. (sender rewriting scheme)

    SRS is a LAME workaround for the fact that SPF breaks forwarding.

    1. Re:Reason #1 That I don't like SPF by DavidTC · · Score: 2, Interesting
      Specifically, SPF breaks forwarding when someone gets one mail account and forwards everything in it to another account. That account will do SPF lookups, see the mail is forged, and reject it.

      There are three solutions here:

      1. Stop forging MAIL FROM when you forward mail, you moron. Yes, it's not you, but your MTA, but that's wrong behavior. If that mail were to bounce, no one at the other end would have any idea what address it bounced from! They send to one address, got a bounce back from another! This has lead to silliness like VERP where the email address you're sending to is encoded in the MAIL FROM.

      Yes, right now, if you didn't do that, the bounce would go back to the forwarding account, and then reforward it, only to bounce again.

      But the point is: SMTP forwarding is already broken. Complaining that this makes is more broken is valid, but not really relevant, considing:

      2. It would be trivial to make a option to disregard SPF lookups for certain IP addresses, and give people an interface to set those. For example, if I use hotmail to receive mail forwarded from example.com, I would go into hotmail and type 'example.com' in the 'accounts I have forwarded to here'. hotmail can then turn off the SPF check for any email coming from any IP address example.com sends mail from. (Which, to complete the circle, they can calculate using SPF!)

      Or, hell, for bonus points, it can try to parse the header and apply SPF rules to the IP address before the mail hit example.com's mail server. Yes, yes, headers are not reliable, but hopefully the accounts you have forwarded are not trying to fake you out. If SPF info is just added via headers, it's trivail to make a rule to pull out email forwarded from your other account before you do any filtering on it.

      So there are at least two solutions, one implimentable on your forwarded account, and one implimentable on your receiving account. But what if you can't change anything?

      3. Well, on every webmail I've ever seen, including free ones, you can set up 'POP3 accounts' to pull in mail from. Maybe you should realize SMTP mail forwarding is just broken and just use those to pull your mail in.

      --
      If corporations are people, aren't stockholders guilty of slavery?
  6. make sendmail look bad by hey · · Score: 4, Interesting

    They caved and send they'd implement Sender-ID.
    It makes Apache and FSFlook good as they
    proved resistance isn't futile.

    http://it.slashdot.org/article.pl?sid=04/02/24/1 44 2237&tid=111&tid=109

    1. Re:make sendmail look bad by IGnatius+T+Foobar · · Score: 2, Interesting

      Actually, the Sendmail program itself makes Sendmail look bad. Something as crufty as Sendmail shouldn't exist in 2004. It's no surprise that even seasoned unixheads are switching to Postfix.

      This isn't a troll, or at least it isn't meant to be read as one. My point is that Sendmail is a perfect example of exactly what's wrong with unix. Nobody wants to be editing cryptic configuration files to accomplish simple things. Remember the mantra: simple things should be simple, and complex things should be possible.

      --
      Tired of FB/Google censorship? Visit UNCENSORED!
  7. Will ITEF make a difference? by milgr · · Score: 5, Interesting

    When Microsoft ships their method on all their mail servers and mail clients, will it matter that it is not a sanctioned standard?

    --
    Where law ends, tyranny begins -- William Pitt
    1. Re:Will ITEF make a difference? by Arkham · · Score: 3, Interesting

      Yes it will, because unlike the desktop OS, Microsoft does NOT have a monopoly on mail servers. Most ISPs run one of the UNIX mail servers (sendmail, postfix, etc) rather than Microsoft's POS.

      The only environment where MS's email has a stronghold is in corporate email. I don't think that's sufficient to force a standard. Even in that market, MS only has about 50% of the market.

      --
      - Vincit qui patitur.
    2. Re:Will ITEF make a difference? by igaborf · · Score: 3, Interesting
      When a small company upgrades to the MS product containing Sender-ID and their server suddenly starts rejecting mail from clients/customers whose mail systems implement SPF instead, will the small company:
      1. tell their client/customer to switch to an MS product if they want to send e-mail to the company; or

      2. turn off Sender-ID?
      I know what I would do.

  8. Time to bug DNS hosters by hey · · Score: 4, Interesting

    I already sent a mail to the company that hosts the DNS A records for my domains (also my DNS registrar) asking when I'll be able to add an SPF record.

  9. Good. Now let's improve SPF. by jefp · · Score: 5, Interesting

    The one feature of Sender-ID that I'd like to see in SPF is checking the header-sender as well as the SMTP-sender. Of course this is expensive, requiring reception of the message body, but it's worth it.

    It occurred to me recently that I could write a separate milter to implement just this one check. It would compare the SMTP-sender against the header-sender, and if they don't match then it would add a header to the message saying "possibly forged". A later step in the delivery process, such as bogofilter, would see this header and weigh it appropriately.

    I'm interested in comments on this idea.

  10. Joe-jobs or just forged headers still are a pain by Artifex · · Score: 4, Interesting

    I'm now seeing 30-40 bounceback emails a day originally sent from people spoofing my vanity domain - I haven't given any accounts out, of course. Makes me wonder how many of their emails got through to their victims, as these are just some of the failures. The most annoying part for me is that I see them come in batches - with very different originating IPs and to different mail servers for each message - so I don't know if it's a pack of zombies and my domain is one of the ones in rotation, picked out of someone's address book, or if someone is doing a deliberate joe-job on me.

    This ought to be considered actionable as a DOS attack - if companies start filtering out my domain name, I can't apply for jobs with them, for example. And if my upstream ever gets tired of explaining to idiots to read their headers instead of thinking it's me, then I'll have to hunt for another provider. Even without those reasons, it still takes me time every day to clean out my admin box so I can get my real mail. In fact, because I'm the only person at my vanity domain, and it's part of my online identity, it also ought to be considered slander for someone to pretend to be from my domain, because they're effectively claiming I'm sending these ads, etc.

    I hope SPF becomes generally accepted, and soon. I'm afraid it won't, though, because there are millions of people running old copies of MS Exchange, etc., and they probably won't want to pay to upgrade or take the performance hit to authenticate messages this way. Still, if I go ahead and stick the DNS entries in, it might at least prevent some of the damage.

    --
    Get off my launchpad!
  11. No patents just because not publicly available? by KWTm · · Score: 2, Interesting

    And in follow up to the parent's post, I want to ask:

    [from TFA] 3) On the issue of ignoring patent claims ... there is at least rough consensus that the participants of the working group cannot accurately describe the specific claims of the patent application. This stems from the fact that the patent application is not publicly available. [emphasis mine] Given this, it is the opinion of the co-chairs that MARID should not undertake work on alternate algorithms reasonably thought to be covered by the patent application. We do feel that future changes regarding the patent claim or its associated license could significantly change the consensus of the working group, and at such a time it would be appropriate to consider new work of this type.

    So... what, if they did publish the patent, they could decide to include it in a standard? Seeing the patent doesn't prevent the patent owner from taking advantage of it, does it? (In fact, seeing the patent makes it easier to be sued for infringement.) Or am I missing something here?

    --
    404555974007725459910684486621289147856453481154 in hex is "You sank my Battleship?"
    [GPG key in journal]
  12. Re:Certificate based sender authentication by nolife · · Score: 3, Interesting

    Who would control or gives out the signatures? What or who would be available to get a these signatures and how would a spammer not get one. There are certificate based methods already in place that many people already use, problem is it costs money and configuration that people are not going to spend. Free/open versions of PGP already exist and are in use but again, no restrictions to prevent the bad people from using it. Your idea would help verify who sent a message but not limit who you could get them from.

    --
    Bad boys rape our young girls but Violet gives willingly.
  13. Re:Joe-jobs or just forged headers still are a pai by Artifex · · Score: 2, Interesting
    We had an old Exchange 5.5 server running on under-powered hardware. We installed XWall on another machine and relayed incoming messages through it. This took the filtering and virus scanning load off the Exchange server and made it usable again. I don't know if XWall currently supports SPF (I'm also too lazy to look), but it can certainly bring additional filtering functionality to an old Exchange installation. It's pretty cheap too.


    That's great to hear that you did even that much. I have no doubts that companies could implement SPF even with MS Exchange, but it will take work. And if you think back to how long it took your company to decide to set up that XWall, it will probably seem obvious that many companies, especially small ones, won't bother.

    I'm really worried that MS will just include Sender-ID as part of their next server releases, or worse yet offer service packs for prior versions, and when/if companies upgrade or patch, they'll think Sender-ID is what they want because they got it "free." Since you said you're too lazy to see if XWall supports SPF, if a service pack for your exchange version came out, and enabled Sender-ID, would you just run with it? That's what I'm talking about.

    These arguments others have about most email servers not being MS-related are feeble, in my opinion, because they do seem to be the largest player in the corporate email market, and that's where the important email goes. Spoofers (at least the ones I've been seeing bounced mails from) connect directly to destination mail servers, they don't use their ISPs as MTAs. So what ISPs might run is not relevant to them.
    --
    Get off my launchpad!
  14. Re:Joe-jobs or just forged headers still are a pai by ScrewMaster · · Score: 2, Interesting

    Yes, it's an issue. When I first set up my mail server about six years ago, the default configuration was to bounce unknown local addresses back to the sender. As I was only getting a few pieces of unwanted email a week that wasn't a big deal.

    Then three weeks ago my ISP shuts off my SMTP access, no warning, no explanation. Now, as it happens my server runs all incoming mail through three or four RBLs, also uses SpamAssassin, has a Bayesian filter system and I use Thunderbird. So after all that filtering I really didn't see much spam in my Inbox ... but unfortunately about ten thousand spams a week were being bounced back to their senders by the server and my ISP figured that I was spamming!. Oops.

    --
    The higher the technology, the sharper that two-edged sword.
  15. Re:SPF is NOT about... by pyros · · Score: 2, Interesting

    I just realised something. I have a domain hosted at a company that doesn't offer secure smtp (over either tls or smtp+ssl), so I just use the smtp server of whatever network I'm currently on, but always using my address within my domain. This system means I'll either have to start using the insecure smtp server of my hosting company, or add the smtp servers of all the networks I regularly used as authorized to send from my domain, right? That sucks.

  16. Re:That's certainly not how I read it by Zocalo · · Score: 3, Interesting
    I largely agree, which is why I made the comments about Microsoft making consessions and only going so far as to say that it "pretty much clears the way" in the submission. The problem MARID has is not that there is a possible patent issue, or even that it's Microsoft, but that Microsoft is not disclosing the details. There is also a seperate problem that the open source proponents in MARID have with Microsoft's license in that appears to prevent redistribution, hence the actions by the ASF, Debian and so on over the last few weeks.

    It's not too late for Microsoft to change their mind, relax the license terms and waive any patent issues to get Sender-ID accepted. Their problem is that they need to do so quickly or they will be trying to push a proprietary standard onto a group that have already stated they don't want to know and will not implement it. Also, standard or not, adoption of Classic-SPF is proceeding apace and is already functional in most FOSS MTAs and anti-spam systems - for a lot of people the herd mentality is all that applies in selecting a solution.

    --
    UNIX? They're not even circumcised! Savages!
  17. Forwarding is wrong by Skapare · · Score: 2, Interesting

    Forwarding is wrong. Always has been. Re-mailing is what should be done in the majority of cases. Mailing lists won't have any problem because the mailing list itself can be the return address, thus not invoking an SPF lookup on the poster herself. Private forwarding is the big issue (e.g. like bigfoot.com) and in these cases SRS and backtracking can be used.

    Now do you a constructive suggestion of an alternative to SPF that both supports the kind of forwarding you want to do, and still informs those who participate that no mail server but mine (or which I say) are valid for sending mail addressed as from my domains?

    --
    now we need to go OSS in diesel cars
  18. Re:Joe-jobs or just forged headers still are a pai by Todd+Knarr · · Score: 2, Interesting

    That's probably because you shouldn't be bouncing mail like that. If the local address is unknown or the message triggers a spam filter, the mailserver should respond with a 500-series SMTP error (probably 550). Note: this only works on machines receiving SMTP connections directly from the world. If you know there's another machine between you and the world (eg. external server receives mail, queues it and then passes it along to a second machine that runs the spam filters and delivers the mail to mailboxes), SMTP errors cause other problems.

  19. Re:It's been said before, but it's worth repeating by BasilBrush · · Score: 2, Interesting

    Go find and read the PRA algorithm. It's of the level of simplicity that it's the sort of thing software engineers do every day. If coming up with the algorithm took the engineer involved more than an afternoon, then I'd want to know why. Why should arbitrary, run of the mill and obvious work-a-day work of engineers be randomly picked out and selected as something not to be repeated by other software engineers? The protection of an invention that has taken man months or man years to develop is understandable. But PRA, and very many of these pathetic software patents are not worthy of such protection.

  20. Diplomacy: Patents have to be clear and public by Spoing · · Score: 4, Interesting
    This is a diplomactic negotiation.

    It's not over for Microsoft's efforts...though it's very close to being over. The important section that points this out -- with highlighted text -- is below;

    1. 3) On the issue of ignoring patent claims, the working group has at least rough consensus that the patent claims should not be ignored. Additionally, there is at least rough consensus that the participants of the working group cannot accurately describe the specific claims of the patent application. This stems from the fact that the patent application is not publicly available. Given this, it is the opinion of the co-chairs that MARID should not undertake work on alternate algorithms reasonably thought to be covered by the patent application. We do feel that future changes regarding the patent claim or its associated license could significantly change the consensus of the working group, and at such a time it would be appropriate to consider new work of this type.

    They aren't saying that the Microsoft patent (or any patent) is bad...they are saying that it can't be publically reviewed or is not clear enough to make a decision.

    This does give Microsoft some wiggle room if they want to 'clarify' what they mean...and in the course of that, possibly elminate the problems they originally introduced.

    Microsoft has a choice to either correct the mistakes (by 'clarifying' them) or what they contributed with patent encumberences will not be accepted.

    --
    A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
  21. Sender ID helps.... How? by gral · · Score: 2, Interesting

    I understand that SenderID and other proposals basically want to make sure that the user is a real user, but:

    Doesn't this put a strain on the REAL system?

    Let's say that a SPAM system uses my email address to send out 50000+ emails to the world. That means that my system will get hit with 50000+ requests to validate whether the server sending is really in a Mail Group.

    So on top of all of the ACTUAL work my system is doing, it now has to support all the SPAM systems that want to use my email address on their system.

    This works great for the AOL, Microsoft, etc, but what about the single server mail systems like mine. I support my own email server on my DSL.

    What about all the email sent as spam that refers to linux.org or openoffice.org that already handle alot of ACTUAL mail through the domains.

    Now it has to put up with all the SPAM messages requests to see if their real as well.

    Why not setup the SPAM filters at the Mail Reciept layer, and return a 5nn message saying SPAM is not excepted at this domain.

    They get paid per the actual receieved email. If everyone is killing their delivery before it is received then they don't get paid.

    Am I missing something here?

    --
    Scott Carr
  22. Re:Certificate based sender authentication by petong · · Score: 3, Interesting
    There is such a proposal to do this:

    domainkeys

    It allows the edge mail servers to sign an email, and it does not break email like SPF does.

  23. A community of Peers? by caudron · · Score: 2, Interesting

    Whatever happened to the idea that the Internet was a community of peers? I mean, my web browser doesn't care if the web site is registered with DNS or not. I can go to 125.10.233.5 as easily as I can go to linux.com. Likewise, my mail client doesn't care about DNS records. That used to be an optional part of the whole.

    Looks like the days when every machine was a peer on the Internet is gone in favor of the day when every machine must register with a superpeer (like DNS) to be considered a valid endpoint.

    Kinda sucks, if you ask me, to fight spam by ruining the best part of the Internet, ESPECIALLY WHEN THERE ARE BETTER ALTERNATIVES OUT THERE! Look at IM2000 or any similar idea. These would work just as well without requiring me to lose my status as valid endpoint and without me being forced to register with a superpeer, like DNS. :(

    --
    -Tom
  24. SPF is not going to solve any serious problems by Arrogant-Bastard · · Score: 2, Interesting

    1. Will it solve spam? Well, it was announced with the statement "Spam as a technical problem is solved by SPF." but -- despite the lack of a full public retraction of that statement, we're now told it was never intended to solve spam. Fine...but let's note that if it didn't claim to have something do with spam, few people, if any, would care about it.

    2. Will it solve forgeries? No. There are still a myriad of ways forgeries can be constructed -- and many of those will pass SPF checks, because they utilize the outbound mail servers as unforged mail. (And while you might think that the people running those mail servers would detect and correct this...you'd be wrong. They've built up multi-year track records demonstrating that they either don't know or don't care -- so it's unlikely they'll wake up tomorrow feeling differently.)

    3. Will it stop bounces from forgeries? Maybe. But it's important to note that it doesn't stop them by actually FIXING the broken mail systems which are generating them. This is a brittle approach to solving the problem which has as its only merit that it may be easier than tackling the core issue. But experience strongly suggests that it would be much better for the long-term health of the Internet's mail system to roll up our sleeves and actually deal with the bounce-vs-reject issue than find a way to sweep it under the carpet.

    4. Will it stop joe-jobs? Maybe -- if enough people use it, if spammers don't game it (and they're already working on it) and if it doesn't impose its own consequences. (Consider what will happen to your DNS servers if some spammer decides to joe-job your domain and N mail servers out there -- instead of just rejecting the traffic and being done with it -- all try to verify your SPF records simultaneously.)

    5. Does it solve an important problem? No. Forgery isn't a serious problem -- spam is.

    6. Does it solve the problem thoroughly? No. Really solving mail forgery requires secure DNS, public-key encryption of message content, secure end-user systems, secure mail clients, and other components. These all exist, but until they're ubiquitous, any sense that the mail forgery problem is "solved" is illusory.

    Despite all this, there are a few good ideas in SPF, and in DomainKeys, and in some of the other proposals which are out there. But those ideas need to be weighed in light of the environment in which we find ourselves, and take into account:

    • Millions of hijacked systems
    • Cheap "throwaway" domains
    • "Bulletproof" hosting
    • Rapidly-updating DNS (which just got worse)
    • non-SMTP methods of sending spam
    • poorly-maintained mail gateways
    • Scalability (SMTP, DNS, etc.) concerns
    among other things. For example, a much better (and far simpler) proposal that could be implemented immediately with minimal impact is: here. Of course, this doesn't solve everything either, nor does it claim to: but what it does tackle, it handles very well. Backers of SPF/DomainKeys/etc. would be well-advised to take a long look at it.