Slashdot Mirror


DynDNS Drops Non-Delivery Reports

jetkins writes "In an email to subscribers, DynDNS announced that they will no longer deliver locally generated non-delivery reports (NDRs) from any MailHop systems. MailHop is a multi-faceted service offering in- and outbound relay services, spam and virus filtering, and store-and-forward buffering. DynDNS makes it clear that they are aware that this goes against RFC 2821 Section 3.7, but explains that in their opinion the increase in spam volume, and the use of NDRs as a spam vector, means that the value of NDRs is now far outweighed by their potential for harm. (DynDNS also points to the far greater reliability of email systems now than when the RFC was approved.) The company notes that other ISPs have quietly dropped RFC 2821-compliant NDRs. Will their public move start a flood (mutiny) of ISPs following suit? Should they have made efforts to have the standard changed instead of defying it?"

23 of 195 comments (clear)

  1. Finally, a service provider with a clue... by Eggplant62 · · Score: 4, Informative

    Well, seeing as how a friend and I have a client who's being bombarded by NDRs as a result of a joe-job on the client's domain name, it's good to know that DynDNS is copping a clue. Too bad you can't get the rest of the ISP gang on board that easily and that quickly.

    1. Re:Finally, a service provider with a clue... by jetkins · · Score: 5, Informative
      A large proportion of Joe Jobs are made possible by lame endpoint SMTP servers which accept incoming mail, close the connection, then check to see if the recipient is valid, and generate an NDR to the address specified in the headers, which are too easily forged.

      A properly-configured endpoint server should check addressee validity during the SMTP exchange, and reject the transfer before it even gets into the system, so the spammer's attempt goes nowhere and "Joe" doesn't get an unwarranted NDR.

      Of course that doesn't help proxy providers like DynDNS, unless they have some way of authenticating their clients' valid addresses in real time via a direct connection or regular updates.

    2. Re:Finally, a service provider with a clue... by r3g3x · · Score: 3, Informative

      Well, seeing as how a friend and I have a client who's being bombarded by NDRs as a result of a joe-job on the client's domain name, it's good to know that DynDNS is copping a clue

      Ignoring a problem isn't the same as fixing it... NDRs serve a useful purpose assuming the original message was actually useful. The problem isn't sending out NDRs. The problem is sending an NDR in response to spam!

      I've had to deal with the whole joe-job+NDR+DDOS scenario on several occasions... I have found that 65~80% of this garbage has already been marked as spam by the bouncer! Why in the would would you bounce a message that you _already_ know to be invalid?!?!? All this does is amplify the volume of crap being flung around...

      A sanely configured email server should not bounce on spam. Spam should be left to rot in the bottomless pit of /dev/null.

      Remember, Its not a turd fight if theres only one monkey :-)

    3. Re:Finally, a service provider with a clue... by totally+bogus+dude · · Score: 4, Interesting

      Bunk. Even if it was true, it's still no excuse for ignoring your responsibilities.

      I run the mail servers for several domains, and brute-force attacks just don't happen. It's fairly obvious why, if you think about it. Dictionary attacks against common names are possible, but I've not seen evidence to suggest that's happening.

      Firstly, I want to get back to "responsibilities". By this I mean that anyone who's connected to the internet has a basic responsibility to make at least a good-faith attempt to prevent their system being used against other people. This goes doubly for people who intentionally run publically accessible servers (e.g. mail servers and web servers). You should treat any mail system which indiscriminately generates NDRs the same way you'd treat an open relay, because that's effectively what it is. You are deliberately making a server available which will accept mail from anyone on the internet, and send it to anyone else on the internet*. This is reckless irresponsibility.

      * - most NDR messages include at least part of the original message's text; at the very least, the subject line. So a system which generates backscatter does in fact carry some payload chosen by an anonymous third party.

      Even if brute-force attacks on your mail server's address list do occur, there are ways to mitigate the effects of it that don't turn your system into a spam engine.

      Having a look through the last 48 days logs on one of my servers, there's 2,308 addresses which were rejected because they're non-existent. The vast majority are either formerly valid addresses (i.e. of people who used to work here), or slightly mangled versions of valid addresses (presumably badly parsed). Particularly common are things starting with "3D" (presumably parsed from quoted-printable data which contains =3D) or people's surnames (smith@example.com) -- our email addresses are in the format firstname.lastname@example.com, and it would appear that some harvesters consider periods before the @ to be invalid.

      The second part highlights why brute-force is impractical: the namespace before the @ is absolutely massive, and only a tiny fraction will be valid addresses. If you have no idea what format email addresses in the target domain take, you have no choice but to try everything, and that will take far longer than a week. Add to this the proliferation of very small domains with only a handful of email addresses (i.e. personal domains, promotional domains). Even with a vast botnet, trying to harvest addresses by brute force against a mail server is so horribly inefficient as to not be worthwhile. There's much easier ways to harvest addresses.

      Then there's technical issues with that kind of harvesting. First, any reasonable mail server will start responding slower to a client which is making repeated errors, before finally shutting them off. This means you have to make lots of connections. Second, brute force or dictionary attacks stick out like a sore thumb versus normal mail traffic, making it trivial to block any IP which is trying to harvest addresses in this manner. The only possible way to do these sorts of attacks would be to use a vast distributed botnet, and even then it's not going to work. It would be easy (and fun) to build a system that watches for such attacks and blacklists any IP involved. Anyone harvesting in this way would then be betraying the IPs of most of their bots during the harvest! Then there's lots of clever things one can do: once you have a known harvester, start okaying its invalid addresses and add them to your list of spamtraps. Not only is the spammer not collecting any valid addresses, but you're poisoning their address list!

      Brute-force attacks are too easy to detect, and too easy to use against the attacker. There's much, much easier and more efficient ways to harvest email addresses. Possibly it could be used if you're targeting a specific company or domain and can do some research into their configuration, but even then there

    4. Re:Finally, a service provider with a clue... by Warbothong · · Score: 3, Interesting

      Just as an issue to note, I sent somebody a relatively important email recently from my Gmail account (accessed using Evolution via POP3 and SMTP). Around a week later I was at someone else's house and couldn't be bothered set my laptop up with their wireless system (their network is encumbered by encryption algorithms) so I used Google's webmail system to check my email. Sitting in the 'Spam' folder was a failed delivery notice from the important email I'd sent earlier (turns out the address I had used hadn't been in use for a while), it had been marked as spam only because it was a failed delivery message. Luckily the issue had already been resolved elsewhere, but with people relying on email for important communications something like this is unacceptable.

  2. RFC-Ignorant.org by nagora · · Score: 4, Interesting
    I did this some time ago for the same reasons and the wankers at RFC-Ignorant.org put my home email server on their blacklist. The twats argued with me that NDRs are such a vital part of email that any amount of spam was a price worth paying for maybe one NDR a year.

    Stupid bastards.

    --
    "Encyclopedia" is to "Wikipedia" what "Library" is to "Some people at a bus stop"
    1. Re:RFC-Ignorant.org by Anonymous Coward · · Score: 3, Insightful

      What's needed is for servers to start determining immediately whether they can deliver the mail or not and respond to the sender with an appropriate error code if not, instead of sitting on it for a few hours and then creating a bounce message. This can even work over multiple hops if the sending server isn't an open relay, the destination server replies to your ISP's mailserver with "550 Cant send shit captain!", and leave it up to your ISP to decide to retry, generate a bounce to you (which it should have no problem doing, after all, everyone sending email through its server has an account at the ISP, right?), or just drop the matter in the bit bucket.

      Once that initial connection is gone though, the receiving mailserver should not assume that it has any way to talk to the sender again.

    2. Re:RFC-Ignorant.org by flabbergasted · · Score: 3, Informative

      Why are you accepting a message for a nonexistent user in the first place? As soon as the sending SMTP connection specifies RCPT you should be able to check if it is valid and terminate the connection if it is for a nonexistent user. This can all be done before the DATA command is issued. Why waste cycles virus scanning, spam screening and bouncing a message for a user you don't even have? You're not just RFC ignorant, you're ignorant of how to properly run a mail server!

      Note that the method above gets rid of NDRs for nonexistent users but still provides them for a user over their quota or suffering other delivery problems.

    3. Re:RFC-Ignorant.org by plague3106 · · Score: 3, Insightful

      That's a great way to determine VALID accounts to spam.

    4. Re:RFC-Ignorant.org by fimbulvetr · · Score: 3, Insightful

      That's a great way to determine VALID accounts to spam. A lot of people bring up this point, but it's only ostensibly helpful anyway. The resources you save from not verifying an address are _quickly_ eaten up by the fact that you're queueing messages for invalid addresses on your domain at oftentimes insane rates. Pretty soon, your lame server is going to start to deliver these zillion+ NDRs and not only ruin the rest of your day for your users by stealing bandwidth and mail server resources, but also for many, many other people on the internet who need to delete the 80+ NDRs you sent them.
    5. Re:RFC-Ignorant.org by Menchi · · Score: 3, Informative

      There's even a solution for this kind of problem: It's called "recipient callout". The proxy SMTP Server will attempt a fake delivery attempt to the endpoint SMTP server before OKing a recipient. If it succeds, OK it, if it fails, deny it. Sure it costs some resources, but less than a bounce. And if you don't have enough resources to run an email server, don't run an email server.

      --
      Today's experiment ...... failed
  3. Change or Defy by Astrogen · · Score: 4, Insightful

    While I do believe they should initiate an effort to update the standard, if they view it as a security threat or a spam vector they are entirely right in shutting down the service.

    If a RFC said all boxes should have a port that users could telnet into with root access, and people start abusing that would you leave it and wait for the standard to change?

  4. SPF? by Southpaw018 · · Score: 4, Insightful

    If SPF were more widely implemented, or required to be implemented, wouldn't this problem be solved? Don't send NDRs to domains without SPFs or when SPF fails. NDRs get through and problem solved.

    And what DynDNS is doing is simply preventing all people from using their service from knowing whether email is being delivered properly. If I typo an email address, I damn well better be getting an NDR from the recipient domain, because simply having it go into an email black hole and never knowing whether it got there is not an acceptable alternative.

    --
    ACs are modded -6. I don't read you, I don't mod you, I don't see you. Don't like it? Don't be a coward.
  5. Re:What I'd like to see... by AuMatar · · Score: 4, Insightful

    And kill mailing lists. Not all mass mails are spam.

    --
    I still have more fans than freaks. WTF is wrong with you people?
  6. to defy the laws of tradition by chef_raekwon · · Score: 4, Insightful

    Should they have made efforts to have the standard changed instead of defying it?

    maybe by defying it, the standards will now be reviewed, and eventually changed.

    --
    We're like rats, in some experiment! -- George Costanza
  7. The Problem Is Not NDR's by jchawk · · Score: 4, Insightful

    It's email in general. The whole system is flawed and we've tried repeatedly to duct tape over the problem.

    The main problem is a you have a system based on blind trust.

    Second trust based duct-tape systems are simply too cumbersome for the average user.

    I don't have the answer but I do know that email in it's present state is broken.

  8. Standards and Implementation by LithiumX · · Score: 5, Insightful

    Tried and true standards make the net go round, but the most effective enhancements or changes to standards usually don't come from a committee working out best practices - it comes from individuals making hard choices on what to support. If those changes turn out to be beneficial, then they become adopted as new standards.

    Going against standards can cause a bit of chaos as well, which is why it's good to avoid deviation - but sometimes a deviation makes sense, and you do it. Publicly announcing this (non-critical) deviation, and explaining exactly why, is the proper way to go about it.

    --
    Do not confuse "Freedom of Choice" with "Free Will".
  9. Turn off original message in the bounce??? by Anonymous Coward · · Score: 3, Interesting

    Did I miss something or wouldn't the problem be solved by turning off the content of the original message in the bounce? If you can't see the original content, it removes the incentive for spammers to use that technique.

    This is how it goes on all our mail servers. All bounced messages have the original content stripped off. You get the error message with the reason the message bounced and that's it.

    NDR are still usefull. There is PLENTY of mail servers not configured properly or messed up on the Net, even from big ISPs. Calling the current system as a whole, reliable, is a joke.

  10. Are DynDNS cluebies? by jani · · Score: 5, Insightful

    With this reliabity levels of modern e-mail systems being substantially higher than its past predecessors, the practical needs for this NDR messages are nil. These practical, anti-spam, merits far outweigh the prevailing RFC 2822 technical requirements.


    Excuse me, but due to the vast amount of spam handling, modern e-mail systems are substantially less reliable than they used to be.

    If you redirect email for your domain name to Hotmail, chances are good that it will disappear without a trace. (No NDR, not in the spam box either.)

    Someone else already mentioned the problem of people typoing email addresses. This is a common problem.

    Email can be bounced for other reasons, too, such as a full mailbox, or that there is a relaying mail server (yes, DynDNS, they still exist, and in abundance!) which gives up on delivery after a week of timeouts for the destination host.

    And so on.

    Someone at DynDNS needs a good whack with the clue bat.
  11. NDR's are not evil by Anonymous Coward · · Score: 5, Insightful

    Throwing out the baby with the bath water comes to mind when I read this...

    The problem is not with NDRs. The problem is that their servers *accepted* the message that eventually had to be NDR'd in the first place, then after accepting responsibility, decided they didn't want that responsibility, so discarded mail that they promised they would deliver.

    If their servers checked validity of local recipients, scanned and filtered the message, etc BEFORE accepting it (via 2xx series SMTP accept response), and instead properly REJECTED it with a 5xx series response, these messages would never be bounced. The NDR mechanism is not at fault - rather, the fact that they can't properly configure their servers to reject the message up front is at fault. If you properly REJECT the messages at the SMTP level instead of accepting the message for delivery, the only thing left to NDR are perfectly valid cases, such as mailbox over quota, etc.

    Once you *accept* responsibility to deliver a message (via a 2xx series SMTP response), you MUST deliver it somewhere, else you have shirked your responsibility - either deliver it to it's destination, or bounce it. To do anything else would be to LOSE mail, which is the ultimate sin of any mail server. The key is not to throw out bounce messages, but to minimize or eliminate unnecessary bounces in the first place by rejecting instead.

    Note that by properly REJECTING the message, you also effectively defeat most spam bots, since they can't "bounce" the mail that you reject to the "real" local sender.

    I always hate it when providers like this take the short cut of *losing* mail intentionally rather than fixing their broken systems to work right.

    One caveat to my comments - unfortunately, some mail software is symply not geared toward todays Internet, such that it can do the scanning and filtering of messages realtime fast enough to prevent a sending server from timing out while it's doing this scanning, so they queue the mail to process it for spam, etc later. Using such software is the first mistake most places make, and is the real reason why there are so many NDR's in the first place.

  12. Re:Start at the top, with secure DNS. by greed · · Score: 3, Informative

    Way to confuse envelope-from, header-from and reply-to.

    Besides, my home-brewed Linux-based mail server has a published SPF record, and anyone receiving mail can verify that machine is entitled to generate envelope-from with that domain. The SPF also spells out my relay provider, since my DSL line is in DSL blocklists.

    What it really needs, at the least, is for people to stop accepting bogus HELO/EHLO addresses and other unverifiable envelope information. If there isn't even an A record for the HELO address, then 554 the message.

    This means mail from many large corporations will be rejected, because they use HELO hostnames that only resolve inside the company.

  13. So USE that information. by khasim · · Score: 4, Insightful

    Yes, the spammer can determine whether it has a valid account. But that means ...

    #1. The spammer already HAS the account name and is checking to see if it still works. Defeat this by generously distributing SpamTrap accounts. And accepting email to them. And then opt'ing out of the email that they receive.

    #2. The spammer is trying to guess a new name. Good luck with that. Sure, maybe SOMEWHERE there is an email account of "frank@example.com" but good luck finding it. If you want to have some FUN, watch your logs for examples of this. Then setup some of them as SpamTraps. And follow #1 above.

    I use both of these approaches. It makes filtering spam VERY easy.

  14. Re:What I'd like to see... by adminstring · · Score: 3, Insightful

    Here's an example of how mailing lists can be useful: I'm into music and bicycles, and I get periodic emails from several online vendors of music and bike gear letting me know what specials they are having, and quite often I see something I like in these emails, so I click a link and go shopping. I opted in to these mailing lists, and they help me find deals I wouldn't otherwise be aware of.

    I wouldn't probably think to check a forum for an announcement on a free-shipping sale or a closeout on last year's tires, but if it comes in an email, it gets my attention, and I appreciate that.

    My ISP offers spam-filtering, but I have turned it off because too many of these mailing lists I like were getting caught in its trap. So I agree that mailing lists make spam filtering more difficult, but I personally see them as being worth the hassle.

    --
    My truck is like a series of tubes.