DynDNS Drops Non-Delivery Reports

jetkins writes "In an email to subscribers, DynDNS announced that they will no longer deliver locally generated non-delivery reports (NDRs) from any MailHop systems. MailHop is a multi-faceted service offering in- and outbound relay services, spam and virus filtering, and store-and-forward buffering. DynDNS makes it clear that they are aware that this goes against RFC 2821 Section 3.7, but explains that in their opinion the increase in spam volume, and the use of NDRs as a spam vector, means that the value of NDRs is now far outweighed by their potential for harm. (DynDNS also points to the far greater reliability of email systems now than when the RFC was approved.) The company notes that other ISPs have quietly dropped RFC 2821-compliant NDRs. Will their public move start a flood (mutiny) of ISPs following suit? Should they have made efforts to have the standard changed instead of defying it?"

Comments

Finally, a service provider with a clue...

[Eggplant62]

Well, seeing as how a friend and I have a client who's being bombarded by NDRs as a result of a joe-job on the client's domain name, it's good to know that DynDNS is copping a clue. Too bad you can't get the rest of the ISP gang on board that easily and that quickly.

Re:Finally, a service provider with a clue...

[jetkins]

A large proportion of Joe Jobs are made possible by lame endpoint SMTP servers which accept incoming mail, close the connection, then check to see if the recipient is valid, and generate an NDR to the address specified in the headers, which are too easily forged.

A properly-configured endpoint server should check addressee validity during the SMTP exchange, and reject the transfer before it even gets into the system, so the spammer's attempt goes nowhere and "Joe" doesn't get an unwarranted NDR.

Of course that doesn't help proxy providers like DynDNS, unless they have some way of authenticating their clients' valid addresses in real time via a direct connection or regular updates.

Re:Finally, a service provider with a clue...

[totally bogus dude]

Bunk. Even if it was true, it's still no excuse for ignoring your responsibilities.

I run the mail servers for several domains, and brute-force attacks just don't happen. It's fairly obvious why, if you think about it. Dictionary attacks against common names are possible, but I've not seen evidence to suggest that's happening.

Firstly, I want to get back to "responsibilities". By this I mean that anyone who's connected to the internet has a basic responsibility to make at least a good-faith attempt to prevent their system being used against other people. This goes doubly for people who intentionally run publically accessible servers (e.g. mail servers and web servers). You should treat any mail system which indiscriminately generates NDRs the same way you'd treat an open relay, because that's effectively what it is. You are deliberately making a server available which will accept mail from anyone on the internet, and send it to anyone else on the internet*. This is reckless irresponsibility.

* - most NDR messages include at least part of the original message's text; at the very least, the subject line. So a system which generates backscatter does in fact carry some payload chosen by an anonymous third party.

Even if brute-force attacks on your mail server's address list do occur, there are ways to mitigate the effects of it that don't turn your system into a spam engine.

Having a look through the last 48 days logs on one of my servers, there's 2,308 addresses which were rejected because they're non-existent. The vast majority are either formerly valid addresses (i.e. of people who used to work here), or slightly mangled versions of valid addresses (presumably badly parsed). Particularly common are things starting with "3D" (presumably parsed from quoted-printable data which contains =3D) or people's surnames (smith@example.com) -- our email addresses are in the format firstname.lastname@example.com, and it would appear that some harvesters consider periods before the @ to be invalid.

The second part highlights why brute-force is impractical: the namespace before the @ is absolutely massive, and only a tiny fraction will be valid addresses. If you have no idea what format email addresses in the target domain take, you have no choice but to try everything, and that will take far longer than a week. Add to this the proliferation of very small domains with only a handful of email addresses (i.e. personal domains, promotional domains). Even with a vast botnet, trying to harvest addresses by brute force against a mail server is so horribly inefficient as to not be worthwhile. There's much easier ways to harvest addresses.

Then there's technical issues with that kind of harvesting. First, any reasonable mail server will start responding slower to a client which is making repeated errors, before finally shutting them off. This means you have to make lots of connections. Second, brute force or dictionary attacks stick out like a sore thumb versus normal mail traffic, making it trivial to block any IP which is trying to harvest addresses in this manner. The only possible way to do these sorts of attacks would be to use a vast distributed botnet, and even then it's not going to work. It would be easy (and fun) to build a system that watches for such attacks and blacklists any IP involved. Anyone harvesting in this way would then be betraying the IPs of most of their bots during the harvest! Then there's lots of clever things one can do: once you have a known harvester, start okaying its invalid addresses and add them to your list of spamtraps. Not only is the spammer not collecting any valid addresses, but you're poisoning their address list!

Brute-force attacks are too easy to detect, and too easy to use against the attacker. There's much, much easier and more efficient ways to harvest email addresses. Possibly it could be used if you're targeting a specific company or domain and can do some research into their configuration, but even then there

RFC-Ignorant.org

[nagora]

I did this some time ago for the same reasons and the wankers at RFC-Ignorant.org put my home email server on their blacklist. The twats argued with me that NDRs are such a vital part of email that any amount of spam was a price worth paying for maybe one NDR a year.

Stupid bastards.

Change or Defy

[Astrogen]

While I do believe they should initiate an effort to update the standard, if they view it as a security threat or a spam vector they are entirely right in shutting down the service.

If a RFC said all boxes should have a port that users could telnet into with root access, and people start abusing that would you leave it and wait for the standard to change?

SPF?

[Southpaw018]

If SPF were more widely implemented, or required to be implemented, wouldn't this problem be solved? Don't send NDRs to domains without SPFs or when SPF fails. NDRs get through and problem solved.

And what DynDNS is doing is simply preventing all people from using their service from knowing whether email is being delivered properly. If I typo an email address, I damn well better be getting an NDR from the recipient domain, because simply having it go into an email black hole and never knowing whether it got there is not an acceptable alternative.

Re:What I'd like to see...

[AuMatar]

And kill mailing lists. Not all mass mails are spam.

to defy the laws of tradition

[chef_raekwon]

Should they have made efforts to have the standard changed instead of defying it?

maybe by defying it, the standards will now be reviewed, and eventually changed.

The Problem Is Not NDR's

[jchawk]

It's email in general. The whole system is flawed and we've tried repeatedly to duct tape over the problem.

The main problem is a you have a system based on blind trust.

Second trust based duct-tape systems are simply too cumbersome for the average user.

I don't have the answer but I do know that email in it's present state is broken.

Standards and Implementation

[LithiumX]

Tried and true standards make the net go round, but the most effective enhancements or changes to standards usually don't come from a committee working out best practices - it comes from individuals making hard choices on what to support. If those changes turn out to be beneficial, then they become adopted as new standards.

Going against standards can cause a bit of chaos as well, which is why it's good to avoid deviation - but sometimes a deviation makes sense, and you do it. Publicly announcing this (non-critical) deviation, and explaining exactly why, is the proper way to go about it.

Are DynDNS cluebies?

[jani]

With this reliabity levels of modern e-mail systems being substantially higher than its past predecessors, the practical needs for this NDR messages are nil. These practical, anti-spam, merits far outweigh the prevailing RFC 2822 technical requirements.

Excuse me, but due to the vast amount of spam handling, modern e-mail systems are substantially less reliable than they used to be.

If you redirect email for your domain name to Hotmail, chances are good that it will disappear without a trace. (No NDR, not in the spam box either.)

Someone else already mentioned the problem of people typoing email addresses. This is a common problem.

Email can be bounced for other reasons, too, such as a full mailbox, or that there is a relaying mail server (yes, DynDNS, they still exist, and in abundance!) which gives up on delivery after a week of timeouts for the destination host.

And so on.

Someone at DynDNS needs a good whack with the clue bat.

NDR's are not evil

[Anonymous Coward]

Throwing out the baby with the bath water comes to mind when I read this...

The problem is not with NDRs. The problem is that their servers *accepted* the message that eventually had to be NDR'd in the first place, then after accepting responsibility, decided they didn't want that responsibility, so discarded mail that they promised they would deliver.

If their servers checked validity of local recipients, scanned and filtered the message, etc BEFORE accepting it (via 2xx series SMTP accept response), and instead properly REJECTED it with a 5xx series response, these messages would never be bounced. The NDR mechanism is not at fault - rather, the fact that they can't properly configure their servers to reject the message up front is at fault. If you properly REJECT the messages at the SMTP level instead of accepting the message for delivery, the only thing left to NDR are perfectly valid cases, such as mailbox over quota, etc.

Once you *accept* responsibility to deliver a message (via a 2xx series SMTP response), you MUST deliver it somewhere, else you have shirked your responsibility - either deliver it to it's destination, or bounce it. To do anything else would be to LOSE mail, which is the ultimate sin of any mail server. The key is not to throw out bounce messages, but to minimize or eliminate unnecessary bounces in the first place by rejecting instead.

Note that by properly REJECTING the message, you also effectively defeat most spam bots, since they can't "bounce" the mail that you reject to the "real" local sender.

I always hate it when providers like this take the short cut of *losing* mail intentionally rather than fixing their broken systems to work right.

One caveat to my comments - unfortunately, some mail software is symply not geared toward todays Internet, such that it can do the scanning and filtering of messages realtime fast enough to prevent a sending server from timing out while it's doing this scanning, so they queue the mail to process it for spam, etc later. Using such software is the first mistake most places make, and is the real reason why there are so many NDR's in the first place.

So USE that information.

[khasim]

Yes, the spammer can determine whether it has a valid account. But that means ...

#1. The spammer already HAS the account name and is checking to see if it still works. Defeat this by generously distributing SpamTrap accounts. And accepting email to them. And then opt'ing out of the email that they receive.

#2. The spammer is trying to guess a new name. Good luck with that. Sure, maybe SOMEWHERE there is an email account of "frank@example.com" but good luck finding it. If you want to have some FUN, watch your logs for examples of this. Then setup some of them as SpamTraps. And follow #1 above.

I use both of these approaches. It makes filtering spam VERY easy.