How are Your SMTP Timeouts Configured?

Asprin asks: "One of the employees at work had a major headache because a very important email was undeliverable for more than 24 hours. Sure, he got an warning from our server about it, but only after an entire day had passed, and the email was no longer timely. Therefore, I shorted the message handling timeouts to send 'delivery delayed' warnings after 30 minutes and to cancel the message as undeliverable after four hours. Now, I don't expect any of the other mail administrators here to bless these timeouts because they're way too short. HOWEVER, the truth is that my users rely on email to be as reliable as telephone messages, and if it can't be delivered immediately, it is better to reject it outright and alert the user so that other communication channels can be exploited such as fax or Fed-Ex. Is anyone else doing this? Are there any non-obvious ramifications lurking? Pros? Cons? Comments? Should we all reduce these timeouts on our SMTP servers?"

Comments

Defaults of 4 hours and 4 days.

[ComputerSlicer23]

I believe our mail server sends an alert 4 hours after an e-mail is non-deliverable, and retries at regular intervals for up to 4 days I believe. I think delivery is either every 1 hour, or every 4 hours. Not sure what the Sendmail defaults are.

That said, even if your e-mail server doesn't send you the outage, that doesn't mean the e-mail actually got there. It could have been received by a secondary MX, not the primary one that delivers it.

I'm sure everybody and their brother will mention that read receipts, and receive receipts are a good idea in this case (even those are reliable, but it's better then nothing). Oh, and that if the message was this important, at the very least a confirmation call. You might look like a character out of a Dilbert strip, but it sounds like confirmation would have been worth the embarrassment in this case.

Kirby

Re:Defaults of 4 hours and 4 days.

[Mattcelt]

That said, even if your e-mail server doesn't send you the outage, that doesn't mean the e-mail actually got there. It could have been received by a secondary MX, not the primary one that delivers it.

At two of the companies I've worked, there has been a stated policy that email was not "reliable" as a communications mechanism and that the IT department made no guarantees about the usefulness or capability of email or other IP-based data exchanges. As one of my managers put it, "There's no SLA for the Internet."

One of the problems with the Internet has been, rather ironically, its overwhelming success. When most of our emails get to their destinations within a matter of seconds or minutes, it begets an unrealistic expectation that it will always be that way, especially for those who do not understand the fractured and codependent nature of the Internet.

Your solution to the problem - letting the user know an email couldn't be delivered in a short period of time, while still trying to deliver for the full timeout period - is probably the best one in this situation.

Re:Defaults of 4 hours and 4 days.

[I8TheWorm]

At two of the companies I've worked, there has been a stated policy that email was not "reliable" as a communications mechanism and that the IT department made no guarantees about the usefulness or capability of email or other IP-based data exchanges

I liken that to the sign you see at the dry cleaner that says "Not responsible for lost or damaged items"

Granted, the sign at the dry cleaner isn't true, and only meant as a deterent for lawsuits. However, wether the IT department puts up a sign that says "I told you it might not work" or not, users are very dependant on that service. And when the CEO calls and says "my e-mail didn't go through" the response is not likely to be something like the above.

Impatience...

[ptaff]

When I was a kid, I used to think that a 2400 modem was really fast. You could download a 300KB game in a few minutes. And I could store dozens of them on my 20MB hard drive.

When I hear newbies complaining about their slow 300KB/s connections and too small 100GB storage units, I feel anger inside. They just can't appreciate the value of technology.

When E-mail was introduced 30 years ago, it was an amazing feat: you could send messages across the country in less time than regular postal service. Wow.

Now we're complaining about limitations of a 30-year old technology that works as intended. Come on. It's still amazing. There are IMs, IRC channels all over the place for these "urgent needs".

Don't blame the hammer if it doesn't mow your lawn.

Re:Impatience...

[turg]

He's not blaming the hammer. He just wants to inform the user in a timely fashion that the lawn still needs mowing.

I completely agree.

[hawkstone]

For God's sake, yes!

Problem number one: For the most part, email is perfectly reliable. If it isn't delivered half an hour, 99% of the time it's because I screwed up the address. I'd like to know after 5 minutes, but I'd take half an hour. And I don't want the computer trying for four freaking days to send an email that I messed up.

Problem number two: Let's say there was a legitimate problem with the network. A router was taken down for maintenance, for instance. These days, people grumble if it's down for more than 10 minutes, and few outages last more than a couple hours. For the re-try interval, 12 hours is probably sufficient, but 24 will cover an overnight outage and its subsequent fix with time to spare. Heck -- How many outages last for more than a day? In the rare event that it does, it may last a week, or maybe a permanent change occurred to keep the mail from ever being deliverable.

So, I have no advice to you other than please, please make everyone you know configure their system as such.

(Flames -- err, I mean opposing viewpoints -- welcome.)

I almost completely disagree

[infra-red]

Uhm... NO!!!

The timeouts are there to handle cases where a remote server is off the net for whatever reason. While I can see shortening the warning message, your not helping yourself if you shorten the period of time that the server attempts to deliver for.

Sendmail (I'm not sure what MTA is being used for this example, but I would hope that would be irrelevant) can handle multiple queue times based on the priority of the message. With this you could have the high priority mail fail in 12 hours while normal mail takes a normal amount of time.

When mail runs great, its smooth and very timely. But when it breaks, it can go down hard.

In my experience, the recovery of a mail server isn't what takes the most time, its the ammount of time it takes to process the backlog from other servers queues.

If you run at 50% capacity, then basically, an outage of 2 hours will take you 2 hours more to get caught up, and thats assumeing that your server is running optimally. Best way to find bottlenecks in your mail servers is to shut it down for an hour and see what stops it from working (syslog is great for doing this if you have unbuffered logging).

Timeouts are there to help the system recover when something goes wrong. Use the priorities to change the timeouts, but dropping mail too quickly is just doing everyone a disservice.

Personal experience. I saw a system setup to drop all messages that were not delivered in 45 minutes. I was floored when I saw this. They had a problem with the machine and their system took almost a week to stabalize and catch up (underpowered systems running too many opposing services. DNS on the mail servers is not good since when you do alot of mail, your lookups steal CPU from your mail servers and the problem gets amplified when you processing the backlog)

Expectations

[sakyamuni]

I think the main issue here is expectations. Make sure you set those right. Your users' expectations sound a little off the mark for E-mail. I don't expect them to understand what's involved under the hood, so maybe you need to educate them. Servers can and do go down. A four-hour window is just way too small.

Use another medium for super-urgent communications, like IM or phone. Or just simply follow up with a phone call: "Did you get that urgent E-mail? No? OK, I'll look into it."

Re:Why rely on email?

[raju1kabir]

I find the best way to use email is to email the important information, then print out a copy and fax the copy, then call the recipient to find out if they received both the email and the fax.

Why an email AND a fax?

Surely it would be faster to send the email, then make the phone call, then send the fax only if the email failed. No further call is required because the fax protocol, unlike email, has confirmation built in.

Plus, both you and the recipient save paper and avoid the disposal hassle for sensitive documents.

Educate your users

[PD]

E-mail is asynchronous. There's no guarantee that it will be delivered in a certain amount of time.

The telephone is synchronous. When it's answered, you're talking to the person real-time.

You've got to pick the right communications medium for the message. If your house is burning down, would you send an e-mail to the fire department? You'd be stupid to do that. And, going the other way, trying to turn a synchronous communications medium into an asynchronous one has given us all the curse of the 'voice mail' and the 'answering machine'. Voice mail is a hack, trying to make the telephone do something it wasn't meant to do. And, as a hack, it's cumbersome to use. Virtually all voice mail systems work differently, and they make you go through their menus in a serial fashion. See how long it takes to delete 100 voice mails, and you'll know what I mean.

Educate your users.