Preparing Your Datacenters for DST Changes?

Cheeze asks: "As I am sure some of you know, Daylight Saving Time is slated to change this year thanks to The Energy Policy Act of 2005. This means nothing to the large majority of the population except they will either sleep late one day or have to commute in the dark. To a select few, this is a crunch time akin to the Y2K fiasco, only there has been almost zero publicity recently. These select few are the ones responsible for updating the millions of computers, both servers and workstations, with the new time zone information. For newer servers, this usually means just install a patch and reboot (which is slightly more than mildly inconvenient). For older servers, this is basically an 'End of Life' declaration. Servers running software for which no patch is available will be unable to update their own clocks. This doesn't seem like such a big deal until you realize Microsoft is only offering patches for Windows XP and beyond, and Sun will not be supporting Solaris 7 and older. That should knock a large percentage of the computers 1 hour off for a few weeks this spring. What are you doing in your datacenters to prepare?"

Comments

Are you saying..

[Anonymous Coward]

..that your servers don't use GMT?

Re:What are you doing in your datacenters to prepa

[Anonymous Coward]

Mod parent up. ISTM that you're better off if all computers have their hardware clocks set to UTC and the conversions to local time take place at the last possible time in the clients (if at all). If a user wants to use some client software or OS that doesn't understand the silly daylight savings rules (which are extremely crazy in some places); at least they don't damage anything in the datacenter and will only confuse themselves.

OTOH, what kind of evil company won't even give as trivial a patch as updating a time-zone file to their customers!?! If I had any such software and the company still existed I'd be wanting a full refund.

You also have to look for what you applications do

[LordKronos]

Thats a good idea, but won't always solve all of your problems. Here's a problem I experienced recently with one of our web apps.

We have a web event calendar where people can schedule events (for far into the future). When people input an event, they specify the date/time it will begin/end. The application then converts the date/time into a unix timestamp and stores it in the database. Later, when an event is being viewed, the timestamp is converted back to a textual date/time.

The conversion from local time to timestamp is done via PHP functions, which uses the systems timezone file. The OS patch to fix this problem simply updates the timezone file and everything should automatically work.

This is fine and dandy for most things, but I ran into one small glitch. For any events that are scheduled between the new start of DST and the old start of DST (roughly a 3 week period), if they were created BEFORE the patch was applied, they are now off by 1 hour AFTER the patch is applied.

The 1 hour difference is simple enough, and I might not have even noticed...or if I had, I probably would have just blamed it on user error when entering the data. However, this caused a strange side effect. The application built a secondary index to simplify searching for events. The index consisted of the event_id and the starting timestamp truncated to midnight. Of course, for those events in that 3 week period that were created before the patch was applied, they now had an index timestamp that was actually 1 AM. Of course, this caused events to mysteriously disappear from the calendar. You could search for them, but not see them when browsing.

I wasted a good couple of hours on this problem (thinking at first that it was isolated to a single event) before I finally did an analysis of the entire index and discovered that almost all events in that 3 week period were wrong. Then it instantly dawned on me what had happened.

Re:Overhyped

[Cheeze]

I don't think you work in a datacenter then. Once our servers are configured and running, there should NEVER be a reason to reboot them. In fact, when we have an odd server failure, the server usually has been up since it was first put in the datacenter. We are just now getting failures of systems put up in the late 1990s. Sure we have new systems as well, but there are critical systems still running old hardware because it works and a reboot is too much of an outage to take.

If you don't think it could be inconvenient to patch and reboot hundreds or thousands of servers, I don't think you are looking at the whole picture.

That vague possibility of some embedded systems being thrown away include, vcrs (good riddance), DVRs, watches, GPS devices, car computers, phone systems, just about all networking gear, etc. These are things that will either need a software upgrade or a hardware upgrade within the next 3 months if they are to give the correct time.