Anyone Besides Zune Owners With New Year's Crashes?

aputerguy writes "My Fedora 8 Linux server crashed sometime between 18:59:40 EST (GMT -5:00) and 19:00:00 EST (GMT -5:00) on Dec 31, 2008 which remarkably corresponds to within at most 20 seconds of the New Year in GMT. I have been running this same hardware non-stop for more than six years and other than the occasional reboot for kernel (or distro) upgrades, it has not crashed more than 1 or 2 times in 2237 days of cumulative uptime. Nothing other than background processes were running at the time of the crash. Could this be a coincidence or was there some 2008/2009 rollover issue going on here? Has anyone (other than Zune 30GB owners) noticed similar year-end issues with their computers or electronic devices?"

Comments

Well

[Anonymous Coward]

Well, you know what they say, this wouldn't have happened with Red Hat.

Re:Well

[binarylarry]

Nobody got fired for going with Redhat?

Re:Well

[palegray.net]

You're fired.

SKY TV set top box

[Anonymous Coward]

Here in the UK, our skytv settop box crashed (lost all tv channels but not the menus precisley at 00:00 1/1/2009 needed a cold boot to get the channels back.

Re:SKY TV set top box

[Thanster]

Replying to myself, as I forgot my login details briefly. Sky tv set top box crashed precisely at midnight (was sadly watching the newyears TV stuff. Had to switch over to the old fashioned arial to watch the london fireworks. Did this happen to anyone else (thinking unlikely to find many people willing to admit watching the newyear on tv!) (personal excuse is having a young child!)

Re:SKY TV set top box

[sentientbeing]

A similar thing, though probably unrelated to the leap second - my parents VHS clock has been flashing 12:00 since 1986.
It would probably bring bad luck for the new year to set it correctly for 2009, so I think ill leave it.

Re:SKY TV set top box

[inKubus]

That's because typically Cable (or Sat) channels are contracted to carriers over a calendar year. So, at midnight on Jan 1st, some channels are added and some dropped. You probably will notice new channels and a few missing ones if you look close.

Re:SKY TV set top box

[MentalMooMan]

My mythtv box (running mythbuntu) crashed within about a second of midnight as I was trying to watch the fireworks, and stopped responding to ping, ssh, everything.
My excuse for staying in and watching the celebrations on TV is that... my dog ate... my shoes.

Yes, that'll do...

Well, after drinking a couple of beers

[Anonymous Coward]

I let a bottle fall and it broke. Does it count?

Re:Well, after drinking a couple of beers

[rishistar]

Being more refined, I spilt a Gin & Tonic all over my keyboard. The keyboard doesn't work now - maybe it was the drink, but as it was an MS jobby I'm willing to bet it was a Zune like crash. Crappy Microsoft products.

Re:Well, after drinking a couple of beers

[Sebilrazen]

For the parent to copy and paste their post with just the mouse had to have been a nightmare.

Re:Well, after drinking a couple of beers

[Anonymous Coward]

Yes, I spilled Gin & Tonic on my laptop a few months ago. I had to replace the Keyboard. Frickin' Vista...

Re:Well, after drinking a couple of beers

[Beer_Smurf]

Sorry to hear about your G&T, were you able to recover much of it?

nope...

[Anonymous Coward]

debian etch, RHEL, centos, all 300 odd servers stayed up. so did irix and solaris boxen from ancient times of the roman empire..

Re:nope...

[bluelip]

why doesn't he just set the time back and let the new year happen all over again?

Re:nope...

[Anonymous Coward]

That wouldn't be a conclusive test. Servers are connected to networks and have persistent storage. Unless the network behaves like it's the last seconds of 2008 GMT and the disk is in the same state as before the crash, a smooth transition is no indication that the problem wasn't date related.

Re:nope...

[jibjibjib]

But if it /did/ crash, then that would be very strong evidence that it /was/ date-related, and then he could find the cause and make sure it didn't happen next time. So, it might still be a useful thing to do.

Re:nope...

[1s44c]

My Debian lenny laptop froze showing 00:59 (CET). Wouldn't respond to mouse, keyboard or ssh.

Thats right when the leap second hit. Time changes can cause arts to freakout which can be nasty if it's running with realtime priority. Maybe other software does the same?

No.

[Anonymous Coward]

No.

You are alone. Very, very alone.

Well this is obvious...

[oskard]

My Microsoft Windows desktop crashed sometime between 18:59:40 EST (GMT -5:00) and 19:00:00 EST (GMT -5:00) on Dec 31, 2008 which remarkably corresponds to within at most 20 seconds of the New Year in GMT. I have been running this same hardware non-stop for more than 5 hours and other than the occasional BSOD and Windows updates, it has not crashed more than 1 or 2 times in 174,237 seconds of cumulative uptime. Nothing other than spyware, malware, and System Idle Process were running at the time of the crash.

Re:Well this is obvious...

[oskard]

It was just a joke, sorry, I don't want anyone's shit on me :(

Re:Well this is obvious...

[Anonymous Coward]

What's with all the 4chan idiocy on Slashdot recently?
4chan is funny when you're a teenage boy, but for those of us that aren't...

Re:Well this is obvious...

[Forty Two Tenfold]

Ever heard of OLPC project? Heeere's the result!

No problems

[sdo1]

Nope. Everything's fine here in New Ampst

<carrier lost>

Errrrrrr

[segedunum]

Why don't you actually boot it, or failing that, take the hard drive out, perhaps look at some logs and actually find out rather than aligning it with a certain set of mystical circumstances?

Re:Errrrrrr

[geekoid]

Because thinking rationally is hard.

Re:Errrrrrr

[aputerguy]

I rebooted it and it went just fine. I looked at the logs and saw now errors. Last entry was in /var/log/maillog at 18:59:40 (not an error). So, not sure how to figure it out - tempted to try to replicate though by setting time back to 18:59 on 12/31/08 (and shutting off ntpd)

Re:Errrrrrr

[nurb432]

Its more fun to be paranoid. Join the club. No wait, we don't trust you to join you might be one of 'them'.

My phone did it

[cr_nucleus]

My phone froze right after midnight and i had to remove the battery to make it work again.
It's a SE w810i.

Probably coincidence.

[Thiez]

> Could this be a coincidence

Yes. People are wired to see causality everywhere, even where there is none. Had your server crashed a week ago you wouldn't think anything of it (maybe 5% of all servers mysteriously crashed exactly one week ago, but because it was an 'ordinary' day nobody noticed). Anyway, since you noticed your server crashed at new year and reported it on /., and with 6 billion people on this planet we will soon hear stories about other computers that mysteriously crashed around midnight. Not because there has to be anything special, but because computers are crashing all the time and new year (and your post) made it appear special.

I doubt it has anything to do with leap seconds, if your computer ran for 6 years it survived the leap second of 2005.

Re:Probably coincidence.

[DoofusOfDeath]

Yes. People are wired to see causality everywhere, even where there is none.

So you see a pattern in people's behavior? ;)

Re:Probably coincidence.

[Snowblindeye]

People are wired to see causality everywhere, even where there is none.

Very true. There is an interesting book by Leonard Mlodinow called "The Drunkard's Walk: How Randomness Rules Our Lives" which is all about the way humans misinterpret random events to see patterns that are not there.

http://www.randomhouse.com/pantheon/catalog/display.pperl?isbn=9780375424045

http://www.amazon.com/Drunkards-Walk-Randomness-Rules-Lives/dp/0375424040

Re:Probably coincidence.

[Thiez]

Let's use that number. The odds of a server failing during the 20 seconds before midnight on 31 december are 1 in 5 million. Suppose there are 50 millions servers. Simple math says the chance of your server crashing is extremely small (1 in 5 million), but there will be about 10 people who have a crashed server. That is normal (using your number there will be 10 servers crashing every 20 seconds every day of the year) but those 10 people will think it 'an awfully unlikely coincidence', while the other 15379200 server crashes during a year are ignored.

Don't get me wrong, I'm not saying the new year can't have anything to do with the crash, I just think it's way more likely that your server crashed randomly and you see causality where none exists.

Re:Probably coincidence.

[O('_')O_Bush]

In 50 million servers in america (number for number's sake), that makes 10 people who crashed at midnight. Most of them were IT people given the nature of owning a server, and IT people often read slashdot.

Hence, you, and the 7-9 other people who shared your experience... and nobody else.

Re:Probably coincidence.

[AbyssWyrm]

I think your logic is incorrect. The original poster did not say "my server went down around midnight, could this be a coincidence?" rather he said "my server, which has a particularly excellent track record of not going down, did so near midnight with very high precision. Could this not be a coincidence?" Given that this happening at any specific time is very unlikely compared to the relative abundance of rollover errors, this is a very legitimate hypothesis. Furthermore your argument is essentially saying that anything with a non-zero probability of occurring randomly is probably not a coincidence. Otherwise, instead of comparing to some 50 million servers you ought to be comparing to a much smaller number of servers meeting the description of the original poster's. I don't think you pose any legitimate argument that this is coincidental, and it strikes me as very probable that it is not.

Re:Probably coincidence.

[Guido von Guido]

I think your logic is incorrect. The original poster did not say "my server went down around midnight, could this be a coincidence?" rather he said "my server, which has a particularly excellent track record of not going down, did so near midnight with very high precision. Could this not be a coincidence?" Given that this happening at any specific time is very unlikely compared to the relative abundance of rollover errors, this is a very legitimate hypothesis. Furthermore your argument is essentially saying that anything with a non-zero probability of occurring randomly is probably not a coincidence. Otherwise, instead of comparing to some 50 million servers you ought to be comparing to a much smaller number of servers meeting the description of the original poster's. I don't think you pose any legitimate argument that this is coincidental, and it strikes me as very probable that it is not.

If you want to show that this is anything but a coincidence, you either need to show that this happened to more than one server, or you need to demonstrate the mechsnism. At this point we have exactly one server and we can't point to a specific bug. Until that changes, "coincidence" is the best answer.

For instance, this could be an entirely local problem. The motherboard or some other hardware component is beginning to fail, and the server will start crashing more frequently until that component dies completely. Or it could have been caused by a power surge, or a problem resulting from some bad wiring. Or the guy who manages the server above it came in to swap out some hardware and accidentally unplugged the server, and won't admit to it. (I have a former boss who did exactly that, after he went to work for a customer.)

Sure, it could still be related to the time. Without any additional evidence, though, it's just speculation.

Given an infinite number of server monkeys...

[melonman]

How many servers in total are watched over by people posting on Slashdot? I suspect that the answer is high enough that it would be amazing if at least one of them didn't crash within 20 seconds of the New Year.

I Second That

[Anonymous Coward]

My parents are using a MythTV box on Fedora 8 (Athlon XP1700+) and it also froze up last night at the same time (right in the middle of a recording :-( ). That was my first thought, too, because that would have been midnight UTC. However, after restarting it today, is has frozen again.

I can't see anything in the logs, but the recording ended at 19:59 AST. It should have kept going for another hour.

I have a second MythTV/Fedora 8 box (P3, 1GHz) that I use and never had any trouble with it last night.

Re:I Second That

[aputerguy]

Interesting, I run mythtv too but wasn't recording at the time of the crash. Has been stable since I rebooted a couple of hours ago.

Re:I Second That

[Anonymous Coward]

I did some comparisons between these 2 boxes and found the following. We were both recording the same program.

messages - both had entries from the channel change script at about 19:29:50 AST.

The next message on the good box was "Dec 31 19:59:59 localhost kernel: Clock: inserting leap second 23:59:60 UTC." This message was not on the box that froze.

When I stat'ed the recording, it was last modified at 19:59:59.431 -0400.

Re:I Second That

[athakur999]

My Mythbuntu-based HTPC also froze up last night.

This is what my /var/log/messages file looks like:
Dec 31 16:03:45 puppet -- MARK --
Dec 31 16:23:45 puppet -- MARK --
Dec 31 16:43:45 puppet -- MARK --
Dec 31 17:03:45 puppet -- MARK --
Dec 31 17:23:45 puppet -- MARK --
Dec 31 17:43:45 puppet -- MARK --
(... below is when I noticed the box was hung and restarted it ...)
Jan 1 14:02:31 puppet syslogd 1.5.0#2ubuntu6: restart.
Jan 1 14:02:31 puppet kernel: Inspecting /boot/System.map-2.6.27-9-generic

Every 20 minutes, I get those "-- MARK --" messages and the last one is at 5:43PM local time which would be 11:43PM UTC (also my system clock is set to UTC, not local time). The next "-- MARK --" should have been at 12:03AM UTC, so there's a good chance the leap second messed something up.

test

[wizardforce]

Could this be a coincidence or was there some 2008/2009 rollover issue going on here?

set the system time back a few mins before the crash occured and see if your server crashes again... otherwise it's idle speculation

Re:test

[$RANDOMLUSER]

First good idea in this whole discussion. Don't forget the hardware clock as well.

Re:test

[LunarCrisis]

You guys aren't thinking far enough. A time machine is clearly in order!

Re:Google searching

[moniker127]

As far as I've been able to tell, this means that there have been a lot of requests from your subnet, enough that it looks like some sort of bot.

driver

[TheSHAD0W]

The Zune crash was due to a specific hardware driver. Perhaps you also have an unusual hardware driver on your setup that was affected?

Ubuntu 8.10 - MythTV Crash

[Anonymous Coward]

I was watching the new years London celebrations on my Ubuntu 8.10 MythTV box. With 10 seconds to go to midnight, it crashed. Missed the start of the fireworks.

I think it may have happened around midnight before, so not necessarily an New Year problem.

RiteAid pharmacy y2k09 bug

[Anonymous Coward]

On 12/30/08, I submitted a request with my pharmacy to refill a prescription to pick up on 12/31/08, and received the following email, verbatim:
 

Your Rite Aid prescription confirmation
Greetings from the riteaidonlinestore.com pharmacy,

Thank you for choosing to refill your Rite Aid prescription(s) online at the riteaidonlinestore.com pharmacy.
The following refills have been sent to the Rite Aid store that you selected, along with your preferred pick-up date and time:

Patient Name: ********
    Rx ******** ********
    Rx ******** ********

Rite Aid Store Location:
    ********
    ********, ********
    ********
    ********

Pick-up Date and Time:
    Thursday December 31, 2009 at 3:00 pm

If you have any questions regarding your prescription, please contact your local Rite Aid directly at ********. Please note that you will need to pay for this prescription when you pick it up. If you have selected to self-pay for this medication, you will pay Rite Aid's price.

Thank you for visiting the riteaidonlinestore.com pharmacy. We invite you to visit us for your other prescription needs and great deals on nonprescription items. We look forward to assisting you!

Some things to note: I've got to wait until next christmas to pick up my drugs, and they were so concerned about patient privacy, they obscured all my contact information, prescription numbers and the pharmacy's phone numbers with asterisks. (I didn't do that myself!)

So, I wonder if their log files are full of java.lang.Exception logs today...

--ob

Mysteriously coincidental with this event..

[vorlich]

my cat hid under the bed at almost 25 seconds into the New Year. Right after he heard the first of the fireworks. However he did restart normally about 22 minutes later after a soothing saucer of milk. I wonder if ...

Re:Mysteriously coincidental with this event..

[tengu1sd]

my cat hid under the bed

Is that a cat-5 or cat-6? Is re-installing M.I.L.K. a normal procedure? I understand some models have patches available after a fix, is your model fixed?

Sandisk Sansa e260 4GB Media Player

[Anonymous Coward]

I had purchased a Sandisk Sansa e260 4GB Media Player for my father-in-law for x-mas and it worked on thru the 30th, but it wouldn't turn-on on the 31st all day. We finally managed to turn it on today. Interesting...

Crashy box crashes, you say?

[Yath]

it has not crashed more than 1 or 2 times in 2237 days of cumulative uptime

Apparently, you have pre-existing stability problems with this box. The fact that it crashed yet again yesterday should come as no great surprise.

Linux 2.6.21 hangs on leap seconds

[Anonymous Coward]

You didn't specify your kernel version, but if it was 2.6.21, you may have hit this:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux2.6.gita=commitdiffh=746976a301ac9c9aa10d7d42454f8d6cdad8ff2b

Thankfully this was a short-lived bug which only affected 2.6.21.

Another anecdote

[CustomDesigned]

I switched from Windows 95 to RedHat 6.2 many years ago, and except for reboots to upgrade the hardware (started with 200Mhz Pentium I w/ 384M and now have Dual Core w/ 2G) or OS (now on CentOS 5.2), it has crashed only twice - due to a defective USB2.0 card which I replaced.

We run LTSP so that the single server runs the entire family, using old '90s hardware for thin clients. We simply could not afford to run Windows (or Mac).

Re:nope

[ArcticFlood]

Fedora 8's end of life doesn't occur until January 7th, so it would still get timezone updates.

Re:Adding some data

[aputerguy]

Froze - couldn't ping or ssh or get console response. I know the time cuz last maillog entry was 18:59:40 and the clock (on my emacs session) said 18:59 at time of crash. Hardware is: ASUS P4P Rebooted without ever but required me to manually poweroff

Fedora 8 locked up here

[AZPolarBear]

My Fedora 8 system locked up after the leap second update was logged at 00:00 UT. I was my DHCP server, so the network went down.

Re:Fedora 8 locked up here

[drinkypoo]

I was my DHCP server, so the network went down.

I was my DHCP server too (I entered my DHCP responses on a hex keypad) but then I got dnsmasq, and now the computer hands out addresses for me.

Anyone With New Year's Toast Accidents

[david_craig]

My toast got burnt sometime between 9:59:40 EST (Eastern Standard Time in New South Wales, GMT +10:00) and 10:00:00 EST (GMT +10:00) on Jan 1, 2008 which remarkably corresponds to within at most 20 seconds of the New Year in GMT. I have been making toast with this same wetware non-stop for more than twenty six years and other than the occasional lapse in concentration while speaking on the phone, I have not burnt toast more than 1 or 2 times in 2237 days of cumulative toasting. Nothing other than background processes were running through my mind at the time of the burning. Could this be a coincidence or was there some 2008/2009 rollover issue going on here? Has anyone (other than Zune 30GB owners) noticed similar year-end issues while operating toasters or electronic devices?

Re:boxen!

[TaoPhoenix]

On Debian, RHEL, Centos & Boxen! On Irix, Solaris, Ibex & Vixen!

Nothing crashed on me -- madplayer hicked however

[billsf]

Madplayer hicked three times at about 0100 CET. I thought it might have been my RAID system I had just repaired. (There was a bad sas/sata controller.) This happened over about 20 seconds. I only use Unix/Unix-like systems and to the best of my knowledge there are no embedded MS devices in this house.

Unix/Linux, etc. handles things like this well. All time sync services like NTP, DCF-77, MSF, WWVB, GPS and the rest give fair warning. I personally are in favour of ditching 'leap seconds'. Time corrections would best be made day to day, the length of today being based on yesterday. That's better, but surely someone can think up the real solution?

BillSF

PS: Frequent updates to Java caused by US daylight saving time are pathetic.

     

Re:Time Mathematics and Microsoft

[ThePhilips]

Try once yourself to code conversion from "seconds since 1/1/1970 00:00:00" to any other user digestible presentation.

It's not as easy as it might seem.

Re:Adding some data

[sjames]

A bottle rocket hitting a transformer isn't out of the question. If it hits near the high voltage terminals it may briefly arc.

Re:Time Mathematics and Microsoft

[Purity Of Essence]

ANSI dates are counted from 1601-01-01 and were adopted by the American National Standards Institute for use with COBOL and other computer languages. This epoch is the beginning of the last 400-year cycle by which leap-years are calculated in the Gregorian calendar. The last year of this cycle is the only one divisible by 100 that is a leap-year, which was the year 2000, and which was followed by a new 400-year cycle beginning with 2001. 32-bit versions of the Microsoft Windows operating system count units of one hundred nanoseconds from this epoch

Wikipedia

NTP and the leap second

[Charles Dodgeson]

A surprising number of NTP servers didn't add the leap second correctly. On the mailing list for pool.ntp.org contributors, it was reported that at just after midnight UTC that about 158 servers in the pool (about about 2000) were reporting times that were around 1000ms off. A few hours later it was only 13 that were doing that.

My own (stratum 3) NTP server got confused and declared that it couldn't determine the correct time. Some of its sources were 1000ms off from others. Given enough time, NTP will sort itself out, but I intervened manually by ditching the upstream servers that hadn't gotten it right.

If enough NTP servers were temporarily in the state that mine was in (was so unsure of itself that it wouldn't serve time to clients) then I could imagine some process that tries to sync the time and fails because ntpdate doesn't return anything useful.

Re:NTP and the leap second

[MooUK]

Failing to update surely shouldn't be a matter of crashing, however. Network services do go down; it's something you plan to cope with.