Managing the Most Remote Data Center In the World

blackbearnh writes "Imagine that your data center was in the most geographically remote location in the world. Now imagine that you can only get to it 4 months of the year. Just for fun, add in some of the most extreme weather conditions in the world. That's the challenge that faces John Jacobsen, one of the people responsible for making sure that the data from the IceCube Neutrino Observatory makes it all the way from the South Pole to researchers across the world. In an interview recorded at OSCON, Jacobsen talks about the problems that he has to face (video), which includes (surprisingly) keeping the data center cool. If you're ever griped because you had to haul yourself across town in the middle of the night to fix a server crash, this interview should put things in perspective."

Comments

You don't need to go as far

[mseeger]

I don't envy someone with a job like that. It is already very difficult to serve locations less remote. E.g. to offer a SLA for a network which spans over 30 locations in the world, we have to make sure that spare parts arrive around the world within a well defined time frame. We don't want anything fancy: a week would be finde by us. But i haven't found any transportation company that guarantees delivery on site in 3rd world countries (big cities) within that time frame. All make exceptions like "customs", which doesn't help me.

CU, Martin

Re:You don't need to go as far

[mseeger]

Or just blow the budget and have a spare set of equipment at each location. When something dies, take out the replacement that is already there, then worry about shipping a new replacement.

According to the customer, this has been tried and failed... Unused spare equipment has a tendency to vanish.

CU, Martin

Re:Space probes

[dargaud]

What I find even more impressive is how NASA, ESA and others manages space probes I think, that's really extreme conditions in every way.

Antarctica can be meaner in several ways: - you don't have a direct line of communication with the rest of the world (space probes do). Hell, you can't even have a direct comm with geosync satellites. - water ! Take thin crystals of ice, add lots of wind and you end up with water deep inside even sealed boxes; hence shorts and very quick rusting of components. - temperature changes. In space you surround your satellite with some heat conductive sheets and the temp basically never changes (unless you go into the earth shadow). In antarctica you can have -80C in winter, -10 in summer. To say nothing that the cold has unforeseen effects on materials (dielectric changes, materials becoming brittle...) - unstable power: the power comes from big diesel generators and is shared between experiments, people, etc... It goes out, the temp of the room where your computer is falls to -60 in 15 minutes. Power comes back, computer tries to boot. Bye, bye hard drive. - budget: experiments for Antarctica have much less than 1/10 the budget of equivalent space experiments. And most of it is eaten by logistics. So you end up with standard computers and a few hack and a guy standing nearby (me) to kick it if necessary.