Slashdot Mirror


Managing Servers In the Frigid Cold

1sockchuck writes "Some data centers are kept as chilly as meat lockers. But IT operations in colder regions face challenges in managing conditions — hence Facebook's to use environmentally controlled trucks to make deliveries to its new data center in Sweden, which is located on the edge of the Arctic Circle. The problem is the temperature change in transporting gear. 'A rapid rate of change (in temperature) can create condensation on the electronics, and that's no good,' said Facebook's Frank Frankovsky."

122 comments

  1. From Minnesota here by onyxruby · · Score: 4, Insightful

    This isn't anything new, anytime you take something from the extreme cold and bring it inside you risk condensation. This is usually dealt with by simply letting something sit at room temperature for several hours before powering it on.

    In the middle of January if you take a freezing cold delivery and power it on right away and fry your new (XXXXXX) you deserve to void your warranty. There is no excuse for stupidity. Why is this on slashdot as news?

    1. Re:From Minnesota here by Anonymous Coward · · Score: 0

      Well, because it's obviously tonnes cheaper to use heated delivery trucks so the whole data center can be powered on immediately. You know, you just back the truck into the warehouse and plug it in. Or someone just spent a lot of money preventing something that would have nearly corrected itself by the time the truck was unloaded.

    2. Re:From Minnesota here by Pieroxy · · Score: 3, Insightful

      There is no excuse for stupidity. Why is this on slashdot as news?

      Because most slashdotters aren't from extremely cold areas. What seems like obvious to you isn't even considered by most, just because most aren't subject to the same conditions as you are.

      Let me ask you another question in return. I think I deserve an answer, since I answered yours: Why do you think everybody should know what you think is obvious?

    3. Re:From Minnesota here by aepurniet · · Score: 1

      not everyone lives in that type of climate, i havent RTFA, but i assume there are some more challenges that people dont usually think about. seems like it would be interesting if your into data centers.

    4. Re:From Minnesota here by vlm · · Score: 3, Informative

      Living slightly to the east, yet just as cold in winter, the strategy is to leave the gear sealed in the box while you prep the racks and wiring and gather tools. Its really not that complicated.
      You don't have to wait until the gear reaches room temperature, merely gets above the interior air dew point, which I assure you is very low in the winter.

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    5. Re:From Minnesota here by prakslash · · Score: 4, Funny

      Why is this on slashdot as news?

      So you can get a taste of the luxuries you can afford after a 100 billion dollar IPO. Why wait a few hours before powering up you equipment when you can transfer it using expensive, climate-controlled trucks. At Facebook, even lifeless plastic and metal rides in style on the gravy train.

    6. Re:From Minnesota here by Anonymous Coward · · Score: 0

      remember in the good old days when renting a movie and a VHS player, it had a sticker on it saying leave it at room temperature for 30minutes before use

    7. Re:From Minnesota here by vlm · · Score: 1

      Let me ask you another question in return. I think I deserve an answer, since I answered yours: Why do you think everybody should know what you think is obvious?

      Temp is a bell curve, what we do for a month every year is maybe one day every other year for you.

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    8. Re:From Minnesota here by Joce640k · · Score: 2

      Have they not heard of an invention called "plastic bags"?

      I hear they do wonders against condensation.

      --
      No sig today...
    9. Re:From Minnesota here by afidel · · Score: 1

      Yep, we never put the tapes from the delivery truck directly into the tape library during the winter for exactly this reason. The tapes get loaded on the truck early in the morning when it's often in the teens outside, taking them from that to the 80 degree datacenter is bad enough, putting them in the 100 degree environment of the tape library without acclimating them would be foolish.

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    10. Re:From Minnesota here by afidel · · Score: 1

      Why would the dew point be any different during the winter, don't you have humidity control in your datacenter?

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    11. Re:From Minnesota here by timeOday · · Score: 1

      This is usually dealt with by simply letting something sit at room temperature for several hours before powering it on... There is no excuse for stupidity. Why is this on slashdot as news?

      Maybe because what you said is NOT the same as what Facebook is doing? If simply letting the servers warm up gradually at the destination works fine, why are the spending extra money on heated delivery trucks?

    12. Re:From Minnesota here by gl4ss · · Score: 3, Informative

      well, it's not particularly interesting article I'm afraid, the interesting part is that they mix incoming fresh air with already circulated heated air instead of having isolated heat exchanger arrangement, other than that it seems like a fairly traditional datacenter - no ssd devices dipped in epoxy sitting outside in ice or crazy stuff like that.

      "woot heated trucks".. well duh, not everything likes to be frozen. ever had partially frozen milk on school lunch? it sucks and we walked to school. both ways. or bicycled(on ice). or used kicksleds(if there wasn't sand on the route).

      --
      world was created 5 seconds before this post as it is.
    13. Re:From Minnesota here by Anonymous Coward · · Score: 0

      RUNNING machines in plastic bags? I'm not sure you understood the problem domain.

    14. Re:From Minnesota here by vlm · · Score: 4, Interesting

      Oh I thought of another one. The problem has nothing to do with temperature. The problem is when the indoor/outdoor/dew points intersect which happens all the time, not just when its cold.

      One excruciatingly humid summer day I was hauling around a protocol analyzer worth about as much as my car, and it cold soaked in front of the car air conditioner duct cooling itself to 40 degrees or whatever the AC output is, then it was dripping condensed water as I carried it into the customer premises, an un-airconditioned factory floor. So I'm sitting there doing nothing and explaining to the customer how I have to do nothing, until the test set dries off because its too cold (customer VP looks out window at blue sky 110 degree day). Yes that was an unpleasant meeting.

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    15. Re:From Minnesota here by LordLimecat · · Score: 2

      Because they dont understand common sense solutions to simple problems, and would rather throw money at the problem?

    16. Re:From Minnesota here by Baloroth · · Score: 1

      Have you ever seen a glass of ice water that showed condensation? Bingo, you should be able to figure out who powering on freezing cold electronics is a bad idea. I'm pretty sure everyone working at a data center is familiar with the idea of condensation, certainly anyone working in any situation where it would be an actual issue.

      --
      "None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
    17. Re:From Minnesota here by vlm · · Score: 1

      It never works that well. I've seen the logs/graphs. Dew point skirts the bottom of minimal acceptable in winter (around 40 F, from memory), and skirts the top of maximum acceptable in summer (around 60 F, from memory). I suppose it depends on your center. I'm thinking specifically of a private couple acre financial services DC in the upper midwest, although the telecom data centers I've been in are about the same.

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    18. Re:From Minnesota here by oodaloop · · Score: 3, Interesting

      In the Marine Corps, I had some cold weather training before a deployment to Norway. We were instructed to leave our rifles outside of our tents. Otherwise, they would accumulate condensation inside the barrels, which would then freeze when you walk outside. Lots of fun stuff like that.

      --
      Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
    19. Re:From Minnesota here by imp7 · · Score: 1

      If I had mod points I would be giving them to you. I'm from MN too and it's common knowledge to seasoned sysadmin's here. I find it sad that facebook has to waste money and energy.

    20. Re:From Minnesota here by hawguy · · Score: 3, Interesting

      RUNNING machines in plastic bags? I'm not sure you understood the problem domain.

      You don't understand the solution.

      Put the equipment into plastic bags before loading it on the truck (if it's new equipment, it's probably already wrapped in plastic.

      Then when you unload it in the warm datacenter, the moisture condenses on the outside of the bag instead of inside your server.

      Once the server is up to room temperature, take it out of the bag, rack it and plug it in, and you're good to go. (Note that in warm humid states, you can have the opposite problem - the cold server is taken from the 65 degree datacenter out to the 95 degree and humid outside air and moisture condenses on it. The plastic bag works here too.)

      You don't need to go to Sweden to experience cold temperatures - many datacenters throughout the USA experience temperatures cold enough to cause condensation problems for at least part of the year. The plastic also helps protect equipment that's exposed to moisture that condenses in clouds and falls to the ground (i.e. rain) as it's transferred from the truck to the facility. A problem that Facebook will discover once they open their first datacenter in a rainforest and perhaps they can invent some self deploying canopy that shields the equipment from this mysterious moisture from the sky since they don't seem to like the low-tech plastic bag solution.

    21. Re:From Minnesota here by petermgreen · · Score: 4, Insightful

      Even if you do have the ability to artificially raise or lower the humidity that doesn't mean it makes sense to keep it at the same level all year round.

      Most electronics is specified for quite a wide range of relative humidity. Usually 5% to 95% or so.

      In winter you want low relative humidity to reduce the risk of condensation on stuff brought in from outside (yeah you try to seal stuff and let it warm up before unwrapping but mistakes and emergencies happen). It's also cheap to achieve low relative humidity due to the low outside air temperature (for a given absoloute humidity relative humidity goes down as temperature goes up).

      In summer humidity doesn't matter so much since stuff brought in from outside will be warm. It's also likely to be more expensive to achieve low relative humidity since it involves active dehumidification (which is achieved by cooling the air to the point where the water condenses out)

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    22. Re:From Minnesota here by Anonymous Coward · · Score: 0

      Exactly what i was about to write. When we get servers shipped wintertime we just unpack them the day before we are about to use them. Its not a problem unless you dont know what you are doing.

    23. Re:From Minnesota here by Blymie · · Score: 1

      Partially frozen chocolate milk is AWESOME when she's hot outside...

    24. Re:From Minnesota here by Anonymous Coward · · Score: 0

      There is no excuse for stupidity. Why is this on slashdot as news?

      stupidity as slashdot news. And this is surprising to you?

      Last week i literally read page after page of idiocy, wrong answers, and what was clearly ignorant propaganda. I actually tried to find valid answers and logical discourse. I would estimate I found *maybe* 5% of the comments to be worth reading. Sorry, but slashdot is increasingly the definition of mass stupidity and idiocy.

      I can honestly say, people who put any faith in the comments of slashdot are less intelligent for it. There are exceptionally few exceptions.

    25. Re:From Minnesota here by DerekLyons · · Score: 2

      This isn't anything new, anytime you take something from the extreme cold and bring it inside you risk condensation. This is usually dealt with by simply letting something sit at room temperature for several hours before powering it on.

      True. But what you're forgetting (generously assuming you knew it in the first place) is that condensation isn't the only issue. Servers are made of a variety of materials - all of which expand and contract with temperature at different rates. Extreme cold can actually physically damage equipment by pulling pins from sockets, cracking PCB traces, etc... etc... That damage (obviously) can't be fixed just by letting it sit in at room temperature for several hours.
       
      Just a bit of trivia since Mars landers and rovers are in the news today... it was the lenses in the PanCams that set the lower temperature bound for the Spirit and Opportunity rovers - the lens mounts shrunk faster than the lenses, and eventually would shrink enough to crack the lenses.

    26. Re:From Minnesota here by onyxruby · · Score: 1, Insightful

      I can certainly understand that people from warmer climates won't understand how to drive during an ice storm or how to recover from a skid. These are things that come from experience with exposure to a certain climate.

      I will answer your question as I wasn't trolling. I think everyone should know consider this obvious because condensation is elementary physics. When you consider that I am on a technology site with a notable science influence it's the kind of thing I just expect that people would know.

    27. Re:From Minnesota here by afidel · · Score: 3, Interesting

      Low RH is bad because you get static buildup. Sure we've got anti-static wax on the floor and all the cages are grounded, but I still don't want to risk frying a computer because I couldn't keep RH in the right range. Also low RH is easy to achieve since CRAC units due it by their nature =) Even in the middle of summer you have to run humidifiers.

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    28. Re:From Minnesota here by onyxruby · · Score: 1

      Very true. I had a dell repair tech keep my laptop (unauthorized) for repair in his car over the weekend once in a cold snap in the winter. He froze and shattered the screen. I ended up with a new laptop from Dell when all was said and done.

      Early in my career I worked for Polaris and used to arrange deliveries of computers to places in the Arctic circle. We took a number of precautions keep the equipment from getting destroyed by the extreme cold. We never kept things in a heated container though and I was shipping computers to places like Nome Alaska. We never shipped anything in a heated truck though.

    29. Re:From Minnesota here by CanHasDIY · · Score: 1

      Let me ask you another question in return. I think I deserve an answer, since I answered yours: Why do you think everybody should know what you think is obvious?

      3rd grade science class

      --
      An enigma, wrapped in a riddle, shrouded in bacon and cheese
    30. Re:From Minnesota here by Shatrat · · Score: 1

      Just because the equipment says 5%-95% doesn't mean it's a good idea. That same equipment is probably also rated to 50c+ temperature.
      We keep all our facilities around 40% rh. Too high promotes condensation and too low promotes static discharge.
      All decent HVAC units can control temperature by condensing or evaporating moisture.

      --
      09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
    31. Re:From Minnesota here by Anonymous Coward · · Score: 1

      This isn't anything new, anytime you take something from the extreme cold and bring it inside you risk condensation. This is usually dealt with by simply letting something sit at room temperature for several hours before powering it on.

      In the middle of January if you take a freezing cold delivery and power it on right away and fry your new (XXXXXX) you deserve to void your warranty. There is no excuse for stupidity. Why is this on slashdot as news?

      Not comparable. Bicycles and cars are custom made for the Swedish market (i.e. you have to custom make a lot of parts and replace all nuts and bolts), because the humidity fluctuations (and to a lesser degree temperature fluctuations) make cars and bicycles made for "normal" markets corrode really fast. E.g. a bicycle made for any other European market would only survive a few months use in Sweden, it would corrode the fastest during spring and autumn, or mild winters, in Southern Sweden, when it is not extremely cold (+6 - +30C). This problem is least worse during Swedish winters (the colder the better) and least worse in Northern Sweden, but even in Northern Sweden (like Luleå) it is still worse then in any other place in any other country that I've heard of.

      The corrosion problem is also present in buildings with a high intake of outdoor air, even if they are kept warm.

      Also, most electronic circuits freeze to death within a few hour by -5C; Luleå stay below that temperature most parts of the year. Any remotely advanced electronic circuits is dead by -20 - -30C, unless isolated from the cold, common temperatures during winter in N. Sweden.

    32. Re:From Minnesota here by Anonymous Coward · · Score: 0

      Why do you think everybody should know what you think is obvious?

      Maybe he thought that because it's a matter of having a grade school education? And that even people from warm climates are familiar with the concept of condensation first hand if they've ever had an iced drink in a glass?

      If you're not just trolling, then it's astonishing that you actually think a rudimentary awareness of condensation should be considered atypical.

    33. Re:From Minnesota here by Pf0tzenpfritz · · Score: 3, Informative

      That's not the point. Every time you move something from a cold place in to a warmer one (higher humidity in the air implicit, since higher temperature means higher point of saturation) condensation occurs, as the air near the cold item cools down and "drops" dew on the cold surface. If the latter is intransparent, like server rails, backplanes or transformator cores/coils the condensed water will collect there. Basically, condensation always occurs where temperature difference exist and it always happens at the coldest surface in the room. (Hence all the trouble with moisture and poorly insulated walls in colder regions.) Now a truckload of servers is basically one large thermal buffer. Move it from arctic cold -supposed the machinery had time to adapt to outside temperatures- to room temperature and you will find a lot of water condensing. We're talking about tons of material -with a lot of surface- that will take hours to warm up.

      --
      Oh, the beautiful gloss of greality!
    34. Re:From Minnesota here by AK+Marc · · Score: 1

      Move faster than condensation. The cold places don't have as much moisture inside (cold air heated is dry), so if you plug in cold equipment quickly, it'll warm up before it gets condensed on.

      Also, if it's that cold outside, then use filtered outside air for room colling, keep the room at outside air temp, and you won't have an issue.

    35. Re:From Minnesota here by AK+Marc · · Score: 2

      Early in my career I worked for Polaris and used to arrange deliveries of computers to places in the Arctic circle. We took a number of precautions keep the equipment from getting destroyed by the extreme cold. We never kept things in a heated container though and I was shipping computers to places like Nome Alaska. We never shipped anything in a heated truck though.

      Nome isn't in the Arctic circle. Call us when you go someplace interesting, like going on a Polaris from Faribanks to Anaktuvuk Pass with a server strapped on the back (no, I haven't. Alaska Air has multiple destinations above the arctic circle, and I've been to all of them, on a plane, I try to not go outside when it's cold).

      My favorite was discussing wind chill with regards to electronic equipment, or finding anything rated to work below -40, as most goes to -40, so anything rated for colder requires non-standard boards, caps, chips, etc., and there's just so little need that nobody does it. Best I got for common things was "buy enough and we'll put in the warranty that cold temperatures will not void the warranty, so if it fails, we'll replace it, but we won't guarantee it will work." You'd be surprised how many things just won't power up colder than -60, I can only assume issues with the power supplies not liking the cold, but I'm not going to spend my time at -60 troubleshooting the exact reason. Interestingly, some will work once booted, if you can boot it, so you take it inside, warm it up, then run it outside and power it up as fast as possible. Oh, and it wouldn't reboot, so if it did have an issue, it'd be off until May or so.

    36. Re:From Minnesota here by Anonymous Coward · · Score: 1

      Put the equipment into plastic bags before loading it on the truck (if it's new equipment, it's probably already wrapped in plastic.

      Then when you unload it in the warm datacenter, the moisture condenses on the outside of the bag instead of inside your server.

      Just be sure it's cold before you bag it. Otherwise, if you load it on the truck warm and bagged, it'll condense on the inside of the bag.

    37. Re:From Minnesota here by mortonda · · Score: 1

      The problem is the air inside the bag - it can have condensation issues too.

    38. Re:From Minnesota here by Anonymous Coward · · Score: 0

      You do realize that a substantial portion of the world only gets to freezing once in a blue moon. And I mean that literally most years they don't have any freezing temperatures at all.

    39. Re:From Minnesota here by onyxruby · · Score: 1

      We sent equipment to anyplace that had a Polaris dealership. Nome was a particular dealer that came to mind as I had to deal with them more than once. We certainly had dealerships in the Arctic circle as we were the effective equivalent of the local car dealership up there (and Canada).

      I'm not arguing the extreme cold is extremely hard on the equipment. Facebook isn't doing the oilfield type of conditions, they are shipping equipment down a highway to a heated data center which is very different from the conditions you describe.

      I have had to ship equipment by dogsled in the winter when we couldn't get a plane in. Motorized vehicles had to be electronically registered within 2 days of sale. I call that interesting - it was also more expensive than small plane delivery.

    40. Re:From Minnesota here by sumdumass · · Score: 1

      I'm not sure how much more expensive the climate controlled trucks are. I think it is probably trivial.

      I used to run a refer unit (yes, refrigeration units in trucks will heat too) and we were always picking up coiled steel and transporting it on the refer unit to control the climate until it reaches it's destination. The difference between out runs and flat beds making the same runs was less then 5 cents per mile to cover the diesel to run the refer units..

    41. Re:From Minnesota here by AK+Marc · · Score: 1

      We certainly had dealerships in the Arctic circle as we were the effective equivalent of the local car dealership up there (and Canada).

      Odd, a search for Polaris dealerships didn't show any above the arctic circle.

      https://maps.google.co.nz/maps?hl=en&client=opera&channel=suggest&ie=UTF-8&q=polaris++dealers+alaska&fb=1&gl=nz&hq=polaris+dealers&hnear=0x5400df9cc0aec01b:0xbcdb5e27a98adb35,Alaska,+USA&ei=lmwgUNruK4iIrAfB_oGgCw&ved=0CLcBELYD

      But then, places like AC sell ATVs without being "dealers", as far as I can tell. Though I looked them up and didn't see any Polaris there. Many would buy them in the cities (Anchorage) and have them shipped out on the annual barge.

      Facebook isn't doing the oilfield type of conditions, they are shipping equipment down a highway to a heated data center which is very different from the conditions you describe.

      Yeah, Facebook is doing the equivalent of wrapping their servers in a batttery-powered electric blanket for the trip from the car to the 5-star hotel. Oh woe is them.

    42. Re:From Minnesota here by grcumb · · Score: 1

      This isn't anything new, anytime you take something from the extreme cold and bring it inside you risk condensation.

      Having worked at a commercial ISP in the Arctic for 3 years, I can tell you that it's a little different from Minnesota. A few hours isn't nearly enough time to let them sit. Our standard was 24-48 hours in the room the equipment was going to occupy before we'd attach cables and power on. Radical heating and cooling also meant that we'd re-seat RAM, NICs and other cards before booting as a matter of course.

      And it's not just cold up there, it's also perfectly arid. Things get dry in southern Canada and the northern states in the Winter time, but not like they do in the high Arctic. It means that static discharge is a really big problem as well. I've seen sparks arc 2 or more inches from a hand to a door handle.

      --
      Crumb's Corollary: Never bring a knife to a bun fight.
    43. Re:From Minnesota here by onyxruby · · Score: 1

      Your not looking very hard. I'll start with a little town called Barrow Alaska. I think we can both agree that it is in the Arctic Circle. They also have a Polaris dealership.

      Eskimos Inc Polaris
      PO Box 1273
      Barrow , AK 99723
      907-852-8000

      If you really want you can look things up directly on Polaris's website. As I said I worked there, I dealt with the dealerships for a couple of years. They also have dealerships in arctic circle in Canada. They have dealerships that operate under everything from Harley Davidson motorcycle shops to general stores. While you always had a limited amount of dealer to dealer sales, we sold directly to dealerships in almost every town northern Alaska and Canada.

      I'm not quite sure what point your trying to prove here.

    44. Re:From Minnesota here by Firethorn · · Score: 1

      Also, most electronic circuits freeze to death within a few hour by -5C; Luleå stay below that temperature most parts of the year. Any remotely advanced electronic circuits is dead by -20 - -30C, unless isolated from the cold, common temperatures during winter in N. Sweden.

      Fairbanks, AK here. We get plenty of -30, and we find that most electronics that survive it just fine. Vehicles are plugged in more to keep the lubricants, coolant fluid within their operating range, and keep the battery a bit warmer so it can start stuff. Even servers can get that cold when not operating; just let them get up to operating temperature before plugging them in.

      --
      I don't read AC A human right
    45. Re:From Minnesota here by AK+Marc · · Score: 1

      They didn't show up on a google search, and on the Polaris website, searching for Barrow gets a result in ME, and Barrow , AK gets no results. And yes, I didn't look too hard. I just gave it a little look and didn't see anything popping up on Google and such, and I did go to Polaris's web site, but the flash version required didn't match what I have installed, though the link you gave went to a page that embedded a Google map frame, which worked better. Closest I've been to the arctic circle in Canada is driving through to get to Alaska. I found it amusing how many people have Harleys they drive for 2 months out of the year.

    46. Re:From Minnesota here by onyxruby · · Score: 1

      Their website is terrible, it always has been. Many dealerships often won't even have "Polaris" in their name which throws things off even more. Small motor dealers often carry multiple lines and it's just the same for Honda or the other manufactures.

      I just happened to recall the northern Alaska / Canadian dealerships in particular with all the stuff I had to go through compared to a typical place. For many of those dealerships at the time it was literally the first time they had ever seen a computer. Needless to say it was a good learning experience with lessons that served me years later for places like oil rigs in the Indian Ocean and strip mines in the outback in Australia.

      What I found amusing was how many Harley dealers bought the first year Victory motorcycles the year they came out. They only made 2000 the first year and each dealership was only allocated one a piece as a rule of thumb. We had over 600 Harley dealerships we counted where the dealer owner purchased the bike personally before it ever hit the sales floor.

    47. Re:From Minnesota here by Pieroxy · · Score: 1

      I'm pretty sure everyone working at a data center is familiar with the idea of condensation

      Did I ever said otherwise? You actually reinforce my point: Most slashdotters DON'T work in a datacenter.

    48. Re:From Minnesota here by Pieroxy · · Score: 1

      Ok, my question was badly worded. Everyone knows about condensation. But then why would you expect most of us to think about it before powering a machine? I mean, unless we've tripped on that one once or twice?

    49. Re:From Minnesota here by AK+Marc · · Score: 1

      I replaced a failed tape drive in a warehouse in Alaska, and I pointed out to the shop guy why it failed. There was over one inch of dust collected in the bottom of the case. The silt-like dust/dirt would get into anything and everything.

      I'll never "get" the Harley. It's slower and more expensive than almost everything else out there, but then, from what I can tell, most of the people who buy Corvettes are old fat white guys who never even hit the speed limit.

    50. Re:From Minnesota here by Anonymous Coward · · Score: 0

      And those that do, much like myself, know that a properlly designed Datacenter has water-under-floor sensors under cooling equipment for this exact reason. Condensation!

    51. Re:From Minnesota here by CAIMLAS · · Score: 1

      I'm from South Dakota, but just got back last month from a year's internment in California - you know, where most tech companies and many data centers are. It's got a similar climate to Texas, supposedly, where there is also a lot of tech. :P

      It gets down to -20F very normally out here during the winter. With temperatures that low, there is very little moisture in the air indoors, either. So there isn't going to be all that much condensation. The best thing to do is leave the server in the box for several hours in a warm room. The ambient humidity is so low that not much condensation will form; likewise, the air is so dry that anything that does form will evaporate very quickly. A general policy I've made is any systems coming in during the winter need to sit in the box for at least 3 hours so they warm up slowly, and the machines have to sit for at least 5 hours before they get powered up.

      At these low temperatures, your biggest concern is ESD. It can be bad. Carpet, linoleum, raised floors, anything. It can get uncomfortable. Don't wear any polyester or wool, you will regret the shocks, and they can be jarringly painful and regular.

      This is news for people in places like California, where most people may never have seen a 'real' snowfall. The concept that not only does it get 'cold' outdoors but it also gets significantly colder than the inside of a walk-in freezer is mind blowing to many of these people.

      --
      ~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
    52. Re:From Minnesota here by CAIMLAS · · Score: 1

      That will only work if the air inside the bag has near-zero humidity. Condensate will form inside the bag just as easily as outside it.

      --
      ~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
    53. Re:From Minnesota here by CanHasDIY · · Score: 1

      Ok, my question was badly worded. Everyone knows about condensation. But then why would you expect most of us to think about it before powering a machine? I mean, unless we've tripped on that one once or twice?

      Logic.

      - It's common knowledge that powering on wet electronics is a great way to let out the magic smoke, right? Right.

      - It's also common knowledge that condensation forms on cold/warm items entering a warm/cold environment (respectively), right? Right.

      Therefore, knowing that electronics which are brought from a cold environment into a warm environment are going to form condensation, and that moisture is bad for powered electronics, the logical assumption is that plugging them in whilst still covered in condensation would be inadvisable, no?

      Ok, so maybe if you've lived your whole life in a tropical zone where the temperature variation is seldom more than a couple degrees (and no one, apparently, owns an air conditioner)... or perhaps if you're in your 80's or 90's and have never actually been around all that many electronics, you get a pass. For everyone else, however, I consider that to be basic knowledge.



      To paraphrase XKCD - Logic: It works, bitches!

      --
      An enigma, wrapped in a riddle, shrouded in bacon and cheese
    54. Re:From Minnesota here by Pieroxy · · Score: 1

      Your view of the world is amazing. I wish you luck with your endeavors.

    55. Re:From Minnesota here by CanHasDIY · · Score: 1

      Your view of the world is amazing. I wish you luck with your endeavors.

      Huh, never had it called that before...

      I appreciate your consideration, and return the good wishes.

      --
      An enigma, wrapped in a riddle, shrouded in bacon and cheese
  2. Try -68 F Room Temperature by Isbjorn · · Score: 5, Insightful

    This is nothing--years ago I deployed PCs at Alaskan oilfield installations. Extreme cold makes everything brittle, kept having problems with things like cracked motherboards, just from setting the PC on a desk.

  3. Suck it up, pussies. by gl4ss · · Score: 1, Interesting

    Sincerely, Finland.

    Been pretty shitty weather all summer here. Oh and I remember vividly when we were kids and we were coming down from a family trip in the winter, we couldn't play the new games we had bought before the next morning since the 8mhz bugger wouldn't boot until the "computer room"(porch thingy, badly insulated) heated up. And many many times we were playing games with our winter jackets on, maybe our parents were trying to discourage from being such nerds but they failed.

    --
    world was created 5 seconds before this post as it is.
    1. Re:Suck it up, pussies. by sconeu · · Score: 1

      You're missing the "now get off of my lawn" part of your post.

      --
      General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
    2. Re:Suck it up, pussies. by gl4ss · · Score: 1

      You're missing the "now get off of my lawn" part of your post.

      I don't have a lawn :(.. but during winter, I'm very happy to live in an apartment building and not a house with my own driveway. I hate shoveling ice.

      --
      world was created 5 seconds before this post as it is.
    3. Re:Suck it up, pussies. by sjames · · Score: 1

      Lawn, tundra, whatever :-)

  4. Insulation? by Anonymous Coward · · Score: 1

    Why would you need an "environmentally controlled truck"? What about just using some basic insulation? Shipping in cardboard boxes would slow the temperature change near the electronics enough to prevent condensation.

    1. Re:Insulation? by jones_supa · · Score: 1

      I agree. You don't need environmentally controlled trucks for packaged basic electronics.

    2. Re:Insulation? by Firethorn · · Score: 1

      Basic insulation like cardboard isn't going to cover it when you're looking at a 100F temperature difference between your server room and the outside, and you're possibly looking at a week's worth of shipping time.

      A 'temperature controlled' truck doesn't even have to turn on said features until like 24-48 hours before delivery, but it's still useful.

      I unloaded trucks in the wintertime in Nebraska. Nowhere near the arctic circle, but the cold seeped into said vehicles. Even with a hot air blower in the store pointed into the truck, with a sort of rubber seal locking out the outside air, I'd be freezing even as I'm tossing boxes as fast as I can(exercise). All the mass involved in a filled semi, the cold 'seeped' into it and acted to cool the air quite well. At that most of the stuff I handled was nowhere near as dense as computer equipment.

      You want the equipment to warm up quickly? You're going to need to spread it out over quite a bit of area. A heated truck isn't that much more expensive - many of them will scavenge the diesel's exhaust to heat for free, and in any case with the touch of extra insulation on such trucks it doesn't take that much more fuel. As a bonus, it allows you to use more of your expensive data center as data center, not equipment warming area. Think of the heated truck as less of a moving extravagance than a short term rental of a warming area that also allows you to use your expensive equipment faster, and saves a bit of wear/tear with cooling/heating cycles.

      --
      I don't read AC A human right
  5. Mineral Oil by Anonymous Coward · · Score: 0

    Transport all non-moving parts (everything except hard drives, fans, optical drives) in mineral oil. Completely avoids condensation problem.

    1. Re:Mineral Oil by Anonymous Coward · · Score: 0

      And completely causes a hazardous materials problem.

  6. Re:Try -68 F Room Temperature by vlm · · Score: 3, Interesting

    Cold drive bearings don't want to spin up / SMART fail from drive motor overcurrent.
    Happens to cooling fans too. Fan can't spin so equipment overheats.
    I've never knowingly had a voice coil bearing seize up, which is interesting because its probably the lowest power actuator in the system yet probably the highest precision / smallest tolerances.

    --
    "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
  7. Heh by Neil+Blender · · Score: 1

    Back in the day, I had to go to my data center when it was around 100 degrees out side so I was of course in shorts, t-shirt and sandals. I was there for 18 hours. Temperature inside was like 50 degrees. Yeah, that doesn't seem cold, but after 18 hours I felt like I had hypothermia.

    1. Re:Heh by Antipater · · Score: 2

      And it was uphill both ways to get there, right?

      --
      Everything is better with chainsaws.
    2. Re:Heh by Neil+Blender · · Score: 1

      It was twice as far uphill on the way, and three times as far on the way back.

      Seriously though, it wasn't so much the temperature, it was getting blasted constantly the by the air conditioners. Wind chill, I guess.

    3. Re:Heh by mooingyak · · Score: 1

      this:

      uphill both ways

      and this:

      to get there

      don't quite mesh.

      --
      William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
  8. Silica Gel Desiccant Packs by Anonymous Coward · · Score: 1

    http://www.lpsind.com/SilicaGelDesiccant.htm

    1. Re:Silica Gel Desiccant Packs by Anonymous Coward · · Score: 0

      Do Not Eat!

  9. Re:Fuck you by Anonymous Coward · · Score: 0

    But Steve obviously does not work there anymore, which makes it again a possibility that everyone there actually is homosexual.

  10. Re:Fuck you by Pieroxy · · Score: 0

    But Steve obviously does not work there anymore, which makes it again a possibility that everyone there actually is homosexual.

    While highly unlikely, given the sheer number of people working there, I have to say it is a possibility. Very very slim chances but hey, if that makes you happy...

  11. Sounds like someone is tripping by Anonymous Coward · · Score: 0

    Northern Sweden is hardly a place for "extreme cold" - transporting equipment like that is simply waste of resources, just as many posters have already pointed out. Let the hardware warm up before unpacking and no condensation.

    The main problem compared to hot climate data centers is the climate difference between summer and winter. During the summer, the weather can be almost tropical, while in winter it gets cold. This puts some pressure on cooling equipment. Cold in itself is not an big deal, although it helps a lot in cooling equipment emergencies when plenty of cold air is available - care has to be taken to avoid heat shock if raw winter air is used for cooling.

  12. In Canada, Nokia DSLAM's Overheat! by N1zaam · · Score: 4, Interesting

    A few years back while doing Tier 2 level support for a major Canadian telco, I started seeing overheating alarms from some Nokia DSLAM's. The odd thing was that it was -40C outside at the time. It turns out the fan's on these DSLAM's froze solid and the devices thought they where overheating and throwing alams left right and centre. We had to put a tarp over them with a heater during the winter to make sure they kept going.

    1. Re:In Canada, Nokia DSLAM's Overheat! by mooingyak · · Score: 2, Informative

      The odd thing was that it was -40C

      Random tidbit: at -40, you can just say -40. Kelvin doesn't go negative, and -40F == -40C. Unless you're using some other scale, but that should cover most of the cases.

      --
      William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
    2. Re:In Canada, Nokia DSLAM's Overheat! by j_sp_r · · Score: 2

      Not including units should be a capital offence. don't encourage it.

    3. Re:In Canada, Nokia DSLAM's Overheat! by AK+Marc · · Score: 1

      I keep my servers at -40 Rankine

    4. Re:In Canada, Nokia DSLAM's Overheat! by mooingyak · · Score: 1

      Rankine don't play -40.

      --
      William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
  13. TIL by Anonymous Coward · · Score: 0

    Today I learned that a rapid rate of change (in temperature) can create condensation on the electronics, and that's no good.

    Thanks slashdot! I knew I was really smart before, but now I feel totally smart! lol wut? I'm going to head over to slashdot B I to get the latest news on business intelligence so that I can maximize my status quo.

  14. Obvious answer is obvious by cod3r_ · · Score: 0

    Move to Texas.

  15. Re:Fuck you by bbbaldie · · Score: 0, Offtopic

    It wouldn't take long for a bundle of twigs to burn in hell...

  16. Wild climate in N. Sweden. by Alkonaut · · Score: 2

    Luleå has to have one of the most extreme temperature ranges anywhere. Summer temp is quite consistently 15-20C with occasional peaks of 30C and winter temp is zero to 40C below. So the range is nearly 90C (130F)! This of course seasonal variation and not "rapid change" so data centers should not be affected by this. The fastest changes there are probably in winter when the temperature in rare cases can go from -40 (and zero humidity) to zero (and damp) in a day or two. That kind of change, especially the other way round, could mean trouble (condensation in air in/outlets etc.)

    In fact, if google just wanted cold/dry climate, there has to be better locations. Northern sweden is mild, and has quite warm summers. Arctic inland climate further from the gulf stream atlantic would be more logical. Border between Russia and Finland for example. But there are probably logistical reasons (huge cargo airport, good port, good roads, railroads, lots of good technical people, ridiculous backbone connection) that placed the datacenter there.

    1. Re:Wild climate in N. Sweden. by Yvan256 · · Score: 1

      Canada has similar ranges. We got above 35C a few days ago and it went below -30C last winter.

      Also, from -40C to 30C is a range of 70, not 90.

    2. Re:Wild climate in N. Sweden. by afidel · · Score: 1

      That's not that bad, here in NE Ohio we've had from 45C to -40C at the extremes.

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    3. Re:Wild climate in N. Sweden. by Anonymous Coward · · Score: 0

      um... if it's from -40 to +30, then the range is 70, not 90. seeing the F range, I assume it's a typo and you meant 70.

    4. Re:Wild climate in N. Sweden. by pipatron · · Score: 1

      Not to mention good access to electricity, and a geologically stable area.

      --
      c++; /* this makes c bigger but returns the old value */
    5. Re:Wild climate in N. Sweden. by Alkonaut · · Score: 1

      Yeah typo. Confused by doing both C and F in my head. And of course inland climates have more extremes, like central us and a lot of russia. Very northernly inland locations should be more consistently cold (that is cool or cold depending on season€

    6. Re:Wild climate in N. Sweden. by AK+Marc · · Score: 1

      What, is this a contest? Go live in Fairbanks Alaska. +40C to -60C over a year.

    7. Re:Wild climate in N. Sweden. by Securityemo · · Score: 1

      Naah, now you're pushing it. -30 is the usual minimum range for the whole of lappland, maybe -35 when it's really cold. But yeah, the summers are hot, you can swim outside in the rivers if you really want to. As for the reasons for the location of the datacenter, you're forgetting Luleå technical university.

      --
      Emotions! In your brain!
    8. Re:Wild climate in N. Sweden. by mennucc1 · · Score: 1

      slow down, cowboy! According to wikipedia, in Fairbanks the average is +22C for highs in July and -28C for lows in January; the record temperatures in Fairbanks are high +37C and low -54C; but these records were not set in the same year; so I doubt that Fairbanks gets "+40C to -60C over a year". An even harsher climate, BTW, is found in Yakutsk: the average is +25C for highs in July and -41C for lows in January: this beats Fairbanks by 6C. The records are even more impressive: +38C for high , -64C for low. (This beats Fairbanks again). This is more impressive when you think that Yakutsk is home to ~ 200thousand people...

    9. Re:Wild climate in N. Sweden. by mennucc1 · · Score: 1

      ...by 6C.

      sorry that was supposed to be "16C"

  17. I still want Maria Sharapova. by Impy+the+Impiuos+Imp · · Score: 1

    Plastic bags. Look into it. You can even reuse them if you're so anal-retentively inclined.

    --
    (-1: Post disagrees with my already-settled worldview) is not a valid mod option.
  18. summer storms can drop the temperature 10-30F by Joe_Dragon · · Score: 1

    summer storms can drop the temperature 10-30F in 1 hour or less.

  19. Re:Fuck you by crutchy · · Score: 1

    being dense appears to come naturally for you

  20. Re:Fuck you by Anonymous Coward · · Score: 0

    Just like sucking 3 cocks at once for you.

  21. Re:Fuck you by camperdave · · Score: 0

    Steve did have children, and seemed happily married to a woman till the end.

    Um... So? What does that have to do with his sexual orientation? There are plenty of homosexuals with heterosexual marriages and children.

    --
    When our name is on the back of your car, we're behind you all the way!
  22. Re:Fuck you by Midnight_Falcon · · Score: 1
    I think these people are considered bisexual or otherwise queer. You're not homosexual until you loose interest in the opposite sex for sexual reasons entirely.

    Homo = one

  23. I worked for a company called Americold Logistics by Anonymous Coward · · Score: 0

    What they did was work with frozen foods (shipping them all over the country in frozen trucks, trains, etc./et al) - the warehouse was a "Blast Freezer" (so cold you'd die if you didn't have proper clothing, and not many skinny guys could last through the job - especially considering it was in Atlanta Georgia for the home office, which is hot)).

    The PC's that were on the forklifts ran DOS (this was back circa 1995-1996) & had INSULATION in a "cold weather hardened case" on them (these were used for your std. "parts picking inventory system" basically, using barcode scanning).

    That was so they did not TOTALLY "got nuts" due to said extreme cold (so cold it made your nosehair freeze), and keep running.

    * It used the actual heat generated by the machine itself to keep it from dying from what I understood.

    APK

    P.S.=> Those things kept on & on too, I never heard of one failing... then, from what I remember also, they never came out of the warehouse (unless they needed work)...apk

  24. Humidity more than Temperature by Roger+W+Moore · · Score: 3, Interesting

    Every time you move something from a cold place in to a warmer one (higher humidity in the air implicit, since higher temperature means higher point of saturation)

    Actually that is not implicit. Up here in the frozen wastes of central Alberta in the winter the indoor humidity drops to incredibly low values of 10-20% because there is no moisture in the outside air because it is at -40C and even then has low humidity. This means that condensation is never really a problem - you might get a bit of it but it very quickly evaporates because of the incredibly low humidity inside. In fact the humidity gets so low that our data centre has a humidifier to bump it up to the safe operating range of machines.

    Conversely in the UK where there is no extreme cold weather (yes I know the beeb goes nuts if London drops below -5C but sorry, that doesn't count!) but lots of humidity. As a kid I used to have far more problems with my glasses fogging up when I came inside during the winter that I do in Canada.

    1. Re:Humidity more than Temperature by clarkn0va · · Score: 1

      Up here in the frozen wastes of central Alberta in the winter the indoor humidity drops to incredibly low values of 10-20% because there is no moisture in the outside air because it is at -40C and even then has low humidity. This means that condensation is never really a problem

      Static electricity on the other hand... Seriously, every had a drink from the water cooler when it's that cold outside? Be sure to touch the water with your finger first, or you will discover first hand that the tooth is possibly the worst place on the body to experience a static shock. And if you think your tooth is sensitive, imagine how the electronics feel.

      --
      I am literally 3000 tokens away from the chaotic crossbow --Stephen
    2. Re:Humidity more than Temperature by CAIMLAS · · Score: 1

      Here in South Dakota, I have a problem with ice forming on my glasses after coming indoors. It doesn't last long, because the indoor air warms the material fairly quickly and the low humidity causes it to evaporate fairly fast.

      --
      ~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
    3. Re:Humidity more than Temperature by Roger+W+Moore · · Score: 1

      I get that occasionally but not often because of the low humidity. However when it hit -44C one year (below -50C with wind chill) and I was out walking the dog I did notice my glasses start to make strange sounds and I was a little worried that the metal frames were contracting and putting strain on the glass lens!

  25. Conformal Coating by Anonymous Coward · · Score: 0

    What about the use of conformal coating on the motherboards? This would help mitigate the effects of condensing humidity. One of the issues would be that repair of the board becomes difficult, especially if the conformal coating is thick, but I don't think most people are repairing motherboards these days. They probably just buy a new one.

    1. Re:Conformal Coating by StripedCow · · Score: 1

      Actually, I don't understand why electronics manufacturers don't do this all the time. It would make devices much more resistant against water.

      --
      If Pandora's box is destined to be opened, *I* want to be the one to open it.
  26. Re:Try -68 F Room Temperature by dsgrntlxmply · · Score: 1

    Circuits must be specifically designed and qualified for low temperature operation. Common low-cost ceramic capacitor dielectrics (Z5U) are rated only to +15C and are useless by 0C. Y5P/Y5V are rated to -30C. X5R / X7R will get you to -55C. Aluminum electrolytics are useless at low temperature; tantalum is required.

  27. Re:Fuck you by Anonymous Coward · · Score: 0

    Yeah.... and? What's your problem with homosexuals?

  28. If this really was a big problem. by Z00L00K · · Score: 1

    Then it would have been well-known in that area already. I have been living in that area earlier and the issue of low temperatures and condensation was not one of the major concerns.

    What people tends to forget is that when the outdoor temperature goes down the relative humidity indoors also drops considerably and that means that the condensation issues aren't that big. And the most sensitive parts are the hard disks, just wait to unpack them from the ESD bag until they have been up to room temperature. Same for other media transported - keep them in their innermost packaging until they are up to room temp, which takes an hour at most.

    What's really a big problem in that climate is actually ESD issues caused by static electricity.

    --
    If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
  29. Re:I worked for a company called Americold Logisti by Anonymous Coward · · Score: 0

    It used a hosts file filter list to block the known cold ranges and redirect them somewhere else. Then redirected the heat to 127.0.0.1 using the hosts file.

  30. Actually, using hosts lessens heat by Anonymous Coward · · Score: 0

    "It used a hosts file filter list to block the known cold ranges and redirect them somewhere else. Then redirected the heat to 127.0.0.1 using the hosts file." - by Anonymous Coward on Tuesday August 07, @09:53AM (#40904811)

    Wrong - see subject line above, & this: Since custom HOSTS files block off adbanner servers (who are a HUGE cpu killer in scripts, sometimes even infected ones!), and between hosts and noscript?

    Well... you are NOT eating up as much parsing time on the CPU

    (Especially string parse which is VERY CPU intensive, and script processing)

    So - & thus, you eat less electricity, cpu cycles, AND generate less heat!

    BONUS!

    By using custom hosts files, You also:

    ---

    1.) Go faster (since you can hardcode in your fav sites which accesses locally, far faster than calling out to remote DNS servers roundtrip, which MAY be redirected poisoned... AND, by blocking out banner ads - dual bonus!) - saving you money too, for bandwidth/speed you pay for: Triple bonus!

    2.) Go safer (blocking out both maliciously scripted infested banner ads, AND, known sites/servers/hosts-domains that serve them up plus malware or other malicious content exploits), & more reliably online (less chance of DNS redirect poisoning here))

    3.) More "anonymously" to an extent (vs. DNS request logs).

    ---

    & more... see details, in depth, below!

    AND?

    For a custom hosts file generating program, both 32 & 64-bit? Well - I wrote one here:

    http://securemecca.com/public/APKHostsFileInstaller/2012_06_01/APKHostsFileEngineInstaller32_64bit.exe.zip

    (You simply extract its files to ANY folder you like (usually one you create for it, doesn't matter where, but you MUST run it as administrator (simple & the "read me" tab shows how easy THAT is to do))

    * The malwarebytes/hpHosts site admin another person/site hosting it (Mr. Steven Burn, a competent coder in his own right), said it's "excellent" in fact and has seen its code too...

    (Write him yourselves should anyone doubt any of this -> services@it-mate.co.uk , or see his site @ http://hosts-file.net/?s=Download )

    A Mr. Henry Hertz Hobbitt of securemecca.org &/or hostsfile.org can also verify that this program is safe - write him @ -> hhhobbit@securemecca.com

    It'll be releasing soon to sites that host 64-bit programs (even though it also has a 32-bit model, line for line the same code except for 32 in place of 64 in its help file & user interface)!

    What's it do for you?

    It's a custom hosts file mgt. program that does the following for end users (Calling it "APK Hosts File Engine 5.0++") after it obtains custom hosts file data from 12 of the reputable & reliable sources listed below:

    ---

    1.) Offers massively noticeable increased speed for websurfing via blocking adbanners

    2.) Offers increased speed for users fav. sites by hardcoding them into the hosts file for faster IP address-to-host/domain name resolutions (which sites RARELY change their hosting providers, e.g.-> of 250 I do, only 6 have changed since 2006 - & when sites do because they found a less costly hosting provider? Then, they either email notify members, put up warnings on their pages, & do IP warnings & redirectors onto the former IP address range to protect vs. the unscrupulous criminal bidding on that range to buy it to steal from users of say, online banking or shopping sites).

    3.) Better "Layered-Security"/"Defense-In-Depth" via blocking host-domain based attacks by KNOWN bad sites-servers that are known to do so (which IS, by far, the majority of what's used by both users (hence the existence of the faulty but for most part working DNS system), AND even by malware make

  31. Re:Fuck you by camperdave · · Score: 1

    Again, you are correlating marriage and sexual attraction. Homosexuals can and do marry people of the opposite sex, and have children with them. That doesn't necessarily mean they are in any way attracted to the opposite sex. All that it means is that, for whatever reason, they are willing to cover up their gender preference with a more socially acceptable relationship.

    --
    When our name is on the back of your car, we're behind you all the way!