Slashdot Mirror


How Peer1 Survived Sandy

Nerval's Lobster writes "When hurricane Sandy knocked out the electricity in lower Manhattan, data-center operator Peer1 took extreme measures to keep its servers humming, assembling a bucket brigade that carried diesel fuel up several flights of stairs. Ted Smith, senior vice president of operations for Peer1, talks about the decisions made as the floodwaters rose and the main generators went offline, as well as the changes his company has made in the aftermath of the storm. He said, 'When the water got to a point that it had flooded the infrastructure and the basement, we were then operating under the reserves the building had on the roof, and our own storage tanks. Literally, at that point we had to do calculations as to how long we could run. And we believed we had enough diesel fuel—between what is in the building, and in our tanks, to about 9 AM the following day. ... You know the bucket brigade—it’s something I’ve never asked the team to do. If you think about what that was at that time, you’re talking about carrying fuel up 17 flights, in total darkness, throughout a whole evening. We had informed our data center manager that we were shutting down, but he kind of took on it himself to say, ‘Not on my watch.’ And he organized himself, got a temporary solution and then more customers jumped in. And at peak I think we had about 30 people helping.'"

7 of 130 comments (clear)

  1. Surprising by Lieutenant+Buddha · · Score: 5, Funny

    I would have thought that they barricaded the doors and windows with wicker baskets and throw pillows. Wait...

    --
    "A casual stroll through the lunatic asylum shows that faith does not prove anything." ~Friedrich Nietzsche
  2. They didn't survive by CastrTroy · · Score: 5, Informative

    From what I hear, based on the StackExchange podcast, and the tweets that went out from SquareSpace and StackExchange during the whole idea is that Peer1 had a complete failure, and it was only due to the hard work of their customers (SE and SquareSpace) that the datacenter was able to remain operational. If your customers have to start carrying buckets of diesel up 17 flights of starirs, you, as a datacenter have failed. Peer1, left to their own devices would have just let the thing shutdown, and apparently head office wasn't aware of how bad things even were.

    --

    Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
  3. Obligatory XKCD by Anonymous Coward · · Score: 5, Funny

    http://xkcd.com/705/

  4. Re:Health and safety? by sunking2 · · Score: 5, Insightful

    Oh please. That is about the most boneheaded thing he could have done. Stairwells full of open buckets of fuel being handed off to people. What could possible happen if a fire occurred? The guy should be fired for needless risk to not only the people, but the building in general. Just so he could keep his stupid uptime hours going after they had already notifed everyone things would be shutting down at any time.

  5. Re:Health and safety? by onkelonkel · · Score: 5, Insightful

    Did you seriously think they were carrying diesel in open buckets? They were almost certainly using proper fuel containers (ie gas cans). Bucket brigade is a figure of speech.

    --
    None of them can see the clouds; The polished wings don't care.
  6. Re:A bucket brigade of Diesel fuel? by operagost · · Score: 5, Informative

    It also doesn't vaporize at room temperature like gasoline does. A spark can start a gasoline fire, whereas diesel fuel needs to be atomized. Geeks should know this.

    --

    Gamingmuseum.com: Give your 3D accelerator a rest.
  7. Re:Health and safety? by Dishevel · · Score: 5, Insightful

    Diesel fuel does not burn the way you think it does.

    --
    Why is it so hard to only have politicians for a few years, then have them go away?