Slashdot Mirror


Lightning Strike KOs Amazon, Microsoft EuroClouds

1sockchuck writes "A lightning strike has caused power outages at the major cloud computing data hubs for Amazon and Microsoft in Dublin, Ireland. The incident has caused downtime for many sites using Amazon's EC2 cloud computing platform and Microsoft's BPOS (Business Productivity Online Suite)."

20 of 189 comments (clear)

  1. So Cloud v Cloud.... by artor3 · · Score: 5, Funny

    ...nature wins?

    1. Re:So Cloud v Cloud.... by Torinir · · Score: 3, Funny

      Flawless Victory!

    2. Re:So Cloud v Cloud.... by Trilkin · · Score: 3, Funny

      There is bad fanfiction depicting this event. Lots of it.

      --
      Nobody cares what the CAPTCHA for your post was.
    3. Re:So Cloud v Cloud.... by Guspaz · · Score: 3, Informative

      That might be true if Amazon didn't have multiple AZs in single datacenters.

      The fault isn't necessarily Amazon's for stuff like this. The whole point of cloud infrastructure is that you use many cheaper instances to scale load and provide high availability caused by the failure of any one (or group of) node. Take Netflix, for example. While they do have their share of outages, they were completely unaffected by Amazon's big EC2 failure a few months ago, despite the fact that a significant portion of Netflix' infrastructure was hosted out of the affected region. Why? Because they built failure into their system, to the extent where they have a process that goes around killing random instances to keep them on their toes. They've planned for and built their system around the possibility that large chunks of the system might just up and vanish without warning.

      If you're building a large-scale cloud system, *geographic* diversity should obviously be a part of any high availability plan. I'd also say that having provider diversity isn't a bad idea, but it seems like a lot of big cloud customers just stick with one provider.

    4. Re:So Cloud v Cloud.... by SMoynihan · · Score: 4, Informative

      I live in Dublin, and that was some seriously targeted lightning. No sign of storms here, that I saw...

  2. Just imitiating Verizon. by DWMorse · · Score: 5, Funny

    I see how it is. Verizon workers go on strike, MSFT and Amazon gotta call in for something strike-related that's bigger and flashier. Show-offs.

    --
    There's a spot in User Info for World of Warcraft account names? Really?
  3. My Sympathies by smpoole7 · · Score: 3, Insightful

    Considering that my radio stations have been getting hammered for weeks now by this horrible weather in the Southern United States, my sympathies are with them.

    I don't care how much protection you put on your system (and when you have giant lightning rods that are hundreds of feet tall, like we do, you DO try to protect things), an occasional strike is going to slip through. When it does it can get ... messy. :)

    --
    Cogito, igitur comedam pizza.
    1. Re:My Sympathies by kent_eh · · Score: 5, Interesting

      I've been in the same spot. (10KW, 3 tower array). It's amazing how far the parts of a capacitor on a P&M panel can spread when propelled by a lightning strike.
      Even with ball gaps, chokes, and all the other effort, ultimately the transmitter has to be connected to the tower. 50 ohms is not that much different than "the shortest path to ground" when you put a few thousand KV against it.

      It took several years after my career change to enjoy the spectacle of a lightning storm

      --

      ---
      "I can't complain, but sometimes still do..." Joe Walsh
  4. The Cloud by keithpreston · · Score: 3, Funny

    Sounds about like
    http://xkcd.com/908/

  5. Cloud fail by Baloroth · · Score: 5, Insightful

    My understanding of the point of cloud computing was that it would be distributed. I.e. the failure of any one data or computing center would mean the data was still available. Hence, the term "cloud": nebulous, non-localized. Apparently, someone forgot to tell Microsoft and Amazon what the buzzwords they were using actually mean. I more or less expected that of M$, but the fact that Amazon failed too, well, thats pretty a little surprising. I guess it's kinda the norm for all large corporations.

    Glancing at the article, it looks like this outage effected only a certain area, but still, cloud should mean other data centers would take over. I particularly love the quote "Dublin has become a key cloud computing gateway." If one city serves as a "gateway", its not a cloud system. I understand using it as one data center, but others should take over automatically for that area in case of a failure. If you don't have a failover system, you don't have a real cloud computing platform. You have a wannabe cloud computing platform. Or maybe they are just taking a buzzword and redefining it to suit their purposes. That's... exactly what we should expect, I suppose.

    Or am I completely misunderstanding the meaning of this latest buzzword? It's quite possible, I never quite got down what "Web 2.0" was supposed to mean either. Beyond lots and lots of Flash.

    --
    "None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
    1. Re:Cloud fail by HTMLSpinnr · · Score: 4, Informative

      For EC2, it's only distributed if you pay to have your "service" running in more than one availability zone.

      --
      $ man woman *
      -bash: /usr/bin/man: Argument list too long
    2. Re:Cloud fail by Wolfling1 · · Score: 4, Insightful

      Ah, yes. There is that.

      At the moment, my company is aggressively encouraging our customers to avoid the Cloud at all costs. Let me explain why.

      Whilst the technology exists for the cloud to deliver fault tolerant distributed storage, when you choose to put data in the cloud, you are choosing to relinquish control of the data. You are placing it in the hands of someone else. Quite probably an organisation that you do not know intimately. Quite probably an organisation that is based in a different legislative region - probably another country.

      You have little to no capacity to audit their system. You have little to no capacity to test their fault tolerance. And here's the sucker punch - you have little to no legal comeback for the consequences if something bad happens.

      If your data contains any personal information about another person, you are placing the privacy of that person in the hands of an organisation you do not control, and upon whom you cannot enforce any legislative restrictions.

      So, unless you are seriously geared up to investigate and audit your prospective cloud provider - and they are willing for you to do so, the only data you can safely put in the cloud is data that would be basically irrelevant to your core business anyway. Until the fundamental issues of privacy, security and accountability are resolved - or dramatically improved - placing core business data in the Cloud is a massive corporate risk.

      They should not have called it the 'Cloud'. They should have called it the 'Arse' - because if your management are planning to stick their heads in one, they may as well stick their heads up the other. I don't imagine that 'Arse Computing' would be as popular though.

    3. Re:Cloud fail by jimicus · · Score: 3, Informative

      Cloud computing is a buzzword meaning "don't run your own hardware, run your business on someone elses". Which might mean anything from a virtual server that you manage at one end of the sophistication scale to a SaaS product at the other.

      All sorts of aspects of this are optional. Including:

      1. Whether or not you manage the underlying operating system - including things like security patches and hardening. You can choose a cloud computing provider that has sysadmins deal with that for you and just run the application yourself; they are a LOT more expensive than Amazon.

      2. How much effort your provider puts into making their systems geographically redundant. Few will talk openly about this; I'm prepared to bet hard cash this is because the vast majority that offer you a virtualised server are just using a web interface to expose a fairly vanilla Xen-with-a-SAN infrastructure to the world with everything sat in one place. Providers that will run the OS for you and can honestly say their infrastructure accounts for complete data centre loss are like hen's teeth.

      3. If you've gone for a SaaS provider - how much effort their developers went to to ensure their application can stand up to everything up to and including total loss of a data centre. And whether or not they test for such an occurrence.

    4. Re:Cloud fail by jimicus · · Score: 3, Interesting

      As for auditing, uptime, and legal consequences, you've apparently never dealt with a service contract. If the contract mandates five nines of uptime, and includes a clause making them liable for all damages and loss, that's a pretty hefty legal comeback.

      I agree with you entirely, it's an absolutely beautiful piece of legal comeback. But every service contract I've ever seen is so full of ifs, buts and other assorted get-outs that it's very rare to actually be able to hold someone to it.

      The one time I have seen an SLA that was actually quite good, the company in question didn't refuse to honour it. Oh no. They went one better - they hadn't even told their staff that it existed, so if you asked about it you'd get a response along the lines of "What's an SLA, then?" The only way you'll get an SLA honoured in those circumstances is to take your provider to court, and you can bet that if you do they'll drop you like a hot potato. So you probably wouldn't bother in any but the most egregious of circumstances.

  6. Microsoft renamed its product by countertrolling · · Score: 5, Funny

    Office 364

    --
    For justice, we must go to Don Corleone
  7. Power Co-Generation by anubi · · Score: 4, Interesting

    While working at Chevron Oil Pascagoula Mississippi refinery, I noted Chevron had the same problem. Loss of electrical power to the refinery would be catastrophic. No one wants to be around tons of petrochemical products undergoing serious chemical reactions when one loses control.

    To mitigate this threat, Chevron worked with Mississippi Power to operate a power generation facility at the refinery.

    I would think that anywhere there is a substantial "data processing farm" with critical power requirements, business arrangements should be made with the power generation utilities to run a natgas power plant in the immediate area.

    The utilities often run these plants as "topping" plants, as they needed anyway to even out short-time load variances on the line.

    But, in the event of a serious loss of grid power, it can be awful handy to have a few megawatts of power coming from down the street.

    --
    "Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]

  8. Re:at that level the safety's tipped foreing a man by Animats · · Score: 4, Informative

    Also Surge Protectors can't really take a direct lighting strike.

    But lightning arrestors can. A serious lightning arrestor is a spark gap (sometimes open air, sometimes in an inert gas) to ground, with a very heavy cable or busbar to multiple ground rods, and no sharp turns in the path to ground. This is followed up by an inductor which is a few turns of busbar. This gear is usually placed where power lines or antenna feeds enter a building. MOV-type protection is further downstream.

    Antenna towers are struck by lightning frequently, and the associated radio gear routinely continues to operate. This isn't rocket science. It's big hunks of copper.

    The Hartford Steam Boiler Inspection and Insurance Company, in their publication "The Locomotive" (they've been at this since 1867) has a good article on lightning protection. Hartford Steam Boiler insures not only against boiler explosions, but things like downtime due to lightning strikes. But only after their inspectors (they have 1200) have visited the plant and are satisfied with the equipment.

    A question to ask your "cloud" provider - who handles your business interruption insurance, and do they inspect your faclities?

  9. marketing bullshit by Tom · · Score: 4, Insightful

    And there is the marketing bullshit revealed. All the promises of the cloud - down by one lightning strike.

    Because, let's face it, the whole "cloud" thing as they sell it is just advanced virtual hosting with a different name. The only real cloud capabilities are those the big companies build for themselves, and they did things like that 10 years ago already, when nobody had ever heard the term "cloud" used in computing contexts.

    In the end, it's about selling something to people who already have the older version and convincing them to buy the new one. So you give it a different name because a "new" product sells easier than the upgraded version of an "old" product.

    Anyone remember when "Web 2.0" was all the hype? It really wasn't a 2.0 as we all know. There was nothing new in it, all components had been around for a long time. It was a conceptual bundle, but not a new version like the name suggested.But "we're doing more Javascript" now doesn't sell nearly as good as "we're moving to Web 2.0 now".

    --
    Assorted stuff I do sometimes: Lemuria.org
  10. Microsoft's BPOS by markbark · · Score: 3, Funny

    Business Productivity Online Suite?
    I always thought it stood for "Big Piece of Sh..... never mind