Slashdot Mirror


How One Drunk Driver Sent My Company To the Cloud

snydeq writes "Andrew Oliver offers further proof that drunk driving and on-site servers don't mix. Oliver, who had earlier announced a New Year's resolution to go all-in on cloud services, had that business strategy expedited when a drunk driver, fleeing a hit-and-run, drove his SUV directly into the beauty shop next door to his company's main offices. 'Our servers were down for eight hours, and various services were intermittent for at least 12 hours. Had things been worse, we could have lost everything. Like our customers, we needed HA and DR. Moreover, we thought, maybe our critical services like email, our website, and Jira should be in a real data center. This made going all-cloud a top priority for us rather than "when we get to it."' Oliver writes, detailing his company's resultant hurry-up migration plan to 100 percent cloud services."

19 of 290 comments (clear)

  1. why cloud? by gl4ss · · Score: 5, Insightful

    or is it just the name for all datacenter hosted servers now? (trick question.. it is).

    --
    world was created 5 seconds before this post as it is.
    1. Re:why cloud? by Anonymous Coward · · Score: 3, Insightful

      And what happens when a drunk driver smashes into them? (or his communications are cut?)

    2. Re:why cloud? by Anonymous Coward · · Score: 5, Insightful

      or is it just the name for all datacenter hosted servers now? (trick question.. it is).

      Your Datacenter + Your HA/DR site = You control where data is replicated.

      Your data + Someone's cheap cloud service = You not having a damn clue when/where your data is replicated.

      It all depends on how critical you think your customer data is, and how much legal control you need.

      And a drunk driver should not be the damn justification line for HA/DR. Common fucking sense should. Mr. Oliver should have used some of that, and New Years Eve was over 7 fucking months ago. Procrastination kills, and my sympathy wanes.

    3. Re:why cloud? by Xest · · Score: 5, Insightful

      Yep, and it works vice versa too. What if your "cloud" data centre suffers downtime, what if your connection to it suffers downtime? Suddenly your staff can't do any work because you have nothing local anymore.

      Article sounds like a cloud services sales pitch tbh.

    4. Re:why cloud? by Anonymous Coward · · Score: 5, Insightful

      The "logic" seems to be:
      If we lost control of the security/secrecy our assets at our place ONCE, put it in a place where its security and secrecy is ALWAYS out of our control.
      But never ever learn anything about good data center or server room design.(What the hell kind of place is that, where you can drive straight through to the most secure back-office room of your company?? No fences? No brick walls? Nothing?? Because *I think* there might be their problem...)

      Because OMGCLOUDXORZ!!!111one(lim (x->0) ((sin x)/x))

      And because this is a Slashvertisement and 100% bullshit.

    5. Re:why cloud? by dejanc · · Score: 5, Insightful

      I am not ashamed of making a mistake when it comes to buzzwords, but the way I see it:

      - My site is hosted on a server in Acme Inc.'s facilities in New York / London / Tokyo. It's in a datacenter.
      - My site is hosted by Web2.0 Inc. I have no idea where it is, but I am hoping they are doing some smart load balancing and backups for me. It's in the cloud

    6. Re:why cloud? by tompaulco · · Score: 4, Insightful

      And what happens when a drunk driver smashes into them? (or his communications are cut?)

      Well, for one, a lot more people are affected, and for another thing, they are not going to be as concerned about the data as you would be if you hosted it yourself.

      --
      If you are not allowed to question your government then the government has answered your question.
  2. Colocation? by Anonymous Coward · · Score: 5, Insightful

    This story isn't remarkable is it. Man shocked when putting all eggs in one basket is a bad idea. Solution: put all eggs in another basket. DR is what colocation and failover is for. The cloud doesn't magically make you impervious to disasters.

  3. Bad design Cloud? by MortenMW · · Score: 4, Insightful

    If a single drunk driver is able to stop your production and that production is critical you are doing something wrong to begin with. While the cloud might (and probably will) offer better HA and DR it will not fix a bad design by itself. The article also states: " I didn't want to create my own internal IT department". I' guessing Andrew Oliver is a PHB.

  4. Re:Lesson not learnt by mysidia · · Score: 5, Insightful

    Moving to the cloud doesn't solve this, per se, if you move all your infrastructure to say Amazon you're still beholden to that company and its internal procedures. A system administration on their part could easily render you down for many hours.

    A data loss on their part could render you down permanently; Do you have a SLA? Do you have proof that your cloud vendors have DR solutions?

    What is your action plan if your leased-line WAN goes down, and your internet service provider tells you that it will be 48 to 72 hours to resolve? May be a fiber cut, or worse. Drunk drivers can take down networks and POPs too.

  5. Re:Lesson not learnt by bickerdyke · · Score: 3, Insightful

    What is your action plan if your leased-line WAN goes down, and your internet service provider tells you that it will be 48 to 72 hours to resolve? May be a fiber cut, or worse. Drunk drivers can take down networks and POPs too.

    When your complete IT is based on SaaS, just send everyone home and let them work from home. All the tools they need are "in the cloud" (or however plain old internet is called today)

    --
    bickerdyke
  6. Re:Yes yes, beat the drum for cloud services by dcw3 · · Score: 5, Insightful

    And besides, if you've shared your data with some company, the government no longer considers it your private data.

    --
    Just another day in Paradise
  7. Re:Bad design Cloud? by FireFury03 · · Score: 4, Insightful

    If a single drunk driver is able to stop your production and that production is critical you are doing something wrong to begin with. While the cloud might (and probably will) offer better HA and DR it will not fix a bad design by itself.

    The article also states: " I didn't want to create my own internal IT department". I' guessing Andrew Oliver is a PHB.

    Because cloud services have never had extended outages...

    Honestly, anyone who sees cloud services as the great fix for reliability problems is an idiot, especially reliability problems caused by a once-in-a-lifetime drunk-driver incident. Most of the cloud services seem to have had their fair share of incompetence-related downtime. I wouldn't mind betting that if he'd put all his IT stuff one one of the commercial cloud platforms for the last 2 years, he would've had more downtime than he had running them in his offices.

    In any case, shoving stuff in the cloud doesn't absolve you of needing a competent IT admin to handle backups and such, unless you're insane enough to trust *everything* to a cloud operator who, at the end of the day, doesn't actually give too much of a crap about one tiny customer who might've lost all their data.

  8. agreed. double up! by leuk_he · · Score: 5, Insightful

    Yup. It is all fancy way to tell services are not in a local closet, but in a specialized center.

    It all seems fancy, until you hit downtime, and your SLA happens to be "best effort" and the response time is nothing more than someon looked at it within a certain time. You will never get a sla that returns money for the lost productivity.

    You will still have to figure out how to get your backups regularly out of the cloud, and retreive the data if the cloud operator stops. You will have to provide a fast internet link, or maybe even a double link, since if one provider fails, it might be cheaper to have a second provider instead of having one with a expensive business SLA.

    Stating "put it in the cloud"sounds simple, but a lot of details are really important. Notice how the Tarticle is a consulting firm in such things? and even they hoose to do in inhouse for quite some time?

    1. Re:agreed. double up! by foniksonik · · Score: 3, Insightful

      Here's a list of things "in the cloud".

      Electricity
      Phone service
      Banking
      Accounting
      Credit card acceptance
      Water
      Gas
      Security / Alarm
      Internet service

      No I'm not being facetious.

      Look at that list. Which ones are not critical to your business operations. Having access to your data is just another item on the list and its likely that its not the most important.

      --
      A fool throws a stone into a well and a thousand sages can not remove it.
  9. so what happens by wbr1 · · Score: 4, Insightful

    What happens when a 'drunken' MBA cancels the service. Or a drunken admin deprovisions the wrong servers?

    --
    Silence is a state of mime.
  10. Re: For all the drunks out there! by Anonymous Coward · · Score: 5, Insightful

    Fake story. Obviously marketing hype for cloud services fad. Reminds me of Y2K

  11. Re: For all the drunks out there! by MrMickS · · Score: 5, Insightful

    I understand it when people with no knowledge go on about Y2K as some sort of hoax. I despair when it comes from people that should no better.

    Any idea why Y2K didn't have the massive impact it could have? Would it be the massive effort testing and patching things to prevent it being a problem. Maybe we should have just left things and picked up the pieces afterwards. Yeah that would have been good. I know for a fact that there would have been issues with the emergency number (999) in the UK. Can you be sure that things would have worked wherever you're from?

    --
    You may think me a tired, old, cynic. I'd have to disagree about the tired bit.
  12. The 30 second version by Minwee · · Score: 4, Insightful

    "How One Bolting Horse Sent My Company Close The Stable Door: We had no high availability or disaster recovery in place, so when a disaster happened our systems weren't available and we couldn't recover from it. That was bad, so we fixed it."

    Next week's article will be "How Losing All Of Our Data Made My Company Start Making Backups", followed in September by "How Losing All Of Our Data A Second Time Made My Company Start Testing The Backups Too".