Slashdot Mirror
How One Drunk Driver Sent My Company To the Cloud
snydeq writes "Andrew Oliver offers further proof that drunk driving and on-site servers don't mix. Oliver, who had earlier announced a New Year's resolution to go all-in on cloud services, had that business strategy expedited when a drunk driver, fleeing a hit-and-run, drove his SUV directly into the beauty shop next door to his company's main offices. 'Our servers were down for eight hours, and various services were intermittent for at least 12 hours. Had things been worse, we could have lost everything. Like our customers, we needed HA and DR. Moreover, we thought, maybe our critical services like email, our website, and Jira should be in a real data center. This made going all-cloud a top priority for us rather than "when we get to it."' Oliver writes, detailing his company's resultant hurry-up migration plan to 100 percent cloud services."
I've been drunk the previous two weeks and it's been awesome!
or is it just the name for all datacenter hosted servers now? (trick question.. it is).
world was created 5 seconds before this post as it is.
Servers Against Drunk Drivers
Self driving cars ...... but what if Google decides to use them to promote the cloud by taking out servers?
This story isn't remarkable is it. Man shocked when putting all eggs in one basket is a bad idea. Solution: put all eggs in another basket. DR is what colocation and failover is for. The cloud doesn't magically make you impervious to disasters.
If a single drunk driver is able to stop your production and that production is critical you are doing something wrong to begin with. While the cloud might (and probably will) offer better HA and DR it will not fix a bad design by itself. The article also states: " I didn't want to create my own internal IT department". I' guessing Andrew Oliver is a PHB.
The issue here is that he didn't have adequate disaster recovery procedures and policies.
The standard solution to this sort of problem is that you have a backup system that sits off site ready to take the load should something happen to primary. This backup system should be located in another data center, with a different ISP etc.
Moving to the cloud doesn't solve this, per se, if you move all your infrastructure to say Amazon you're still beholden to that company and its internal procedures. A system administration on their part could easily render you down for many hours.
The lesson hasn't been learnt.
Because when a car drivers over your computer, a generator isn't going to do it much good?
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
Someone would have been injured or killed.
And besides, if you've shared your data with some company, the government no longer considers it your private data.
Just another day in Paradise
If a single drunk driver is able to stop your production and that production is critical you are doing something wrong to begin with. While the cloud might (and probably will) offer better HA and DR it will not fix a bad design by itself.
The article also states: " I didn't want to create my own internal IT department". I' guessing Andrew Oliver is a PHB.
Because cloud services have never had extended outages...
Honestly, anyone who sees cloud services as the great fix for reliability problems is an idiot, especially reliability problems caused by a once-in-a-lifetime drunk-driver incident. Most of the cloud services seem to have had their fair share of incompetence-related downtime. I wouldn't mind betting that if he'd put all his IT stuff one one of the commercial cloud platforms for the last 2 years, he would've had more downtime than he had running them in his offices.
In any case, shoving stuff in the cloud doesn't absolve you of needing a competent IT admin to handle backups and such, unless you're insane enough to trust *everything* to a cloud operator who, at the end of the day, doesn't actually give too much of a crap about one tiny customer who might've lost all their data.
http://blog.nexusuk.org
Yup. It is all fancy way to tell services are not in a local closet, but in a specialized center.
It all seems fancy, until you hit downtime, and your SLA happens to be "best effort" and the response time is nothing more than someon looked at it within a certain time. You will never get a sla that returns money for the lost productivity.
You will still have to figure out how to get your backups regularly out of the cloud, and retreive the data if the cloud operator stops. You will have to provide a fast internet link, or maybe even a double link, since if one provider fails, it might be cheaper to have a second provider instead of having one with a expensive business SLA.
Stating "put it in the cloud"sounds simple, but a lot of details are really important. Notice how the Tarticle is a consulting firm in such things? and even they hoose to do in inhouse for quite some time?
Amazon Cloud Service Hit By Car Crash
NB: The message above might reflect my opinion right now, but not necessarily tomorrow or next year.
Didn't this cloud used to be known as hosted services, the only difference being that your server is now some VM running on shared hardware and you still have to hire someone to configure/install and upgrade your computing infrastructure.
AccountKiller
What happens when a 'drunken' MBA cancels the service. Or a drunken admin deprovisions the wrong servers?
Silence is a state of mime.
Google, Amazon and Microsoft can't be trusted with your data, but it's better than taking the risk of having a little downtime due to a freak accident.
Exactly what I was thinking. Part of the two (three?) pronged PR strategy to stem the hemorrhaging of international customers due to the Whistlblower Snowden revelations and subsequent fallout:
Step 1: "We are planning (only planning we dont want to rock the boat too hard), a very stern letter to the feds. Even so, we only comply with the law, pinky promise. Ignore those docs Snowden released showing the contrary and our willingness to hand your cloud/email/chat/phone data over to not only 'the feds' but every private Military Industrial Complex company out there, such as Booz Allen."
Step 2: Fear! Yes make them fear if they not not using cloud servers - drunks could take out their servers OMG!
Step 3: ...?
They are fighting an uphill batter to regain trust - you would be mad or incredible ignorant to hand private information over to these companies given what they have done...
I guess I'm supposed to go scrambling for my acronym dictionary, but I just don't care. I'll assume he means laughter and medial attention.
- First they ignore you, then they laugh at you, then ???, then profit.
Drunk drivers have been sending things to the clouds for a hundred years.
Wait. That's not funny. :-/
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
...or worse, your data may be governed by the laws of multiple countries because that "cloud" provider has servers in both the U.S. and Europe and shuttles backups across the pond (which is good from a backup standpoint, but bad from a security standpoint)
"His name was James Damore."
I work for a company that would have been in a good place, and better off than they were, if they had gone to the cloud a year or more before they hired me. However, they hired me because they were experiencing rapid growth and part time IT support from brother of one of the owners was not longer adequate. When they hired me their IT infrastructure was about three years overdue for replacement from top to bottom. The owners wanted to go to the cloud as part of that change. As we investigated options it became obvious that we had outgrown where the cloud would have been a good solution for us (it is not just size, it is also the way that we do business). We are at a size where it is cost effective to build out our own server infrastructure, including what is needed to ensure business continuity rather than pay someone else for it. The cloud might be a viable option as the location for our business continuity redundancy, but it is not cost effective as the location for our day to day operations.
The truth is that all men having power ought to be mistrusted. James Madison
They were running on a badly designed system to begin with. Honestly. why did they not have an offsite failover? If 8 hours of downtime is expensive then the CIO needs to be fired for his incompetence for not having a failover system in place.
Do not look at laser with remaining good eye.
Well, I hope he enjoys lots more downtime now that everything is in the cloud except 0% chance of him fixing it himself. And just wait for that extended downtime when the cloud host goes out of business without warning.
"How One Bolting Horse Sent My Company Close The Stable Door: We had no high availability or disaster recovery in place, so when a disaster happened our systems weren't available and we couldn't recover from it. That was bad, so we fixed it."
Next week's article will be "How Losing All Of Our Data Made My Company Start Making Backups", followed in September by "How Losing All Of Our Data A Second Time Made My Company Start Testing The Backups Too".