Slashdot Mirror
Could a Category 5 Hurricane Take Down East Coast Data Centers?
TheNextCorner writes "With more data moving into the cloud, there is an increasing danger of data loss when one of these cloud computing data centers fails. Hurricanes pose a real threat to infrastructure located in Virginia and North Carolina, where Google, Apple & Facebook have opened large data centers. 'Where would the most damaging hit be? It's debatable, but the most detrimental hit may be in Virginia. Amazon Web Services (AWS) has one of their major centers in Northern Virginia. ... In a study involving millions of people, a third of those surveyed reported visiting a website every day that used Amazon's infrastructure. In 2011, Amazon's S3 cloud stored 762 billion objects. It's possible that Amazon's cloud alone holds an entire 1% of the Internet.' Could a category 5 Hurricane become a problem for these cloud data centers and take down parts the Internet?"
Could a category 5 Hurricane become a problem for these cloud data centers and take down parts the Internet?"
Only if they haven't switched to Cat 6 cables yet.
Ezekiel 23:20
A Cat 5 impacts the East Coast and we are worried that Facebook or Amazon might be down?
This sig is not paradoxical or ironic.
You may or may not believe in global warming causing hurricanes, but if it could take down a good part of internet then is a cyberthreat, could be even seen as cyberterrorism. What country we should invade this time to prevent that danger?
I live in Virginia. Yes, hurricanes do a decent amount of damage on a regular basis (oddly, my internet is more resilient than my power - I can hook up a generator and still get internet).
But everywhere has a risk. West coast has earthquakes. Midwest has tornadoes. Northeast has blizzards and nor'easters. Maybe some are less of a hazard, or are more mitigate-able, but nowhere is "safe". Or at least, no affordable place is "safe". There's just varying amounts of danger.
First, these companies probably have catastrophic recovery plans in place. Amazon, in particular, is not know for just sitting around leaving its business blowing in the wind.
Second, the loss might slow down the internet, but unless the data hosted at these data centers was unique (which is unlikely) then the other data sites just pick up the slack. Again, that might be slower, but it wouldn't result in loss of data or "teh internet." That is to say, they will act like every other functional part of the internet, route around the damage and carry on.
A regular cloud yes, but not an iCloud.
Sorry, but gray text on gray background is making my eyes bleed.
Oh, no! People won't be able to update their status to show they have gone to the toilet!
it's the cloud right - if you've built your application correctly you're already load balancing across multiple data centers operated by multiple cloud providers and can endure a nuclear attack with little more than a blip of increased latency before the application keeps on chugging, right?
riiiiiiiiiight.
(posted as AC as I don't have a /. account and post maybe once every 4 years)
I've never heard of a major cloud storage facility that would keep all their servers in one location. They usually have all their data backed up to remote locations, usually far from their main site. We are taking about Amazon, and Google here, not Black Berry RIM. I'm sure their data is safe.
-- By all means let's be open-minded, but not so open-minded that our brains drop out.
Hurricanes don't really sneak up on people. Anyone with anything near the coastline will know about 5 days in advance whether or not to call their backup procedures.
Anyone that hosts their entire web presence at Amazon Virginia (especially after the other outages they've had), or really, in any single Amazon region is getting what they pay for and what they deserve if there's a regional disaster.
It's not hard or expensive to have a cold- or warm- spare site in a different region ready to take over (even if it's a manual cutover), especially since Amazon's new(ish) US-West region in Oregon is the same price as US-East.
I like that Amazon lets me pay for the level of redundancy I need - a small bump for multiple availability zone within a single region redundancy, with a larger bump for multiple region redundancy. Not everything I do needs to ride out an East Coast hurricane, but for those things that do, it's really not hard to have a backup site in a different region.
I live on the east coast in an area that sees tropical storm activity every year. I can safely say that a cat 5 hurricane hits data centers will be the last of my concerns since I would have no home or job left.
Global warming is caused by man building huge coal powered data centers, Earth sends hurricanes to destroy data centers, life on planet is saved.
" It's possible that Amazon's cloud alone holds an entire 1% of the Internet.' Could a category 5 Hurricane become a problem for these cloud data centers and take down parts the Internet?"
Yes. Which is why you need redundancy. Plan to fail.
Personally, I would only setup edge servers in areas that have high risk of damage (basically anything south of Montreal in the Eastern time zone) On the West coast, because of the earthquake risk, is still a much lower potential than severe weather. It seems absolutely insane to have any critical data in Florida.
Where should you put your data?
Iceland. Kelowna BC, Regina Winnipeg. All geographically stable with cheap energy.
Edge servers should be in every capital city and major population base, except those in tornado alley. It's important to state that only edge servers should be in potentially dangerous areas, since you don't want to risk data loss, but you can risk losing the hardware.
But storing data in the eastern Timezone is more hazardous than the western time zone. Unfortunately the East coast is more central to Canada and the US's capital cities, and distance to Europe.
West coast has earthquakes. Midwest has tornadoes. Northeast has blizzards and nor'easters
There is quite a lot of geologically stable space entirely lacking in natural disasters between "West" and "Midwest". Like all of Utah and Colorado and New Mexico and Arizona (leaving out Wyoming because of the supervolcano).
Locating in Virginia probably gives them a cheaper supply of power though.
Hardening against a cat5 hurricane is probably a decent tradeoff for them.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
The dericho on June 29 caused a power outage that took down a data center in Virginia. That wasn't even a hurricane.
Dont put your stuff in the cloud... Problem Solved!
First off, a Category 5 hurricane is highly unlikely striking that region of the country. Historically, there have been only three confirmed Category 5 landfalls, two of them in Florida and one in Mississippi (the 1935 Labor Day hurricane in Florida, Hurricane Camille in 1969 and Hurricane Andrew in 1992.) There has been Category 4 storms that have struck the Cape Hatteras area, and South Carolina did have Hurricane Hugo in 1989. But the odds of a Category 5 hitting that specific region of the US is extremely low.
Additionally, these data centers are not located along the coastline, but a significant distance inland. Facebook's is west of Charlotte, while Amazon's located west of Washington DC. Of the list, the Amazon one that could... and I mean could be impacted by a hurricane, but there really hasn't been a good strike in the Chesapeake Bay area in a while. They were taken down by the derechos that rolled through last month, and a derecho could happen pretty much anywhere west of the Rockies.
So while the chances of a hurricane taking down one of the datacenters is low, it could happen. It's one reason you don't see data centers built anywhere within 150 miles of the Gulf Coast or in Florida as a whole, the entire region is a target zone for Mother Nature. (Disclaimer: I've lived along the Gulf Coast now for over 30+ years and have been through a Category 5, two Category 4 and a host of other hurricanes over my time.)
Here we have a risk that requires mitigation. If you owned the facilities in question you would know your disaster preparedness and would know how much effort you are willing and able to put into enhancing it.
But since you don't own these facilities you have to trust the companies that do own them to do what you would do (or better). The only real controls you have are in negotiating the initial contract (regarding SLAs, especially) and in designing your system to withstand a failure of one company to protect their facility. That means you have to either buy resources on both coasts from one company or buy resources from multiple companies whose facilities are geodispersed and make sure your code/platform understands and deals with losing one or more of them.
The leggy gal on the sales team won't tell you any of this. I think most people don't find out about it until the disaster actually happens. It's pretty much like any other piece of your tech stack: the vendors will whitewash the risks and your job is to see through that and manage it.
I submit this isn't a risk caused by the use of "the cloud" (egad, do I hate that term!) so much as a risk that's part of any IT project and you deal with it the same way.
So to answer the original question, maybe a CAT 5 hurricane can take those facilities down but the question you should be asking is, "Have we completely understood the risk to the business and have we taken appropriate steps to protect it?".
Don't believe anything I say. I crash test crack pipes for a living.
After seeing Andrew in 1992, I can say no, your data centers are just waiting to be ripped out of the ground.
The only point of this story was to attempt to get you to click the link to Slash BI that is at the end.
Dude, we lost a data center in 2002 to a SuperSoaker. One of the admins offered to do a tour for "Bring Your Terror of a Child to Work Day" and failed to check SuperSoaker's at the door.
I've never been more proud of my son.
Seriously though, yes, a hurricane could easily knock out east coast data centers. https://encrypted.google.com/search?q=hurricane+isabel+baltimore
..It'll get moderated down to -1 if it goes anywhere near Virginia.
Forward! -- Emperor Norton, 2012
... in their Ashburn DC. The freak thunderstorms that hit the East Coast a few weeks ago knocked the Ashburn DC off-line completely, killed a bunch of major websites, including Netflix, who had strutted loudly how its multi-AZ redundancy strategy prevented them from being subject to any one AZ dying. They were wrong, as were many other sites.
Read Don MacAskill's blog (CEO of SmugMug, heavy consumer of AWS services: http://don.blogs.smugmug.com/). He addresses AWS outages a number of times and has intelligent and worthwhile insights into how to design your apps for failure and fail-over.
One of the interesting things about Amazon's most recent big outage is that providers in the SAME building did not have an outage - none at all (no power going out, no Internet going down, nada). Either Amazon has done things on the major cheap - possibly - or their power design is very poor.
I'm not sure if there is a firm grasp on the geography of these states. Virginia and NC are medium sized states, that yes, touch the ocean where a large Cat 5 hurricane might launch itself upon their beaches. It's never ever happened before, but hey it might. Now, I'm willing to bet that these large data centers are NOT on the beaches of these states, but perhaps just a wee bit farther inland. Perhaps even as far as the mountainous areas that run through the western portion of these. I don't know. I didn't look these things up, but if so a hurricane would die a fairly quick death as it moved inland in either area. Now this does not include the possibility of a hurricane coming up the Chesapeake bay and into Baltimore or Washington DC. However if that happened, and DC was flattened on the way to hitting the Amazon data center in Northern Virginia, I think we'd have a few other problems than lost data.
Many California data centers are within range of major fault lines.
A few years back I belatedly discovered (the hard way) that my web hoster had located its servers in Hurricane Alley. My site was down for over a week as they trucked their server farm to a new location because the local utilities weren't going to be back until God knew when. I've since been paying attention to where things are located, physically, and anything that might be threatening to that area.
The internet will be largely unaffected: the amorphous network of routers which makes up the internet is specifically designed to route around damage like this. The internet will be fine. The web, on the other hand, could suffer a large loss of access to content.
Highly independant small, redundant data hives with redundant power sourced seperately from the grid (aka wind, solar, geothermal, hydroelectric). Let's face the catastrophies which annihalated the dinosaurs and not the insects with the same solutions mother nature has proven time and time again can succeed.
We discussed this a few weeks ago, when the AWS US-east availability zone puked (of it's own accord, no hurricane's required). If you had stuff that lived only there (a single EC2 instance and no ELB, for example), you were screwed. This does not change the fact that "the cloud" (defined as a particular set of services that are readily available from AWS, and probably others) can survive the loss of this or that location just fine. Note that just having the word "cloud" in the name of this or that service does not automatically imbue that service with that level of availability. Read the previous sentence again. And again. Yes, I'm beating the "no magic in the cloud" drum now too, because apparently too many of us don't get it. Yes, you can buy highly available stuff from AWS and others, but a single EC2 instance ain't it.
Any cloud services should be geographically redundant such that any disaster anywhere in the country would not affect 'cloud' based services.
blindly antisocialist = antisocial
The biggest issue in a windstorm event is (tornado or hurricane), is likely going to be damage to the support infrastructure, and possibly generator fuel. For example the external heat exchangers mounted outside the building would likely be blown away or damaged, thus effecting the datacenter's ability to keep cool. Also if the datacenter has an external fuel tank or external generator those could be damaged and made inoperable.
In a very large hurricane scenario I'd think that fuel deliveries might be problematic in the first 24-72hrs after the event. Probably due to inability to safely get the fuel trucks into the facility due to downed lines/trees/bridges.
Funny. Here I thought regular clouds would be fine, seeing as they are high up in the sky and would reform on their own anyway, whereas all of our manmade iClouds are, ironically, stationed on the ground.
Sure I sold you robot insurance. But you were attacked by a cyborg. Not covered.
Arizona, Colorado, and Utah have wildfires.
Come on. Unless you build a data center out of wood in the middle of a forest that is totally irrelevant. I live in Colorado, the whole front range and the middle of the mountains (in short, the places where you would actually build data centers) are totally immune from forest fires. New Mexico is much the same way.
Arizona and Utah have literally nothing to burn.
Utah has tornadoes (even in Salt Lake City).
There are tornadoes, and then there are tornados... the truth is that Salt Lake City (and Denver) only ever see mild tornadoes that might tear off shingles from a roof or hurt a single weak structure (like a barn or shed), not the kind of town-devistating stuff you get in Kansas. Too close to the mountains means the air simply has not had enough time to build up energy.
New Mexico has terrible flooding when hit by a Hurricane.
?????? Citation REALLY needed.
Arizona has had earthquakes.
Very slight ones, it's not on a fault that is going to bring down anything.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Too excited about Facebook being taken out by a hurricane.
Yes.
If the supervolcano went off, the only states that would still be livable are the three Pacific ones.
The rest of the states can give up if they like. Colorado will be just fine thanks. 20 feet of ash? Not an issue. We'll become the new capital of the U.S. if you want to huddle in fear.
You estimate is way off though, just look at projected ash fall maps... most of the country is OK, including both coasts.
Colorado has huge snowstorms that leave people trapped in their homes.
HA HA HA HA HA HA HAH AH HO HE HAH HAH EHHAHAHAH.
Yeah for a DAY!! If that. The last snow storm we had like that (several years ago) was three feet, the day it stopped snowing we could have gone anywhere we wanted in a Jeep after shoveling out our side street (took the neighbors about four hours). A day after that we could use a normal car. Hospital personnel and other important people were driven to work by people that had trucks.
You don't understand, the thing about Colorado is it's semi-arid region. We get snow of significance (meaning you have to shovel a walk or driveway) three or four times a year. Any snow that does come is eradicated within a day or two by the incredibly dry conditions and the 300 days of sunshine per year.
Snow that STOPS you from leaving the house? Once every ten years, if that. And even then it's not stopping you if you have a jeep or other heavy truck, which loads of people have and so would data centers...
"There is more worth loving than we have strength to love." - Brian Jay Stanley
I have my business data hosted primary in Somoma/Napa CA (wine country), a secondary in Monterrey CA (wine & beach), another on the big island of Hawaii (pacific island), and I am really thinking of adding a server in southern Louisiana -probably New Orleans. It is a bit rough having to take a long weekend and go check that the colo is maintaining infrastructure as per our agreement, but as long as I keep checking on one every couple months it is liveable...
Seriously though, keep your data in multiple locations, keep multiple backups, and don't worry too much about any one going offline -just as long as they don't all go offline at once.
"You want to know how to help your kids? Leave them the fuck alone." -George Carlin
This seems a silly topic. We've recently seen what a bad thunderstorm can do. Of course a Cat 5 is a risk, don't ask stupid questions.
It was Bugs Bunny posting AC. Duh.
A Cat 5 impacts the East Coast and we are worried that Facebook or Amazon might be down?
Without Facebook, how else would I know about the cat5???
"There is more worth loving than we have strength to love." - Brian Jay Stanley
New Mexico has terrible flooding when hit by a Hurricane.
?????? Citation REALLY needed.
He's probably mixing up the state of New Mexico and the country of Mexico -- something more common than one would think. Who knows why. . . .
Has anyone ever used the term "Eastern Seaboard" outside of the movies? It gets under my skin, the cliche use of that term in all the movies, and yet I never hear it used anywhere else.
If you are not allowed to question your government then the government has answered your question.
If the loss of only one data center takes down a cloud then you are using the wrong cloud provider. It's like having your disks set up in a raid 5 array. If you lose one disk and your raid array fails, then you are using raid controllers from the wrong vendor. If the loss of one data center slows the cloud down, then that's OK. The loss of one data center or disk should not cause a catastrophic failure.
they should model the datacenter's like Terremark's NAP of the Americas. located near downtown miami this datacenter can withstand category 5 hurricanes.
http://www.terremark.com/data-centers/americas/nap-americas.aspx
If the data center is anything like Terremark Worldwide (recently acquired by Verizon), I would say there's not a snowballs chance in hell.
http://www.terremark.com/data-centers/americas/nap-capitol-region.aspx
I'm more concerned whether these data centers will stand up to a Zombie Apocalypse. You know, breaking in, ripping all the wires out...
Seriously, datacenters in NC aren't on the coast. But there is a nuclear power station south of Wilmington, NC that's pretty much on the coast. If *that* is hit by a Category 5 hurricane - then I don't know what would happen.
A nuclear power plant going kaboom is more concerning to me than a datacenter going kaboom.
During Hurricane Andrew, the University of Miami data center was one of the major data centers located within the area of major damage. The UM data center came through relatively unscathed. My understanding is that sourcing diesel to refill the generators was a minor concern, but resolved before there was any loss of power. By comparison, the data centers in downtown Miami are much stronger buildings, and I would expect that anything less than an F4 tornado would be a non issue.
--- Generation X: The first generation to have SIG lines inferior to their parents... ---
"Any headline which ends in a question mark can be answered by the word 'no' ".
This is high school journalism, Slashdot. Pick up the pace a little.
I want to delete my account but Slashdot doesn't allow it.
Asking if [some natural disaster] could adversely affect [some technology], is pretty much one of the most vapid questions a person can ask.
Can a [flood] take out the [Asian semiconductor industry]?
Can an [earthquake] take out a [bridge]?
Can a [tsunami] take out a [nuclear reactor]?
Can an [iceberg] take out the [Titanic]?
The bigger question to ask is, "Do those in charge of these technologies assume they are impervious to natural disaster?" If the answer is yes then invest elsewhere or sell short and never climb on board..
I haven't thought of anything clever to put here, but then again most of you haven't either.
If the data center is built with the correct architectural planning and resilience, then no it shouldn't affect it at all. these are the same centers that withstood the Earthquake and the horrendous storms recently. Granted that Amazon was a casualty of the storms, but there were so many other organizations that ran without a single hiccup. this is the reason yo want to load balance your service across geographically separate areas. its highly unlikely that a hurricane is nailing your east coast center the same time the Mississippi is flooding your mid west center and an earthquake takes out your west coast center. SOME users will be temporarily inconvenienced, but the service as a whole will be online.
the main things that relate to a data center staying operational:
1) will the building physically survive the incident. no brainer.
2) can power be maintained, including generator support. no brainer
3) can the telco connections stay active. no point in humming servers if a tree takes out your connection to the world.
http://seekingalpha.com/article/289607-data-centers-prove-earthquake-proof
I work for a major bank in a data center and we have taken 2 hurricane hits, granted the worst was a Cat 3 storm but the site was not effected by either. The site can run for 3 weeks on it's own power and the internet connections are pretty hardened. At worst was that we had no water on site for a week. I am sure any major data center on the East Coast will plan the same way.
Anyone remember that time when all of Valve's Steam networks went down because of a really big storm in Seattle that took down the power grid in 2006?
Took them +/- 24hours to restore all services, they had to move lots of servers to data center that had power.
Looks like the closest data center to the coast is in Charlotte, NC. This is (roughly) 150 miles inland. The chance of severe hurricane damage is quite small. Ideally, large, global companies should be duplicating their data across data centers separated by large geographical distances anyway.
"No Black Swans here. Move along."
Well, Typically a Hurricane leaves a larger footprint than a Tornado
That's an understatement.
A hurricane has a footprint, what, 10,000x that of a tornado? Hurricanes are hundreds of miles wide.
With the first link, the chain is forged.
Huricane/Hisacane -- not to worry.
Friends that have been in these monster sites are mighty closed mouthed
but the concensus is that they are a lot like hardened bunkers.
Genarators.. yes, food, water, cooling.... check....
But how about a backhoe. A backhoe cutting the data links
is much more likely followed by a host of crazy other things.
Should the site have fibre heading out to all four points of the
compass it is unclear if the long haul dark and dim fibre centers
are as well provisioned.
Then there is that last 20 mile leg to your company or home
that is even more fragile.
The risks are real but when companies understand this
a simple contract line can mandate redundancy a third
of the continent away. And with the price of storage the
local machine rooms could be backup data centers when
five years back they were data processing centers. i.e.
minimum CPU, max local disk ready to push to another
cloud.
Truth is stranger than fiction, but it is because Fiction is obliged to stick to possibilities; Truth isn't. Mark Twain.
I work in a data center in New Orleans, and so I am fairly familiar with this topic. Of course they're vulnerable. If you live in a region like this you better have a well thought coop/dr plan. We have a hot Site an actually migrate production services there several times before hurricane season and testing services there is part of our development/test process before things go into production. Outside of that, our building was designed to withstand a category 5 storm but the biggest problem is power. We are good for 72 hours on fuel for our generator, but if a Katrina size storm hit again refueling could become problematic. Also, there are personnel issue (evacuation orders, etc). If im at a small data center (relative to amazon/fb/etc) I would have to guess they are already all over this.