New York Data Centers Battle Floods, Utility Outages
miller60 writes "At least three data center buildings in lower Manhattan are struggling with power problems amid widespread flooding and utility outages caused by Hurricane Sandy. Flooded basements at two sites took out diesel fuel pumps, leaving them unable to refuel generators on higher levels. One of these was Datagram, which knocked out Buzzfeed and the Gawker network of sites. At 111 8th Avenue, some tenants lost power when Equinix briefly experienced generator problems."
The NY Times has a running list of Sandy-related problems, including 5,700 more flight cancellations, 6 million people without power, rising water levels at a nuclear plant, official disaster declarations from President Obama, and a death toll of 38. On the upside, and despite the high water levels, the Nuclear Energy Institute was quick to point out that all 34 nuclear facilities in Sandy's path made it through without problems.
Barack Obama doesn't care about white people.
If we're really lucky, it'll take out all the high frequency traders systems for a few days and we can have an actual market without parasites.
Nah, who am I kidding. If that actually happened they'd keep Wall Street closed.
I still have more fans than freaks. WTF is wrong with you people?
Why aren't there more datacenters in Kansas, Nebraska, North Dakota, etc.? Surely the threat from Tornados could be mitigated and the electrical infrastructure built out more cheaply than the losses due to coastal disasters, no?
If you do what you always did, you get what you always got.
As another poster yesterday mentioned it was sunny and pleasant in Virginia just hours before landfall.
The airport simply closed as a precaution.. No nuclear power meltdowns or any catastrophy as the media reports.
So some waves hit the shoreline. Big deal. New York will funciton perfectly tomorrow like nothing ever happened.
This is as strong as a case forever to stop paying attention to the media.
The city's collective-arrogance towards the rest of the world makes it really tough to pity the celebrities, financial crooks, government-thugs, hipsters and other undesirables.
The best thing we can hope for is maybe Anderson Cooper will wash away.
Not a sysadmin myself, but isn't it poor planning to not have your sites mirrored in a remote location for disaster recovery in situations like this?
Oblig XKCD:
http://www.xkcd.com/
I was promised a NUKULAR OPOCALIPSE yesterday, and here we are with satellite images that don't even show the entire East Coast as a glowing radioactive wasteland.
I think that this complete lack of NUKULAR meltdown is 100% absolute irrefutable proof of two key concepts:
1. Capitalism is an abject failure and we need U.N. control of everything and everyone right now.
2. NUKULAR power is obviously far too dangerous and should be banned right now before somebody doesn't get killed again JUST LIKE FUKUSHIMA.
AntiFA: An abbreviation for Anti First Amendment.
Then everybody can swim for themselves.
I thought it was standard procedure to teach servers to swim these days.
How about not putting mission critical equipment susceptible to water damage in the one place all water will go.
Your hair look like poop, Bob! - Wanker.
It's like the old joke about trailer parks attracting tornadoes. There seems to be some sort of physical law that says that if a data center is going to be set up, it will be in a basement, and in a low-lying area.
Proverbs 21:19
"What were they thinking building a city between two bodies of water near the ocean so close to sea level"?
"Maybe we should just move the city to further inland, where its safer and we won't have to go through this over and over again...and keep wasting money"
I know I've heard that before..hmmmm.
Seems like the should have known not to build so close to the water, and if they did...not to put things that react badly with salt water underground, eh?
Sorry if it sounds like sour grapes...and I do feel horrible for the people that flooded and lost things, I know first hand how that feels from Katrina.
But I do get a bit uneasy..seeing how differently things are treated during the storms and afterwards....depending on where in the US you are situated.
I mean, this storm, while large in breadth....was a weak Cat. 1 when it made landfall. I could see it being worse, if it had happened and turned into a blizzard over NJ and NYC as was a worst case scenario.
But c'mon...if you have property ON the freakin shoreline, beach front houses, guess what...you're gonna get damaged with a hurricane or other strong storm.
Aside from the areas right near the water..I didn't see all that much damage. Sure, people are going to be without power a couple or more weeks in some areas. Those of us that live in the Gulf south take this as normal a couple times a year...that's what you get for living close to the ocean.
Again....not to make light of anyone's loss, flooding is very difficult to deal with...it sucks.
But this was a weak storm, and did about normal damage as happens to places with a storm like this hitting a coastal area.
I feel bad for those that will next have to deal with FEMA.....then again, maybe their a bit better by now...but I still have less than fond memories for them.
Anyway...people in those areas up there....you'd better get used to these storms hitting more often.
These things go in cycles..and ya'll have been lucky the past decades. I believe back in the 50's a number of storms made landfall up there...but has been so long, that people forget.
Last year, Irene and this year Sandy....hope you're better prepared for next year....if nothing else, you're a dolt if you don't purchase flood insurance. It is DIRT cheap....most everyone would be well served buying it if you ever have even occasional flash floods in your area....
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
how much of that is real, and how much is financial, as the airlines decided to cancel flights that wern't full of passengers, or decided to cancel flights to NY on the grounds that the planes would fly out half empty, which is a purely economic decision
If that movie taught us one thing, it was to make sure critical infrastructure was in the physical middle of the country, away from the massive tidal waves that would be generated by impacting asteroids.
And to think they thought they had any hope.
Nature just shit in his cereal. HARD.
After this huge mess, disbanding it would be horrible.
There'll likely be worse storms around the corner, this winter certainly ain't looking up to be a nice one.
It is more a thought of the fact that it is becoming more unpredictable that is worrying. The climate is chaotic as hell right now.
Who knows what kinds of hell it might be in 5 and then 10 years from now.
Think I might go build a mountain city in Scotland. Yes. Everybody secretly wants to live in a cave anyway. Caves are just awesome.
You don't like caves? Well, you are WRONG.
Callcentric apparently had a single datacenter in NYC with no backup power generator. Lots of discussion here.
typo - meant to type EPA
As I understand it, it's the availability of fiber. DC needs a fuck ton of fiber, and a big city is more likely to have it already present then some podunk town.
Did you realize that fiber connects things? And as a result there is a "fuck-ton" of fiber not just in a big city but anywhere the major fiber trunks go, which is all across the country?
What did you think all that traffic did when it left the city? Get printed out and put on a Greyhound?
"There is more worth loving than we have strength to love." - Brian Jay Stanley
that flooding ruins backup generators, pumps, fuel storage. I hope that disaster mitigation plans are reviewed.
I also hate people who judge negatively from hindsight, but disaster planning is about considering the most probable of the improbable. Flooding looms most threatening and probable of the improbable.
Perhaps putting all the backup infrastructure on a higher floor makes it harder to maintain, access, and/ or protect from mischief/ terrorism. However plain old flooding seems to be an issue time and time again in disaster scenarios and really needs highest priority in disaster plans.
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
http://en.wikipedia.org/wiki/Manhattan,_Kansas
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
Is that president Obama declared the situation a disaster area.
Thank you captain obvious.
Eh, religion probbably killed more than 38 yesterday
Remember the data center in the movie "Wall Street"? That was the real life data center of Telerate Systems -- on the 103rd floor of the World Trade Center.
They never did get flooded.
Yes, because a single fiber cable in some backwater town with zero redundancy
Why do you think there is zero redundancy?
Even if you had only one carrier in a town that had a fiber run go through (very unlikely), if the cable gets cut on one side of town the fiber provider can just run all the traffic around the nationwide loop the other way until it gets where it needs to go - and believe me a cut will be fixed VERY quickly.
A town where "a single fiber" dead ends is obviously not a good location, but there are many towns across the U.S. with bundles of fiber going through.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
What happened to the generator underneath Grand Central that the Alphas could not even stop?
The war of the previous ones that attacked New York gave Bush another mandate. Now Obama to get reelected must declare war on weather, explain that they have weapons of mass destruction, and nuke clouds, that will teach them who they are dealing with.
Sandy's wrath of ghod on the banksters.
Notice it didn't hit when OWS was in town.
First Tampa, now this. Oh the inhumanity!
The biggest data center in the path of Sandy was Bloomberg's huge mile-long data center at Washington Street And Houston, Right next to the Hudson River!
When I worked for Bloomberg, we took a tour through that data center. It was not on the ground floor but it would be seriously suffering for power and how were those Bloomberg Terminal signals supposed to get out of that island called Manhattan.
But Bloomberg will keep it secret if anything happens. If he can.
All Linux VPS located in New York City, NY VPS are currently offline. They are located in Internap's LGA6 facility in 111 8th Avenue.
Please be advised that Internap's LGA11 facility is experiencing significant flooding in the sub-basement of the 75 Broad Street building as a result of Hurricane Sandy. The flooding has submerged and destroyed the site's diesel pumps and is preventing fuel from being pumped to the generators on the mezzanine level.
Thankfully, our NYC server nodes are not directly located in LGA11's facility, but rather LGA6. The cause of this temporary outage is that LGA6 routes certain parts of the network's backbones through LGA11 which is currently offline as explained above. URPad's downtown NYC facility, located at 111 8th Avenue, is currently experiencing a network-only outage. The datacenter is not located in the storm surge zone, and is not suffering from any flooding. All URPad hardware and assets are safe and remain powered on. Engineers are aware of the network outage and all efforts will be made to restore network connectivity as soon as possible.
Internap & URPad will continue to work hard to assess the situation and our recovery plans, and will communicate those plans as soon as possible. Thank you for your patience and understanding during this crisis. Please trust that we are doing everything we can to bring your services back online as soon as possible.
So it's impossible for a nuclear meltdown to happen?
Because unless it isn't, then your rant was unfounded (as unfounded as what you think the warnings were about).
before we can tell all those people along the jersey and CT shores, ya wanna live on the ocean, get your own darn flood insurance, we the taxpayers are tired of picking up the tab so you can have a great view ?
This is a category 5 reductio absurdum event.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
"Nuclear Energy Institute was quick to point out that all 34 nuclear facilities in Sandy's path made it through without problems."
I by that for a dollar!!!!!!
"The NY Times has a running list of Sandy-related problems, including ... official disaster declarations from President Obama"
Ooookay.
They better have made copies of the servers in case of flooding. Million dollar HFT algos and matching algorithms are in those servers. Here is the simple stock exchange I made. I am going to integrate live market data soon http://sigmatrader.com/trade.php
Think I might go build a mountain city in Scotland
Just to warn you then, the weather here has been more shite than usual lately. Maybe better than other places in the world but the recent decade has been much worse than the previous three. I can't comment on weather prior to that on account of being too young to remember and ultimately not even existing.
I live in PA and we got hit hard people for 1 to 2 weeks expected with no power or phones. We have lost over 16 power stations. I am a ham radio operator and we have been helping people who do not have phones and no way of communicating. We got hit hard and we are on out skirts so I can only imagine what NY and NJ going threw.
http://www.thetechnologygeek.org
Why does the USA have these disasters each time?
Now it's New York.
Their biggest financial center TOTALLY disabled and destroyed for an estimated 50-100 billion dollars.
No real protection against water.
Our 2nd Maasvlakte -an industrial/harbour area- has 18 meters of levees.
What height of protection did Manhattan have?
Do we need to send aid to build some protection against the water for our former colony?
Anyone living in the low lying costal areas like NYC should be well aware that storm surges will wipe their little assets out. Building a data center in such a place is absolutely foolhardy. Living there is stupid enough.