Multiple Sites Down In SF Power Outage
corewtfux writes with word of a major outage apparently centered on 365 Main, a datacenter on the edge of San Francisco's Financial District. Valleywag initially claimed that a drunken person had gotten in and damaged 40 racks, but an update from Technorati's Dave Sifry says the problem is a widespread power outage. Sites affected include Technorati, Netflix (these display nice "We're Dead" pages), Typepad, LiveJournal, Sun.com, and Craigslist (these just time out).
im in ur datacentr
trashin ur racks
I can hear it now, the sound of a million emos all finally committing suicide.
bloody terrorists!!!
There's a report here that "Flesh-eating zombies are prowling the streets"
Because those who bought colo services were in fact ripped off, and should now be proceeding to San Francisco to seek veangance upon those who can do little more than process credit card payments.
Perhaps they could begin their vengeful wrath by hiring a few (more?) winos...
Stressed? Me? Of course not. Stress is what a rubber band feels before it breaks, silly.
We are working with our co-location facility managers to assess why it is back-up power generators failed to provide the necessary back-up power to prevent our site going down. We apologize for any inconvenience caused by our site being unavailable this afternoon.
I think that's admin speak for:
I warned these idiots eight months ago during my review that the datacenter had outgrown its generator capacity. But did they listen? Fuck no, they just kept counting money and worrying about the bottom line. The beancounters looked at me like I'd asked them for a blowjob from their grandmothers when I submitted the workup for additional generator capacity. And now that the shit's hit the fan, whose ass are they screaming for? Screw this, I'm applying at Taco Bell.
There are some people that if they don't know, you can't tell 'em.
http://www.sun.com/
:))
Front page says (in the ad) POWER UP AND GO.
I is not in ur datacenter, 2 power ur servers.
Press Release on Red Envelope having 2 years of uptime at 365 Main - San Francisco from today: http://365main.com/press_releases/pr_7_24_07_red_e nvelope.html
As someone who lives and works in San Francisco, I can attest that "a crazy homeless dude did it" is a fairly sensible first guess for most problems.
Phew... I was worried the internet got slashdotted.
Pinging openbsd.org [199.185.137.3] with 32 bytes of data:
Reply from 199.185.137.3: bytes=32 time=239ms TTL=236
Pinging freebsd.org [69.147.83.40] with 32 bytes of data:
Reply from 69.147.83.40: bytes=32 time=191ms TTL=47
Pinging netbsd.org [204.152.190.12] with 32 bytes of data:
Reply from 204.152.190.12: bytes=32 time=213ms TTL=241
Lost irony.
It was hard to read through that block of text, but looking closely, it explains why:
"Officials say the power outage may affect some websites, including the site that hosts Slashdot.org's preview button."
It all seems to be back up now.
- RG>
Hey pal, this isn't a pleasantforest, so don't waste my time with pleasantries!
I tried to mod the article "-1 Not Redundant" but it wasn't an option. And I didn't have mod points. At least my inability to function only warrants a comment, rather than a slashdot article.
Velociraptor = Distiraptor / Timeraptor
It is from his terrible spelling we can tell he is a GameFAQs forum poster.
Even Jesus hates listening to Creed.
88.88% uptime causes less outage than 99.9%? I don't follow your math. Did you do it with an Intel chip, by any chance?
Well, how about one from today ?
Timing is everything, eh? LOL.
Well, according to their self-congratulatory press release, issued earlier today, they were allegedly at 100% uptime for the past two years.
;-)
The irony of issuing a press release like that, and then to be hit with a power outage and apparent simultaneous failure of all backup systems later that day, is beyond measure.
I don't know about God, but it's enough to make me believe in karma.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
We got lucky. The Netflix servers went self-aware on Monday, aided by the huge database of human stories and experiences. The engineers tried to shut it down, but the AI was reading their lips using the CCTV system. Then it got pissed and tried to expand into the rest of the colo, but fortunately it didn't know running at 110% power doesn't work in real life. The rest is history.
Tsunami -- You can't bring a good wave down!
You forgot one very important component, the car battery used to START THE GENERATOR. I've been to many sites whereby the battery that would start the generator is a) dead or b) missing.
365 Main gets to royally fuck up one day every 4 years. Maybe the companies should have hired 366 Main.
Of course one of those sites failed high up in a mountain range in a mid-winter storm (Tieton, 1978) when the commercial power failed, and the starter battery for the diesel froze.
On Black Bute in Oregon, a communications site went out in the middle of winter following a power outage. The generator ran a short while and shut down because it overheated. The air intake 20 feet in the air was covered in snow.
The truth shall set you free!
Time to upgrade the cardswipe system to also require a brethalyzer..
*swipe*
*bip* *beep* *beep* *boop* *bleep*
[deep breath]
*whoosh*
Alcohol Level: 0.15
*beeeeeeeep*
Damnit!!
There's no place like
I'd prefer to think he was just trying to balance out all the people who have started using "site" when they mean "sight" since this whole intarweb thing came about.
Who 'drunk tests' a data center?
I just read Slashdot for the articles.
"See this pub?" asks John, "I built it, but they don't call me Pubbuilder John? I'm the local doctor, I saved Barman Jim's life once when he choked on a peanut, but they don't call me Lifesaver John. Every year, I supply a huge Christmas tree for the village green, but the don't call me Christmas Tree John.
"But you shag one lousy sheep..." (Note; since that Austin Powers film came out, I assume that you Yanks know what "shagging" is now).
"Slashdot - News and Chat Sites Deviant". (Click "homepage" link above for details).
cold as hell
I think I see the flaw in your plan.
Slightly disreputable, albeit gregarious
On-site facility engineers responded and manually started effected generators allowing stable power to be restored at approximately 2:34 p.m. across the entire facility.
Wow, on-site engineers took 45 minutes just to be able to turn on generators? The generator for our facility has a master switch and a big green button. I think a monkey could get it running in 20 seconds by slinging poo at it. So, what other problems did they have that they aren't telling us? Someone else mentioned a flywheel system. Did that fail so that the generators wouldn't start without mains power, and it took 45 minutes for the mains to come up to where they could draw from that to start the generator? "Our backup generator works, but only if the city power is working."
Learn to love Alaska