Slashdot Mirror
Confirmed Gmail / Google App Outage
mbone writes "Earlier today there was a confirmed Google outage which got a lot of attention from network operators. From a post to NANOG after everything calmed down: 'Google ack'd a maintenance on their core network did not go as planned-Forced traffic to one peer link that was unable to handle all the traffic. Maintenance has been rolled back. Issue has been restored.' This is exactly what makes me nervous about cloud computing and data storage. It's bad enough when I screw up a config and it takes down my mail, but what about when it happens to the entire globe at once?" Several readers also point to CNET's coverage of the outage.
Update: 05/14 19:25 GMT by T : CWmike adds this: "Steven J. Vaughan-Nichols writes that what may be happening is a massive DDoS attack. Based on the size of the attack that would be needed to interfere with Google, I believe that it's quite likely to be the result of an attack from the controllers of the Windows worm, Conficker. Another theory that has been put about — that the problem was due to AT&T NOC routing problems — does not appear to hold water, writes Steven."
Update: 05/14 21:01 GMT by T : Google's put up a low-detail explanation on their blog that says "An error in one of our systems caused us to direct some of our web traffic through Asia, which created a traffic jam. As a result, about 14% of our users experienced slow services or even interruptions."
My Google voice account went all sorts of haywire.
1) Text messages sent from the web got duplicated. One person got near 10 duplicates in quick succession. I also got duplicate messages back.
2) My number doesn't work. If you call it you get a "Currently unavailable"
3) A few calls that came in before the outage aren't showing up in the Received/Missed calling list.
>>And what's -really- the difference between a server going down locally that affects you and a server going down globally that affects you? Nothing.
Actually, I disagree. There is a difference. If it's local and I own it, I have to fix it. If it's outsourced and Google owns it, I sit back and let Google fix it. Which is nice.
ThePlanet.com had a bad switch install a few days ago which brought down part of our cloud. Our website was down, as was our access to Google DNS gave an IP down there for Google. If you look at the last year, the cloud solution has had a better uptime than what I was providing computing in planned maintenance, patching, updates and all.
It was nice to leave at 5pm, knowing ThePlanet would fix the switch and get us back up. And they did. It's a lot easier to gripe about the cloud being down and sit back, than to manage and fix your own local servers switches and such. When you get to managing hundreds of servers, it becomes time to know what to outsource.
You don't even need to run POP to do this; just use the gears widget for offline access.
It could be that the only purpose of your life is to serve as a warning to others.
ack'ed is short for acknowledged, by way of TCP (which sends ACKs and NACKs). In the networking world, saying ACK as shorthand is pretty common.