Slashdot Mirror
Outages Leave Google Apps Admins In the Hotseat
snydeq writes "This week's Google outages left several Google Apps admins in the lurch — and many of them are second-guessing their advocacy for making the switch to hosted apps, InfoWorld reports. The outages, which affected both Gmail and Apps, 'could serve as a deterrent to some IT and business managers who might not be ready to ditch conventional software packages that are installed on their servers,' according to the article. 'If we began to experience a similar outage more than about two or three business hours per quarter, we'd probably make Google Apps and Gmail a backup solution to a locally hosted mail system, if we used it at all,' said one Apps admin. 'And it would likely be years before we'd try a cloud-based collaborative system again from any vendor.' Coupled with recent Apple and Amazon cloud issues, these Google outages are being viewed by some as big wins for Microsoft."
It is not a big win for Microsoft, it is a big win for corps hosting their own app servers. I would think that eventually Google will release google apps on a server that corps could install in their own data centers.
I scan Slashdot nearly every day and didn't remember seeing anything about outages at Google this past week. A search through the story history confirmed that fact. So I thought I'd visit google.com and see what Google itself had to say. Nothing on the blog; nothing in the press section.
So why is this the first time these outages have been discussed here? From reading the article it appears we're talking about multiple outages over the past couple of weeks. Doing a Google search for "google outages" brings up one blog posting about these recent events. The blog posting includes this unsourced quotation, "Google spokesman Andrew Kovacs said via e-mail that 'a small number' of Gmail users and 'some' Apps users were impacted by the problem, which is still outstanding and being worked on as of 5:30 p.m. U.S. Eastern Time on Friday."
So all these events seem rather shrouded in mystery. How big was the outage? What explanations did Google give for the outage? I've certainly had servers go down, lost network connectivity, etc., etc., but I don't maintain huge server farms with enormous redundancy and multiple high-bandwith connections to the Internet. I don't recall search on Google ever going down; what's up with gmail and Apps?
The suspicious among us might start to think that outside parties might be responsible. After all, if companies start migrating to the "cloud," disrupting those services could have a substantial, economy-wide impact.
We have all seen it. Ebay a couple of years ago going down due to Oracle corruption. Royal Bank of Canada failure due to an improper software upgrade. Now, Google with Gmail and other Google Apps failing. All of these organizations were geared towards having the highest uptimes available and failed spectacularly.
Whether you host your own or use someone else its the illusion of control that somehow clouds our judgment into believing that it would somehow be different if I did it. Example: Is it better to drive or fly? Pure numbers state that its safer to fly on a commercial carrier by an order of magnitude but somehow we feel safer when we drive. Whether we choose to acknowledge it or not the world is full of 6 sigma events. As long as you are doing everything you can and within your budget when your hosting your own apps or auditing your provider to ensure they have, backup systems, redundancy, offsite bunker, etc. then you have done everything you can to prepare for this inevitability.
In a lot of ways designing systems is like playing poker. You can play your hand perfectly, design all the systems redundancy and recovery you like, but sometimes even after all that your opponent (risk) draws a lucky card on the river to beat you. Just because you got beat doesn't mean you shouldn't continue to play the same way, it just means you hit one of those events that you cannot plan.
Expecting five-9 or 0 downtime for a system used by only ONE company might be a very high expectation with a high cost vs. usage obtained from it afterwards.
But how many companies rely on Google's systems? When you offer your application or suite to the whole nation or WORLD, and campaign for its use - then YES, you do need to keep a very near-0 downtime to be really successful.
Non-Microsoft can be :
I think the basic problem is impatience. I can understand that people want for business purposes something that is quickly implemented, but my experience is that when a Microsoft implementation is chosen, you have two long-term issues : you will time and time again have to solve the same problems over and over, and you can be sure that Microsoft will try to pressure you into upgrades, willing or not.
My experience with Linux and associated programs is this. Over time, everything gets better and better. Sometimes, you might need some time to investigate a problem and solve it, but once solved, it will not recur again (be sure that you have a good system to record such findings, but that would be same when using Microsoft).
I have already three people (not much, yes, but important for me) using Linux : my father, my brother (who shares with my father's PC) and my sister. Unless there is a hardware problem, I can be sure that I do not have to solve software issues on a regular basis, only help them with functional questions : what software to use and how to use it.
They use on a regular basis :
I am pretty sure that for most parts of a business, this would be enough.
Now, I think that the usage of Exchange is more of a perception thing, than a real technical obstacle. At my work, Lotus Notes was swapped for Exchange, but I do not consider this a progress, as it reminds me too much of PCTools 4.0 or 5.0 (about 1990) : I really do not see anything innovative in this area (and while some people here seem to loathe Lotus Notes (mostly without any reasons given), it was much faster than Exchange, I find speed very important for computer programs).
Anyone here which as implemented or is using alternatives to Exchange ?
I migrated my company of 80 users to Google Apps hosted email about a year ago, and yeah, sometimes there has been interimitent issues. People want to use it like Exchange via IMAP, but there are quirky issues, like Thunderbird sending the wrong delete command, Thunderbird somehow corrupting the user's password (the only way to correct is to login to the user's account on the hosted Gmail site), etc. So there definitely are some quirks sometimes.
That said, it's free. Somebody a few posts back posted the cost of an RHEL install with server costs etc. Using Exchange, the price increases even moreso (license costs, CALs, etc.). Ultimately, you're getting a hosted, web-based email solution with the capability for shared calendars and document collaboration, all for absolutely $0.00.
Free vs. $20k+ solution? In my oh-so-humble-opinion, users can deal with (and quite frankly, should continue to periodically expect) some downtime.
Our mail platform has beaten google in uptime and security "bugs" for the past 40 months. Why? I attribute it to using proven technologies and not everyone wanting an account being able to get one: we charge every system user. You would be surprised how much this cuts down on spammers/excessive usage.
Google has had their mail in beta for years. The last time I checked SMTP was ratified as an RFC over a decade ago.
Website Hosting
Do you honestly believe that you or your employees are going to build a system with higher availability than Google? In the magical fantasy world we all wish we lived in, you may have the budget, skill, manpower, and infrastructure resources to do this. In the real world it is not even remotely possible.
Do I believe it? You betcha! While my company doesn't have 100% uptime for every employee all the time, we haven't suffered an across-the-board outage of a critical system (i.e. email, ERP, core business applications, etc.) in the 11 years that I've been here. Sure, we'll lose an email server once in a while, but we have many such servers, so the loss of a single system only impacts a few thousand employees tops. That's far better than impacting ALL our employees if Google has an outage. And don't get me started about the idea of not being able to do word processing just because a WAN link is down. How on earth could you run a business that way???
And it's possible to provide uptime even in the event of widespread events, such as flooding, tornadoes, etc. We have multiple datacenters, geographically dispersed. Each center has multiple Internet connections through multiple providers carefully chosen such that the lines go to different cities (i.e. one link to Chicago, one to Denver). Similarly, our power is connected to multiple grids, with the feeds coming in on opposite ends of the buildings. Critical centers have on-site generators spec'd to handle 100% load of the datacenter and requisite support stations, plus enough battery backup to allow for all systems to continue running between loss of grid feed and when the generators are spun up, not to mention on-site diesel sufficient for several days of operations and contracts to get more as needed.
Was this cheap? Not in the least. Was it worth it? Definitely. We kept our main datacenter running without interruption during a week that saw multiple weather events (i.e. tornadoes, flooding, lightning-related power loss, etc.) when every building around ours for multiple miles was without power.
Precisely. And while I use Gmail most of the time, and the rest of my office use an Exchange server hosted in same said office, guess who has the better uptime...?
That would be me. They frequently (3-4 time a month) loose half a day, as the under resourced, high maintanence, auto-destructing, sorry updated, blackhat honey pot splutters in the corner. I've lost two half days in the however many years I've used Gmail.
Because you can - or because you should?