Extended Gmail Outage Frustrates Admins

CWmike writes "A prolonged, ongoing Gmail outage has some Google Apps administrators pulling their hair out as their end users, including high-ranking executives, complain loudly while they wait for service to be restored. At about 5 p.m. US Eastern on Wednesday, Google announced that the company was aware of the problem preventing Gmail users from logging into their accounts and that it expected to fix it by 9 p.m. on Thursday. Google offered no explanation of the problem or why it would take it so long to solve the problem, a '502' error when trying to access Gmail. Google said the bug is affecting 'a small number of users,' but that is little comfort for Google Apps administrators. Admin Bill W. posted a desperate message on the forum Thursday morning, saying his company's CEO is steaming about being locked out of his e-mail account since around 4 p.m. on Wednesday. It's not the first Gmail outage. So, will this one prompt calls for a service-level agreement for paying customers? And a more immediate question: Why no Gears for offline Gmail access at very least, Google?"

Comments

The benefits of cloud computing

[mlwmohawk]

Someone else deals with all the problems, right?

Re:The benefits of cloud computing

[Thelasko]

Remember folks, it's still in beta!

Re:The benefits of cloud computing

[ionix5891]

yeah i was getting "invalid password" messages all day in outlook with gmail hosted accounts

Re:The benefits of cloud computing

[jellomizer]

Yes. In your organization how many times have your servers went down or had a problem... Compare that to Google Mail... You will probably find that there is a lot less downtime. The problem is just like flying on an airplane. You are statically safer flying an airplane then driving. However because you fate isn't in your control you feel more scared then if you could just drive there yourself. The same thing with SaaS models, you actually get better service however because you don't have the same amount of control you feel like it is riskier. But it isn't

Re:The benefits of cloud computing

[an.echte.trilingue]

As somebody who admins google apps in a business environment, I can say, that is not what they have to offer. What they have to offer is up-time that is better than what internal solutions could ever possibly offer at a price an internal solution could not pray to beat. Is it 100%? Is it free? Nope, but neither is the exchange server in the basement. Do I control my data? Nope, but realistically the alternative would be to contract my data storage out to somebody else anyway.

Bill W. is probably taking heat because he sold google apps to his superiors as having 100% uptime with no disadvantages, which of course it does not.

Re:The benefits of cloud computing

[electrictroy]

That's no excuse. When I need a word processor, I need it NOW, not tomorrow. I do not want my software to be dependent upon anything except my Cl drive. No net connection required.

Re:The benefits of cloud computing

[EraserMouseMan]

If Google doubts it's readiness for mission-critical usage it gets a "beta" slapped on it. Do real professionals actually think Google Apps is ready for prime time usage?

Re:The benefits of cloud computing

[mlwmohawk]

because you don't have the same amount of control you feel like it is riskier. But it isn't

I have a real problem with "cloud computing" and the lack of control is just once piece. With google, there is no assurance that *my* problem is being worked on. *My* problem will get handled in the order in which it was reported. (if at all) To me, MY problem is the most important problem.

The problem with "cloud" computing, and probably the biggest IMHO, is the importance of "you" and your interests to the company providing your service. Suppose that you build your own business on a company providing virtual machine services. All is going well, you are profitable, and poof!! they decide to drop the service because it isn't profitable for them. What if they see what you are doing and say "hey, that's a great business idea, how does he do that, lets look at the code." and so on.

I could go on, but there is a lot to be said about "building" your own business, and my rule of thumb is: "Committing to a single vendor lock-in, in the long run, will always be worse than doing it yourself."

Re:The benefits of cloud computing

[rennerik]

And also, I suppose, much like an airplane, is that when something happens, there's statistically a greater chance of a disaster.

So, I suppose, in this case a full day of downtime is pretty disastrous.

Go airplanes. And Gmail.

Re:The benefits of cloud computing

[jellomizer]

If you paid a marginal fee. Then you are a paying customer, and your problem gets priority. If you just using a free service, then you are not that important, as you got it for free.
That being said. I have worked with many IT shops if the Email goes down the IT Guy is rarely interested in any ones persons emails but the server. What is the difference.

Re:The benefits of cloud computing

[Jerry Beasters]

So we're voting things insightful that are patently untrue now?

Re:The benefits of cloud computing

[Sancho]

Statistics aren't magical. It's entirely possible that a safe, conscientious driver is safer driving than flying (I haven't seen any statistics which break it down that way before.) There are a whole lot of considerations that need to go into a statistic like that for it to have any real meaning.

Re:The benefits of cloud computing

[jmelchio]

I'm not sure what to think of the comparison of flying and driving statistics. Those statistics assume every driver has the same chance of getting into an accident as any other driver. This is obviously not the case (ask insurance companies) because some drivers are simply a lot better than other drivers even if they don't control all of the environment factors.

I assume the same can be said about system administration. It depends on the organization. Some organizations are very good at it and might do as well or better than Google. Others, well, you get the picture.

Re:The benefits of cloud computing

[twistedsymphony]

Something tells me that an email system that only depends on your hard drive with no net connection wouldn't be very useful.

FWIW I don't seem to be having any problems getting gmail through my gmail-lite install.

Re:The benefits of cloud computing

[xant]

I started out as a personal gmail user. I was very happy with it, even routing my work email through it, but when the question for our mid-sized business came up, "should we outsource our email to gmail?" I said no. I said let's do due diligence, there are other outsourced solutions, this is something we really ought to get right.

Our CEOs (we have two, yeah..) both tried it and liked it, so we went with it.

So I'm in the unique position of having argued to management that we shouldn't risk anything on Gmail, and us doing it anyway because management wanted it. And you know what? I was wrong. Gmail has been a great productivity booster for our business, it's saving us money on salaries, and the downtime is less than we experienced when we were half-assedly running it ourselves.

Plus, when shit does hit, I just smile, and nobody tries to blame me. :-) On the ~two occasions that we had any noticable gmail outage, our CEOs weren't the ones complaining. They have realized that email may be important, but we can still get work done while gmail is futzing around with it.

Re:The benefits of cloud computing

[MightyYar]

Take a step back for a moment... Your email is down for a day. So what? For most people, this is really no big deal. Especially if it only happens once in a blue moon.

Have you never experienced a local failure of any kind? You never had a day without a computer before? Your local net connection never went down? Never had a power failure?

I'm sure some guy on here has some worst-case, niche scenario where losing email for a day means the end of the world - but for most of us, it just means people who REALLY need us use a phone. If your email is so important, you need some kind of redundancy that a single provider is just not going to ever give you. You'd better have a hell of a data center with redundant connections through multiple providers.

Re:The benefits of cloud computing

[A non-mouse Coward]

You are statically safer flying an airplane then driving

That's true, I always get shocked while riding in cars. Can't remember a time in a plane, though. Must be that the plane is off the "ground".

Re:The benefits of cloud computing

[jellomizer]

Well even if you are a good driver, chances are not everyone around you is, thus you can still get rear ended, or you car may malfunction at the wrong time. In Airlines, you have in general no one around you and it is a very stilled pilot flying the plane for you, and a crew to make sure the plane is in good conditions.

Re:The benefits of cloud computing

[elrous0]

My status as a Google shareholder is in beta too.

Re:The benefits of cloud computing

[cailith1970]

The problem I think isn't the fact that there is down time, the problem is that when you're performing internal maintenance, you can choose the best time to do it by coordinating with everyone else in the organisation. When downtime is imposed with little or no warning externally (or simply just goes down "for maintenance"), that's when the online model comes unstuck and people get frustrated.

Re:The benefits of cloud computing

[AncientPC]

Messaging application features

- Gmail and Google Calendar

- Gmail storage: 25 GB / account

- 99.9% Gmail uptime guarantee**

** The 99.9% uptime guarantee for Gmail is offered to organizations using Google Apps Premier Edition, as described in the Google Apps Premier Edition Terms of Service.

Re:The benefits of cloud computing

[ajs]

Indeed they do, and since I've never worked for a company that has had zero email outages (including my own home mail server before I moved to google.com/a), I'd have to say that my Google Apps experience has been as good as or better than the experience I've had with any other mail service that I've used for at least a year.

Re:The benefits of cloud computing

[elrous0]

As a friend of Bill W., I can tell you that he probably really needs a drink right now. God knows I do.

Re:The benefits of cloud computing

[MightyYar]

With google, there is no assurance that *my* problem is being worked on.

While that is true, it is also true of your electricity and net connection. And any other utilities feeding your building that are critical to your business.

For most businesses, losing email for a little while is nothing compared to a snow day. It just means more telephone calls. And probably more productivity :)

Re:The benefits of cloud computing

[tolan-b]

Google Apps is a hosted service sold to businesses, Google are meant to provide that redundancy, and in theory they should be in a much better position to provide it than most small to medium size companies' IT departments.

In practice they seem to be sucking a bit.

Re:The benefits of cloud computing

[fremsley471]

An interesting comparison. How do the risks of death appear in the general driving accident figures if you're not drunk/tired/stupid? Extrapolate for mail server admins.

Re:The benefits of cloud computing

[afidel]

Yes. In your organization how many times have your servers went down or had a problem... Compare that to Google Mail... You will probably find that there is a lot less downtime.

Sorry, but the total downtime I've ever caused ALL of my employers over my career has been a LOT less than 28 hours! Heck, even if you add up the downtime for all of the single systems I've admined their collective downtime is probably only close to that. I'm not bragging, I'm pointing out how bad of an outage this is. The only other outages I've personally heard of that were this bad are hosting providers who have critical systems physically damaged and a failed Exchange 2000 pilot at Cisco (They had a corrupted datastore that was so bad that MS and HP and EMC couldn't recover it so they had to fall back to a tape restore which took something similar to this gmail outage)

Re:The benefits of cloud computing

[Rakishi]

Yet amazingly enough planes crash. They crash quite often in fact. In fact from what I remember the chance to die per hour of travel is roughly the same between airplanes and cars. In other words the chance of dying from some random outside event is probably much higher in an airplane per hour of travel. So yes, a safe driver is much less likely to die in a car than in an airplane.

Re:The benefits of cloud computing

[Moebius Loop]

Honestly, I've maintained my own mail server for 5 years, and my company's corporate server for 2 years, and I can count on one hand the number of times either of them have failed in that period. When they did fail (because I was being irresponsible about configuration changes, or hardware failures, etc), there was pretty much no way I was going to be getting in bed before I got them back up.

Granted, I don't have millions of users and petabytes of email. But I also am not any kind of real system administrator, I don't have a massive redundant data storage facility, and neither do I have millions of dollars and endlessly brilliant engineers working at my beck and call.

Some GMail downtime is, of course, to be expected. But these kind of high-profile outages from Amazon and Google are truly shocking. I don't think it puts the nail in the coffin of SaaS by any means, but it does indicate a significant necessity of SLAs for paying customers.

I would desperately love to divest myself of the responsibility for these mail servers, but I want to know that I can trust GMail's response time during crisis as much as I can trust my own.

Re:The benefits of cloud computing

[tolan-b]

+1 "Oh god it burns"

Re:The benefits of cloud computing

[Anonymous Coward]

What's a chlorine drive?

Re:The benefits of cloud computing

[wfeick]

If you paid a marginal fee. Then you are a paying customer, and your problem gets priority.

It's not actually that absolute. What is your priority relative to all the other customers who are competing with you for finite rexources? The reality is that more profitable customers may move through the customer support phone queue faster than you, and their issues may be addressed before yours.

Also, it's not uncommon, particularly after a company acquisition, for customers to be reassessed and prioritized according to profitability. Companies decide to cede whole markets to their competitors if they're not sufficiently profitable or they decide to go in a different market direction.

The customer is often not told they are no longer a priority, they just find their rates go up, the quality of their support goes down, their packets are routed through over subscribed network fabric, etc.

The company won't actually tell the customer to go away; they'll continue to accept money until the customer figures it out and goes away on their own.

Re:The benefits of cloud computing

[Kleen13]

So what????? Well, I can say that my feet would be up on my desk for the better part of the day, and I'd finally get caught up on all that sleep I've been missing... Ok, thats a lie. I would have ALL of management breathing down my neck until I "fixed" the problem at Gmail. All the production people on the floor would have feasts in my honor, and everything would come to a grinding halt. In my opinion, there are too many hops that I don't control to rely on hosted services. So far. I'd sure love to free up all that cash we spend on cals and office licences though.

Re:The benefits of cloud computing

[SaDan]

Sorry, but I've worked in plenty of organizations where our email didn't go down unexpectedly for days on end. Not at all difficult to keep a lowly email server from going tits up.

This is also the SECOND extended outage for Google this year.

Smaller IT departments > cloud computing if you absolutely want to have uptime. Sure, it costs more for the services provided, and there is more to manage, but if you need uptime, you pay for it.

Re:The benefits of cloud computing

[Kleen13]

I agree. All the downtime we have on our Exchange server is administrative.

Re:The benefits of cloud computing

[Bozdune]

...not everyone around you is [a good driver]...

That's certainly true. But since we all believe that we are magnificent drivers, we can make some important generalizations. First, let's erase all the driver deaths in single-car accidents, because we wouldn't get into that accident in the first place. Then let's erase all the driver deaths in multi-car accidents where the driver did something really dumb, like run a red light, because we would never do that. Then let's erase all the driver deaths that occur after midnight (because we don't drive after midnight, when statistics show that as many as 1 of 3 drivers are drunk). Then let's erase all the driver deaths where the driver was impaired by drugs or alcohol, because we don't drive under the influence. Finally, since we are slashdot geeks and we care about how things work so we maintain our equipment properly, let's erase all the driver deaths due to equipment failure.

Now show us that our risk of being killed on the road -- by SOMEBODY ELSE when WE ARE DRIVING COMPETENTLY and where OUR DEATH WAS NOT AVOIDABLE -- is higher than on a mileage basis than climbing on an airplane.

Re:The benefits of cloud computing

[Sancho]

Sure, but a good driver who takes care of his car can avoid most accidents, and even if they can't, they can mitigate damage in some ways. Where you're driving (to, from, and which road) will also make a difference. My whole point is that you can't just say "Flying's statistically safer than driving, so you shouldn't be as afraid of it" (which I realize, of course, you did not exactly say) because individual circumstances can have a huge impact on the actual safety of your drive.

Re:The benefits of cloud computing

[Firehed]

Yeah, that's usually what I was saying about email when the exchange server at my previous employer went down every third week or so. Or when some rodent chewed through the line that handled the VOIP. Or when some transformer down the street got plowed by some idiot driver causing sporadic access to the mains for half a day. Word processors in the cloud may not make a ton of sense right now, but email is fundamentally useless without every machine in the chain working properly.

You try running your software without a power connection and let me know how it goes for you. Laptop batteries don't last that long, and desktop UPSs even less so (assume that the generator, if present, can only keep the servers online indefinitely, not the whole building).

Gmail being down for a few hours is a minor inconvenience at worst. If your dirt cheap or free and completely awesome email being unavailable for two hours a year is causing you to lose business, then you seriously need to rethink your operations. You have a landline, a cell phone, and a fax (among others), and if those are all also out of commission then chances are you've got bigger problems. You know, that mushroom cloud hovering overhead.

Re:The benefits of cloud computing

[Anonymous Coward]

Isn't that a chlorIDE drive?

Re:The benefits of cloud computing

[reboot246]

I've not had any problems with my password, "invalidpassword".

Re:The benefits of cloud computing

[SaDan]

My home basement server and the servers I admin where I work have no where near the downtime Google has had this year, including the time I take them offline to do core software updates (all scheduled and late at night).

Is my solution cheaper? No. Is my uptime noticibly better than Google's? Absolutely.

The company I work for relies on email, and we would have serious problems with ANY downtime. If we were on Google (and we did look at using their services a while back), we would be losing customers and money every time they go down.

Re:The benefits of cloud computing

[Anonymous Coward]

FFS PEOPLE!!! IT'S FUCKING "THAN"!!!

"IT'S SAFER THAN DRIVING"

Filter error: Don't use so many caps. It's like YELLING.

BECAUSE I AM FUCKING YELLING!

Re:The benefits of cloud computing

[sribe]

It most certainly is riskier. If you own your computers & data, and your company goes out of business, you no longer need access to that data. If you SaaS provider goes out of business, you probably still need that data.

Re:The benefits of cloud computing

[Anonymous Coward]

Statistics don't mean *you* will get better service with SaaS - it means the average user will. If your company's IT department is above average, then it's very possible that you'll get better service and less downtime than with Gmail.

Re:The benefits of cloud computing

[Anonymous Coward]

This sentence makes no grammatical sense:

You are statically safer flying an airplane then driving.

Unless you meant to say that you are STATICALLY safer flying an airplane THEN driving. But why would you say that?

Perhaps you wanted to say:

You are STATISTICALLY safer flying in an airplane THAN driving.

Which may be true but of little consequence to the 150 people in the plane wreck.

Re:The benefits of cloud computing

[DragonWriter]

Yet amazingly enough planes crash. They crash quite often in fact. In fact from what I remember the chance to die per hour of travel is roughly the same between airplanes and cars.

Looking at the same data in another (and for more relevant to any useful comparison) way (and assuming it is completely accurate), the chance of dying per mile in a plane is a small fraction of that in a car. Rarely does anyone face a decision of the form, "Given that I need to spend N hours traveling, should I take a car, bus, train, or airplane." Rather, they are more likely to decide "Given taht I want to get from Point A to Point B, should I take a car, etc." So, really, the risk of death per hour of travel is pretty much irrelevant.

Re:The benefits of cloud computing

[Belial6]

People keep claiming that Google has better uptime than in house systems. I have to ask, what kind of monkeys are administering these systems? In the last decade, we have not had a single unscheduled email server outage. We haven't even come close to 28 hours of SCHEDULED downtime. Heck, my personal mail server hasn't come close to 28 hours of unscheduled downtime in the last decade. There just isn't that much that can go wrong on an even half assed administered email server. I get why it would take longer for Google to restore their backups, but that is just a weakness of centralized data storage. It's everybody trying to get a restore at the same time so they have to stand in line. That is just one reason that gmail is only good for personal email.

Re:The benefits of cloud computing

[aztektum]

FWIW I don't seem to be having any problems getting gmail through my gmail-lite install.

Likewise via IMAP on my WinMo 6.1 phone. In fact I didn't even know there was an outage until just now; my mails been coming in fine.

Re:The benefits of cloud computing

[Bryansix]

That gmail-lite project appears dead. That or they just forgot about the blog since 2005.

Re:The benefits of cloud computing

[Bryansix]

The exchange went down because once the database corrupts you are supposed to move all of the email out of it and into a new one.

Another thing... my faxes come to my email.

Those two things aside though I have seriously thought about moving our company to gmail.

Re:The benefits of cloud computing

[swimsaturn]

You are statically safer flying an airplane then driving

so the drive to the airport is more dangerous than the drive from the airport

Re:The benefits of cloud computing

[Bryansix]

Uhm, and in which case is doing it yourself not going to result in Vendor Lock-In?

Re:The benefits of cloud computing

[Rakishi]

That's assuming the choices are "go from A to B using method X or Y" instead of "should I stay home or fly from A to B."

Re:The benefits of cloud computing

[Anonymous Coward]

The question of whether YOUR problem is being worked on is ONLY relevant because it pertains to the total downtime that you experience. If an outside solution manages to provide better uptime than you can internally, order of resolution is irrelevant. Unless, perhaps, you are affected more by rarer, longer outages than by more frequent, shorter ones, or something along those lines.

In fact, even using an in-house solution, you still have exactly the same problem -- it's important to remember that in most cases the people actually working on tech problems and fixing issues only exist to allow others to do the "real" work. The important perspective to look at this from is not the admin's, it's the general user's. When that user has a problem, he doesn't have any assurance that HIS problem is being worked on, either. Maybe some other user has preference.

I'm not saying that cloud computing is a great idea, and the point about the possibility of the service provider just vanishing is certainly very valid and very important -- and, IMO, a deal breaker in the case of critical services. Still, I feel that you're talking about some issues that don't actually exist except as perceptions.

Re:The benefits of cloud computing

[Tony Hoyle]

Been using mine all day with no problems.. clearly the outage didn't affect everyone.

Re:The benefits of cloud computing

[Mista2]

Then think carefully about this before you rely on the cloud. Sometimes it's sunny and there is no cloud 8)

Re:The benefits of cloud computing

[Kalriath]

I don't think it puts the nail in the coffin of SaaS by any means, but it does indicate a significant necessity of SLAs for paying customers.

There is an SLA for paying customers, but with Google's track record they really need to up the penalties for non-compliance.

Re:The benefits of cloud computing

[Anonymous Coward]

Bill W. is probably taking heat because he sold google apps to his superiors as having 100% uptime with no disadvantages, which of course it does not.

Or because they heard what they wanted to hear, instead of what he was actually saying. In my experience, one happens just as much as the other.

Re:The benefits of cloud computing

[Mista2]

We can also tell our users when there is going to be an outage, what we are doing to fix it and what the expected ETA is.

Re:The benefits of cloud computing

[street struttin']

Something like this, offering 24/7 uptime, should be configured in such a way that routine maintenance never ever requires any amount of downtime. If failover and redundancy are configured properly, this is easily doable.

Re:The benefits of cloud computing

[kestasjk]

The worst thing is the update to iGoogle (which is extensive and undoubtedly the cause of the outage) is quite a step backwards. It's a pretty clunky hybrid of a window based system and a widget based system, with a lot more AJAX and a lot fewer clickable links.

Re:The benefits of cloud computing

[electrosoccertux]

How do you complain to google? I have a phone number I can call if the electricity goes out. I want a phone number, or a chat service at least, that ensures me they know of the problem and are working on it.

Re:The benefits of cloud computing

[pushing-robot]

so the drive to the airport is more dangerous than the drive from the airport

Yes. I'm usually running late.

Re:The benefits of cloud computing

[Bryansix]

DO you have real-time backups? Hard drives do fail. They have moving parts you know. Most of your solid uptime is luck. Part of it is planning. Still mostly luck.

Re:The benefits of cloud computing

[Anonymous Coward]

I do not want my software to be dependent upon anything except my C: drive. No net connection required.

Different strokes. If you were traveling to see a client and your laptop got stolen, you might see the upside of having your documents online. Viewable and editable whether on a free library terminal or iPod Touch.

I can't count the number of times myself and co-workers have mislaid a USB thumb drive. Can I just VPN in? No, that's why I have the damn thumb drive.

And I use Photoshop, so there's no promise that my client will have a spare terminal in their office with the latest version installed. I guess that's why Adobe is shooting for online apps too.

Re:The benefits of cloud computing

[Cramer]

Right. And the logic process of car vs. plane comes down to how long it takes to drive vs. fly. How much it costs to drive vs. fly. And the hastle of flying vs. driving. While distance is the bottom line factor, time is what people consider.

Re:The benefits of cloud computing

[Anonymous Coward]

Yes, because I ALWAYS take my servers down for 24hrs + during business hours....

Re:The benefits of cloud computing

[electrictroy]

Well, that's partly true, but not totally. When my Verizon DSLAM decided to burn-up and knock out our neighborhood, I was still able to work the email through my Netscape Phone line connection. A 50k dialup works just fine.

Working on the military, I've learned not to rely on just a single source, which is the main problem here: People don't have a backup for when Google goes down.

Re:The benefits of cloud computing

[Ihmhi]

You are statically safer flying an airplane then driving.

I don't know about that, there's a whole lot of lightning up there where planes are flyin' around.

Re:The benefits of cloud computing

[brasscount]

Let's see, so 99.9% uptime...
Hours in a year: 8760
99.9% of hours in year: 8751.24

Hours in a year that Google has lost in this incident: >27 or 99.691780821917808219178082191781%. Lets round to 99.692% (I'm feeling charitable.)

Wait, we just said 99.9% uptime. Really we meant 99.9% uptime every ten years.
So, 99.9% of 87,600 hours allows us 87.6 hours out of service. No problem boys, we've got two weeks to get this thing working!

Re:The benefits of cloud computing

[orangenerd]

I could go on, but there is a lot to be said about "building" your own business, and my rule of thumb is: "Committing to a single vendor lock-in, in the long run, will always be worse than doing it yourself."

Have you noticed that if you do it yourself, you are committing to a single vendor?

Re:The benefits of cloud computing

[Cramer]

I have to agree here. I've worked for several ISPs, telcos, and software companies over the years, and I've never left anything this critical (effecting so many people) down for so long. No mail server I have ever managed as been down for more than a few hours -- and only then because there was no power in the building. The system mastering server at my current employer is the longest I've left anything down, but it wasn't stopping anyone else from getting work done. (raid and mirrors won't help when a bug in the filesystem screws things up and an oversight in the backup software (default) configuration meant it ignored a fair amount of Very Important Files (directories named core.) It needed rebuilding before it failed anyway. Dead servers are good budget approval makers. *grin*)

Re:The benefits of cloud computing

[Anonymous Coward]

email is fundamentally useless without every machine in the chain working properly.

Ummm, no. SMTP was designed a long time ago when computers & networks were much less reliable than they are now. You can have multiple email servers on multiple internet providers on multiple backbones for outstanding reliability and graceful failover.

Of course, you do have to configure the redundancy - it doesn't magically appear.

Re:The benefits of cloud computing

[meeotch]

Nope - active as of 8 months ago. See CVS.

But not *that* active.

Re:The benefits of cloud computing

[home-electro.com]

Exactly. Only dumb admin's would use gmail for their corporate email. If only for one reason -- by doing so you outsource your own job. That is dumb.

Re:The benefits of cloud computing

[Anonymous Coward]

"Committing to a single vendor lock-in, in the long run, will always be worse than doing it yourself."

So, what do you do for electric power?

Re:The benefits of cloud computing

[Mister Whirly]

Yeah. If there is one thing businesses hate, it is CUSTOMERS.

Re:The benefits of cloud computing

[Cramer]

As a wild generalization, I would agree. However, experience has shown that such a generalization is merely that. It's not hard or expensive to setup a mail server that works perfectly with virtually zero downtime without needing a team of engineers constantly maintaining it. The linux (sendmail/cyrus) mail server I built 5+ years ago is still running perfectly fine with no non-power related disruptions, ever. And nobody has touched it in over a year. The other Communigate mail servers I've built over the years are the same way. As long as the hardware doesn't break, they just keep on going.

Btw, the very reason that mail server was built was because of horrible experiences with outsourcing email. If you run it, performance and reliability are in your hands. Outsourcing puts you at the mercy of your provider -- who can provide shitty service, collect your money, and close up shop.

Re:The benefits of cloud computing

[hvm2hvm]

POP3 on thunderbird is working fine for me.

Re:The benefits of cloud computing

[Bandman]

It's an issue for personal customers, but for corporate entities who have outsourced their email to Google, it's a bit more than an irritation. For the admin who decided to outsource the mail to there, it could be a Resume Generating Event

Re:The benefits of cloud computing

[pherthyl]

Uhh.. No. Drive from Vancouver to Halifax, you're in the car for a few days. Fly that distance and it's a few hours.

Risk/hour is completely irrelevant when comparing the safety of airplanes to cars. When was the last time you took a week long plane trip?

Re:The benefits of cloud computing

[Bandman]

You've got to love IT, where one-in-a-million events occur every other day...

Re:The benefits of cloud computing

[Belial6]

No. You just set up a replica, or if you have the need a simple cluster on a second machine and have a secondary MX entry. If one system drops, everything continues to chug along on the second machine. At most, the user should see a message telling them that the primary server has failed, and they are being redirected to the secondary server.

With this simple setup you get 99 with a lot of .9s uptime, and if you want to add 9s all you have to do slap in more machines. As a bonus, by adding the extra machines you can get load balancing as a bonus.

This isn't rocket science.

Re:The benefits of cloud computing

[Bryansix]

I'm talking about the storage of the email not the routing of it. I already have redundant mx records with my outsourced spam filtering service.

Re:The benefits of cloud computing

[xaxa]

The same can also be said about flying -- the professional pilots working for commercial passenger airlines are less likely to get into an accident than the rich guys with light aircraft. They're better at landing in fog too.

I think the GP should have used train vs. car to save this discussion.

Re:The benefits of cloud computing

[Bandman]

I'm getting ready to move my company to a hosted exchange provider.

It took a while to decide, but I submitted a proposal to host it with Apptix. I spoke with an engineer, and it sounds like they've got a very reliable clustered backend. Add their Blackberry services and compliant archiving, and it sounds like a good solid solution.

Of course, two years and I might change my mind, but that's the risk you take.

Re:The benefits of cloud computing

[Bandman]

"so what" isn't a valid option for corporate clients, which they sell to.

Re:The benefits of cloud computing

[xaxa]

You are statically safer flying an airplane then driving

so the drive to the airport is more dangerous than the drive from the airport

I'm not sure about that one, but in London you're safer on your train journey than crossing the road outside the station.

(e.g. except for the terrorism on 7/7, the last death from a London Underground accident was in 1975. Before that 1958. The heavy rail isn't quite so safe, but still -- about 100 pedestrians are killed in London each year. I bet more money is spent trying to improve rail safety than road safety though.)

Re:The benefits of cloud computing

[Allnighterking]

So let me get this straight. You run windows (hence the C drive) and you think that your e-mail client is a word processor. Is this correct?

Further I gather from your note that you feel that others need to run the same thing you do because you say so. Perhaps you might consider that many other people (self included) have found ways to be connected to things like Gmail 24/7 almost anywhere we go, (In the middle of Yosemite I did have problems early this summer. Dang bears.) Without lugging a full blown "C Drive" around with all of it's umbilical cords.

Re:The benefits of cloud computing

[twistedsymphony]

yeah glite is on life-support in an official capacity, but there is work being done here. Though it's mostly just enough to keep it functional with the changes that google impliment.

It works well enough that I can run it from my own server and access my email in places where gmail might be blocked or otherwise difficult to access directly. which is all I need it for.

Re:The benefits of cloud computing

[Belial6]

So, what part of the rest of my post doesn't explain how even the simplest of admins can have close to 0 downtime?

Re:The benefits of cloud computing

[xaxa]

You are STATISTICALLY safer flying in an airplane THAN driving.

Which may be true but of little consequence to the 150 people in the plane wreck.

What about the 40000+ people killed on the roads *in the USA* that year?

Re:The benefits of cloud computing

[Kugrian]

The worst thing is the update to iGoogle

Dear god yes! After 3 years of having a perfect homepage, I now have to change, as the new one no longer fits my needs. I'm all for change, but a 'Use original IG' option would be nice. Even if it wasn't updated and newer applications would no longer work. My perfect homepage is now destroyed :(.

Re:The benefits of cloud computing

[Bandman]

As long as it's a planned outage. Otherwise, you'll be letting them know as you're trying to fix it.

Re:The benefits of cloud computing

[Bandman]

Out of curiosity, what did he say that was untrue?

Re:The benefits of cloud computing

[DocSavage64109]

Well, the Exchange server where I work reboots on a nearly weekly basis for the seemingly continual Microsoft patches. That has got to be bad for uptime stats.

Re:The benefits of cloud computing

[chis101]

FWIW I don't seem to be having any problems getting gmail through my gmail-lite install.

Likewise via IMAP on my WinMo 6.1 phone. In fact I didn't even know there was an outage until just now; my mails been coming in fine.

There isn't an indication of whether or not this is part of the outage, but, as said in the article,

Google said the bug is affecting 'a small number of users'

. Chances are you aren't one of the affected users anyways (I just use the web interface and haven't had any problems).

Re:The benefits of cloud computing

[WitchDoc83]

Re: Gmail-lite - FYI, your link is to a blog site with comments closed since 2005 and dead links to both gmail-lite and libgmail development sites...

Re:The benefits of cloud computing

[Bryansix]

I just glanced at what they have. It is nice but it's on the expensive side.

Re:The benefits of cloud computing

[Threni]

> Yes. In your organization how many times have your servers went down or had a problem...

That's one question. Another is "How long did it take them to fix?". Yet more include "did you get a more or less instant description of the nature of the problem, allowing you to predict the time it'll take to fix", "can I use a backup temporarily","how can we prevent this happening again" etc.

Re:The benefits of cloud computing

[MightyYar]

I hear you, and I agree that they blew it... I'm just commenting on the "sky is falling" mentality that seems so prevalent here - as if using Google Apps is the Single Worst Business Decision that one could make.

Re:The benefits of cloud computing

[Bryansix]

What part of my post didn't explain how even with close to 0 downtime you STILL might have data loss?

Re:The benefits of cloud computing

[mollymoo]

An uptime guarantee or SLA doesn't mean you'll get that level of service, it means you get compensation when you don't. The level of compensation virtually never represents the cost to your business of an outage and often doesn't even represent a significant incentive to the provider to meet the guarantee.

The GMail SLA is 99.9% on a monthly basis, so more than around 40 minutes in a month is a failure. This outage takes GMail's availability below 95% for the month, which qualifies for the maximum compensation under Google's SLA - 15 days free service. If GMail was out for three weeks you'd get the same. You have to claim it, it's not automatically credited. With such minor penalties, GMail's SLA is worthless. It doesn't compensate you adequately for the loss of service and doesn't provide an adequate incentive to Google to meet it.

Re:The benefits of cloud computing

[SaDan]

Obviously, I don't run Exchange. I run Zimbra on a dedicated (stripped down OS) server, so updates are very minimal, and very quick to apply.

Re:The benefits of cloud computing

[zooblethorpe]

Re-read the parent post. What businesses hate isn't customers, it's unprofitable customers. You know, the ones that cost more to provide services to than they bring in in terms of revenue.

Cheers,

Re:The benefits of cloud computing

[Belial6]

Yes, you might have a 1 in 100 trillion chance of losing data every century or two. I explained very simply how you get as many 9s of uptime as you want. I certainly hope YOU are not an admin if you don't understand that.

Re:The benefits of cloud computing

[Bryansix]

Boy you are dumb. I know I can get a million 9s if I just throw money at the problem. Luckily for my company this system admin has a business degree and knows that throwing money at 9's is not a very bright idea.

Re:The benefits of cloud computing

[MightyYar]

Pay them money and you'll get a support number.

Re:The benefits of cloud computing

[MightyYar]

They should pay for the lapse in service, and that is fair. I was just saying that - in a business sense - losing a day of email connectivity is not the end of the world for most businesses. I mean, just this year I can think of a Sprint outage and at least one "Exchange is down for emergency maintenance" message. Last year there was a power outage and the generators didn't kick in. Every year there are snow days. Business moves on...

Re:The benefits of cloud computing

[rdoger6424]

switch to IMAP!

Re:The benefits of cloud computing

[Anonymous Coward]

Committing to a single vendor lock-in, in the long run, will always be worse than doing it yourself.

I understand all the concerns, but its worth noting that you are not committing to a single vendor lock-in. A Google Apps customer can choose to switch vendors or run their own mail server, as long as they control the domain name.

Re:The benefits of cloud computing

[Anonymous Coward]

It's very cost effective and frees up time to concentrate on other parts of your job.

If your job is only as a email admin, then you cut out 75% of the work. Not dumb, but you might get bored.

Re:The benefits of cloud computing

[aztektum]

When I first read the story I was not able to get in via the web interface. The progress bar filled up then sat there. I left it for a few minutes and tried a refresh, same thing. It's fixed now, but I was having the issue.

Re:The benefits of cloud computing

[DragonWriter]

That's assuming the choices are "go from A to B using method X or Y" instead of "should I stay home or fly from A to B."

Since the comparison was being driving and flying, only the former choice is even remotely relevant.

The absolute risk of the best method available is relevant to the latter choice, of course.

Re:The benefits of cloud computing

[MadnessASAP]

Yes well inside computers, billions of event occur every second. Hell something as insignificant as a cache miss is a one in a million event.

Re:The benefits of cloud computing

[MadnessASAP]

Hey so did I, buty of course I didn't bring a PC, laptop, cellphone, watch or anything more complex then my 20 year old camping backpack.

Re:The benefits of cloud computing

[TheLink]

"I bet more money is spent trying to improve rail safety than road safety though"

Rightfully so, because it's better for the species.

How many genes can get selected for "fitness" by crossing a busy road or driving a car, compared to choosing to sit in a train or plane? :)

Re:The benefits of cloud computing

[Anonymous Coward]

How are sites slashdotted when nobody reads TFAs?

They just want to look at the pictures.

Re:The benefits of cloud computing

[Anonymous Coward]

Uhh.. No. Drive from Vancouver to Halifax, you're in the car for a few days. Fly that distance and it's a few hours. Risk/hour is completely irrelevant when comparing the safety of airplanes to cars. When was the last time you took a week long plane trip?

Suppose you were deciding where to take your vacation. You've decided you can either fly from Vancouver to Halifax, or you can drive to some lake a hundred kilometers from Vancouver. Suppose each trip takes the same number of hours. Then why do you say that risk/hour is "completely irrelevant"? Obviously you can say that it takes a long time to drive from Vancouver to Halifax compared to flying. But what about if you compare trips of the same amount of time instead of the same distance?

In some of cities, it can take longer to get to the airport than it takes for the plane to fly to a different city. And you generally have to drive or take a train or bus to get to the airport. Most people don't fly a plane from their house to the airport.

So sure, you can say it's safer to fly a plane from Vancouver to Halifax. But what about trips with the same number of hours on a plane or in a car? (Probably trips to different places). Also, you don't normally get searched and x-rayed when you go for a car trip. To get on a plane you need to get rid of all your metal objects, submit your luggage to x-rays, maybe submit yourself to a radar scan which can see through your clothes, discard all liquids in a container over 100 mls (WHY?), not be on a magic 'no-fly-list', maybe have your laptop seized and searched, and generally get the government's permission.

Re:The benefits of cloud computing

[TheLink]

There certainly are badly run IT depts out there.

1) In some places, the email systems on average fail twice a week. BUT they don't go down for 20 hours. If an email service goes down unscheduled for 10 minutes every day it is not equivalent to the email service going down for 2.5 days once a year.

If your heartbeat is late by 20 milliseconds all year, you probably won't notice. If it's late by 7 seconds once a year, it might only be late once in your lifetime ;).

2) Also for small/medium sized companies, a disaster big enough to bring down an email server managed by a semi-competent IT dept for 20 hours is often a disaster big enough to _independently_ bring down the rest of the company's operations for 20 hours. For example the company's office + factory burns down.

When that happens the email servers being "down" is not high on the Boss's list of concerns.

At other times, the IT dept should be able to get up and running in less than 8 hours.

If you're down when nearly everyone else is down who cares?
But if you're down when the other departments in the company are running fine (except for you not doing your part), guess who gets the stares?

If your company is mainly a bunch of people roaming the world/country, then going gmail might be OK, but you should have a backup email service provider (definitely not Hotmail though - but Yahoo Mail seems OK).

Re:The benefits of cloud computing

[pin0chet]

OK, let's say you can cut the rate of death by a factor of 10 by driving safely.

There are, on average, 40000 automobile deaths per year in the U.S., or roughly 1 per 7500 people. So a safe driver's risk of dying in a car accident would be 1 in 75000.

http://www.ntsb.gov/aviation/Paxfatal.htm

But there are only 16 deaths per year, on average for the past six years, due to airline crashes in the United States. That's 1 per 18,750,000 people.

So even if you could reduce your chances of dying in a car crash by a factor of 100, airline travel would still be significantly safer than driving.

Re:The benefits of cloud computing

[im_thatoneguy]

Also our exchange server is also our domain server (I know bad idea... blah blah blah...) but we have a backup machine which gets cloned every night.

If our exchange server went down. Everything would stop anyway. (Never have had this happen actually but if it did it wouldn't be a world of hurt.) If GMail goes down that's a second failure at a point in time when work *can* be done.

It's like a power outage. If Google had power outages at the same time our office did no problem. Since Google is in a different location the chances of our power outages and their poweroutages overlapping is exceedingly low. It effectively doubles our down time. Especially because an exchange failure on our part should be rectified in less than 20 minutes.

Re:The benefits of cloud computing

[invalid_user]

Thief! You stole my password!

Re:The benefits of cloud computing

[Anonymous Coward]

You named vital directories "core"??? Ha ha ha ha ha ha ha ha ha!!!

Re:The benefits of cloud computing

[Belial6]

Look who's talking. With two systems, which is very little money, you will get better uptime than with google. Actually, with just one, non-redundant system, you will generally get better uptime than with google. First you try to play the "Well, there is always the possibility of downtime" in the context of extremely small numbers. Then when you are called on it, you switch to the "If I compare ridiculous amounts of redundancy to google, google will be cheaper."

It's real simple. Hardware is not that fragile, and administering an email server is brain dead simple. The is no excuse for internal email system to be down, and Googles uptime is apparently not very good.

For personal email, that is fine. For companies that rely on email, it is a poor choice.

Re:The benefits of cloud computing

[reboot246]

I thought your password was "12345".

Re:The benefits of cloud computing

[syousef]

Take a step back for a moment... Your email is down for a day. So what? For most people, this is really no big deal. Especially if it only happens once in a blue moon.

Personally I'd just be annoyed. For the CEO of a company on the other hand this could be the difference between sealing an important deal in time, getting data that's vital to his or her decision making process, perhaps even the difference between saving the company and not. Yes, for anything that urgent there's still fax and other forms of communication, but people tend to assume email is always up and may not notify someone of vital information in a second way just to be sure.

Re:The benefits of cloud computing

[Niten]

Something tells me that an email system that only depends on your hard drive with no net connection wouldn't be very useful.

What? You've got to be kidding, right? It is incredibly useful.

Except for a mail server and spam filtering software running on a VPS, my entire mail system resides on my laptop. I download new messages with fetchmail, sort them into mailboxes with procmail, read them with mutt, write them with vim, send them with msmtp. The upshot of this is that when my home Internet connection goes down, or when I'm out and about but don't have an access point, or should my server somehow catch fire -- I can still access all of my mailboxes locally. I can still edit and save drafts. I'm not stuck wondering, "Crap, when did she say that meeting will be?"; it's all right there.

This GMail outage exemplifies Reason #1 for using POP3 for email access.

Re:The benefits of cloud computing

[definate]

The paid for version of Google Apps has the ability to do dual delivery. This means you can setup extra redundancy if you want. I believe you can do this 2 ways, through the Google Apps and, through Postini.

If you have such a critical email system, you should use this feature, and perhaps sign up for another email interface in the cloud (Maybe Microsoft's? So you're certain they aren't using the same infrastructure).

Then you can do dual delivery to that server, and you can also add in extra mail delivery servers in the case that Postini completely goes down.

That would be as secure and redundant as anything I have seen at my work place.

Re:The benefits of cloud computing

[Cramer]

I didn't. The linux kernel tree does. And I said it was a bad idea at the time :-)

Re:The benefits of cloud computing

[jt2377]

If your CEOs are smart, they should fire you and just outsource everything to third party or Google.

Re:The benefits of cloud computing

[Anonymous Coward]

Not the same thing at all. Telephone calls are synchronous, so you need the whole chain online for it to be useful. Email is inherently asynchronous.

So while I won't be able to carry on an email *conversation* (or a phone conversation) unless the whole chain is up, I can see the previous state of my inbox as long as I have (a) the bits that make up my inbox, and (b) a powered computer in front of me. For example, a laptop with a non-dead battery, or my desktop (even during a power outage, if there's time left on the UPS).

Webmail like Gmail takes the biggest benefit of email (its asynchronous nature) and turns it back into the telephone system.

(And don't get me started on them removing useful features without notice, and then trying to resort to hacks like "Older version".)

Re:The benefits of cloud computing

[Anonymous Coward]

I don't want a hard drive they crash, get infected with malware, don't get backed up and generally accumulate crud and slow down.

Just give me a net connection with an online service that goes down less then your typically enterprise PC.

I will be happier and I can layoff half my IT department and save a boat load.

Re:The benefits of cloud computing

[Anonymous Coward]

Sure it's and excuse - and a valid one...

Everyone knows gmail is in beta - that leaves a huge loophole for issues like this in Google's favor.. If you're in a car running on beta tires, and you have some understanding of what that means (as such, the admins that have outsourced their email do by nature of their position) - if your car gets a flat due to one of those beta tires going boom, are you going to whine to the tire company that you missed a meeting because of it? Only if you want to look like more of an ass than you already do for using a beta product in a production environment.. People who know what all things "beta" entails use gmail, but they also have backups just in case..

Re:The benefits of cloud computing

[TimSSG]

You are statically safer flying an airplane then driving

I am not sure about the static part, but I consider my self a safe car driver. At least, I am safer driving a car than flying a plane. Now, if you are a pilot the answer might be different. Tim S

Re:The benefits of cloud computing

[citizenr]

The worst thing is the update to iGoogle

amen to that!, just look at this
http://www.google.com/ig/directory?hl=en&url=www.google.com/ig/modules/builtin_gmail.xml

I commented how much it sucks an hour ago, and already 30 more people commented how much it sucks and my comment is gone (it displays only last 30 comments). People HATE IT with passion.

also this page
http://www.google.com/support/bin/answer.py?answer=20324&topic=13918
is missing pictures of the "great" new Canvas crap that everyone uniformly hates.
We got TABS in our browsers, we dont need no canvas

Re:The benefits of cloud computing

[TimSSG]

There is an SLA for paying customers, but with Google's track record they really need to up the penalties for non-compliance.

For some reason, I think Google are more likely to lower the penalties because of this than raise them. Tim S

Re:The benefits of cloud computing

[SmallFurryCreature]

Take a step back for a moment... Against this wall? No thanks, I don't smoke, why do you want me to wear a blindfold?Your email is down for a day. Eek! So what? So what? For most people, this is really no big deal.What people, cavemen? Especially if it only happens once in a blue moon.Everytime the moon is blue I loose email, you are scaring ME!

Have you never experienced a local failure of any kind?Well, once when I spend an 12 hour session checking my mail I sorta had a leak... You never had a day without a computer before?Yikes, stop being so nasty, even in a post apolyptic world there must be computers, how else am I going to run my pip boy. Your local net connection never went down?Nothing ever went down on me. I got quadrupple redundancy and keep a stack of modems in air thight containers just in case. Never had a power failure?Please, my coffee machine got UPS

I'm sure some guy on here has some worst-case, niche scenario where losing email for a day means the end of the world - but for most of us, it just means people who REALLY need us use a phone. If your email is so important, you need some kind of redundancy that a single provider is just not going to ever give you. You'd better have a hell of a data center with redundant connections through multiple providers.You are SPYING ON ME? Which of my dozen webcams have you hacked into?

Re:The benefits of cloud computing

[citizenr]

btw try javascript:_dlsetp('v2=0') on the igoogle homepage, it worked for me

Re:The benefits of cloud computing

[isorox]

Homepage: about:blank
Google has a search thing in the top right of the browser

Re:The benefits of cloud computing

[Sobrique]

User expectation has also changed. We've got service level agreements that specify delivery times on emails being ... erm. Something low, like under 5 minutes.

However much I try and explain what the point of an asynchronous protocol is, it's hard to get that through to a salesman who expects his emails to arrive instantly.

Re:The benefits of cloud computing

[jalefkowit]

Something tells me that an email system that only depends on your hard drive with no net connection wouldn't be very useful.

Not necessarily true -- in that scenario you can at least still access your old messages (assuming your client has cached them locally), and write new ones & queue them up for sending when your net connection comes back.

Re:The benefits of cloud computing

[MadMidnightBomber]

Admins watching helpless with no way of fixing the problem? Sounds a lot like sendmail 8.11.3.

Re:The benefits of cloud computing

[CXI]

Something tells me you don't work in a position that cares much about history. In our office, individuals access their old email (and files) constantly as part of contract negotiations. They could continue to function without a network connection, which isn't the case if Google owned all their data and Google was completely missing for a day+.

Re:The benefits of cloud computing

[omkhar]

If you were smart, you wouldn't route corporate email through a public service. Especially internal possibly confidential info. That's security first principals.

Re:The benefits of cloud computing

[mlwmohawk]

So, what do you do for electric power?

Almost every mission critical business has a back-up generator.

Re:The benefits of cloud computing

[mlwmohawk]

I understand all the concerns, but its worth noting that you are not committing to a single vendor lock-in. A Google Apps customer can choose to switch vendors or run their own mail server, as long as they control the domain name.

Not in a way that is easy to do during an outage. It takes planning to move emails, passwords, etc.

Re:The benefits of cloud computing

[NothingMore]

"If your dirt cheap or free and completely awesome email being unavailable for two hours a year is causing you to lose business, then you seriously need to rethink your operations" Just because its cheap/free doesnt mean that google suddenly dosent have to honor the SLA that people paid for. Im assuming they will in the end give everyone the free 3/7 days they are entitled to in the SLA but you cant simply say because a company offers something "cheap" that they dont have the live up to the obligations and expectations of the product.

Re:The benefits of cloud computing

[geoffspear]

Well even if you are a good driver, chances are not everyone around you is

Duh... that's why you should always drive over 85 miles per hour so you can pass all of the unsafe drivers.

Re:The benefits of cloud computing

[MightyYar]

Yes, for anything that urgent there's still fax and other forms of communication, but people tend to assume email is always up and may not notify someone of vital information in a second way just to be sure.

You are right, of course, that people will make this assumption... however if I were a big-shot with an important news source, I'd have my secretary call all of the relevant people on my contacts list to let them know that the email was down. Or text message them. Or email them from a home account.

In general, I know that the phone has made a big comeback with the big-wigs, at least in the US, because email leaves a paper trail and you have these newish SarbOx rules that force you to retain data.

Re:The benefits of cloud computing

[SenseiLeNoir]

but then along came the spanmmers, who took advantages of the open relays, and requiring the need for more hoops than you can shake at. Now email is point to point

you send to your SMTP server on your ISP/work, and that gets delivered direct to the SMTP server at the destination. in LARGE companies it may go through 2 or 3 SMTP servers within the same company, but in general point to point these days.

still the protocol does allow some failover, so you are still right in essence

Re:The benefits of cloud computing

[myz24]

Wow, really? Really?

IMAP in Thunderbird lets me download messages and read them while being offline. I can even write a message and save it as a draft.

Re:The benefits of cloud computing

[Insanity Defense]

Yeah. If there is one thing businesses hate, it is CUSTOMERS.

Company to Customer: You can have any colour you want, so long as its black.

Company I used to work for (Head office not my branch): Thats the way we make/package it take it or leave it. No matter how important the customer or how much they were willing to pay for customized service.

DRM.

Sometimes it seems that way.

Re:The benefits of cloud computing

[initdeep]

except that in most cases, the "admin" who changes to Google Apps does so because they can tell the "PHB" that's it's "FREE" and it's Google, so they never go down.

Thus they use the free version of Google Apps and "look how much money we saved" get's plastered all over the department memo's.

Then something like this happens, and the PHB's upstairs suddenly start thinking "didn't this happen BEFORE the switch"?

That thinking then leads to the thinking of "well if it happened before the switch, and it happened after the switch, what's the common denominator"?

which of course leads to the conclusion that the Cd is in fact, the idiot RUNNING the system, who must then be fired and replaced.

Way to start yourself down the road to unemployment.

Re:The benefits of cloud computing

[definate]

Well if the Admin said "it's free" (when he could have paid for a slightly more prioritized service) and then said "they never go down", then he wasn't doing his job. Management doesn't want yes men, they want the right decision. (However, right is very subjective)

Additionally, companies make decisions based on all sorts of reasons, it's too simplistic to reduce it to such a linear stream of logic. Especially since, if it took you 5 seconds to walk through that, it also takes them 5 seconds, so why didn't they see this coming?

If you're talking about manging the short run with the long run (or should I say vs.), then that's a normative statement and it's going to be different for each organization and it's position, and so on.

Business isn't as simple and straight forward as people like to think it is.

Re:The benefits of cloud computing

[Anonymous Coward]

I think the outages will reduce and the uptime will increase. Picture it this way. Look at the amount of load Gmail is getting daily. Then picture the load which an average Exchange server is getting. I bet you any money in the short space of time Gmail has been in existance it has far surpassed the uptime to data transfered then you could ever hope to pass through an exchange server, which in my experience had to be reset every weekend. IT Guys don't admit these things because they begged to get rid of Lotus Notes. I suspect if every Company published Exchange downs on the International News. I doubt there was be any question of reliability.

Re:The benefits of cloud computing

[hmar]

So I'm in the unique position of having argued to management that we shouldn't risk anything on Gmail, and us doing it anyway because management wanted it.

What's unique here again?

And you know what? I was wrong.

Oh, I see, management was right!

Re:The benefits of cloud computing

[Bryansix]

No no no. We are talking about two completely different things here. My point was that with Google EVEN WITH downtime there is no loss of data. I then pointed out that RAID controllers and the hard drives attached to them sometimes fail in such a way that nothing is recoverable. Then I pointed out that you don't have real-time backups. Yes, the email gets delivered but nobody can get to their old mail until you do a restore in which case they probably lost some email between when the backup was done and when the server failed. Then you mentioned that you can throw extra servers at it at which point I mentioned that this really is cost prohibitive when you compare it to Google's solution.

If you want to argue features or speed or something like that then I will listen, but arguing against Google Apps because of reliability is lame and you don't have much going for that argument.

Re:The benefits of cloud computing

[Anonymous Coward]

Do you really think there are the same number of people driving as there are flying? You need to calculate deaths per number of participants. Or are you intentionally skewing things to try to support your views? You must support Obama.

Re:The benefits of cloud computing

[Glendale2x]

With google, there is no assurance that *my* problem is being worked on.

While that is true, it is also true of your electricity and net connection. And any other utilities feeding your building that are critical to your business.

Depends where you live. The last time I reported a power outage to my local power company they sent two linemen to look at it. They told me nobody else had called it in, but since I did, they showed up. I suppose most people just sit around in the dark waiting for someone else to notice.

Re:The benefits of cloud computing

[Anonymous Coward]

Pedestrian vs. train is never an accident - the train is huge and has a very limited area of motion. People getting killed crossing train tracks are too retarded to bother to look and/or pay attention to crossing signals.

Re:The benefits of cloud computing

[Bandman]

Except that we receive client data through email. So if we don't get client data for 2 days, we're going to get fired, and that means a heck of a loss of revenue.

I work in a 24x7 computing environment for a financial services company. I literally don't have a service window for disabling client access. This means I've got to have backup systems that can take over at a moment's notice.

Just because there are 20 people in my company doesn't mean that there isn't a large, large amount of money at stake to our clients. Individual machine uptime isn't as big of a deal as service availability being as close to 100% as possible.

Re:The benefits of cloud computing

[MBGMorden]

:) I love all the posters here who believe that they are always the good driver. The truth is that a) EVERYONE believes that they're a good driver, b) even good drivers can make mistakes, and c) good drivers are at the mercy of bad drivers.

It's kinda similar to when I used to hear parents talking about their problem children. Their kid always "started hanging out with the wrong crowd". Each individual participant in this "wrong crowd" was always the victim being brought in by the others, but it never crossed any parents minds that their kid might actually BE the wrong crowd.

Re:The benefits of cloud computing

[Anonymous Coward]

My utility company doesn't have a draconian EULA that says they effectively own my business data. Yeah the power goes out but I don't have to worry about the electric company stealing a cool idea my company spent millions innovating and working hard on.

Re:The benefits of cloud computing

[MightyYar]

You would fit the description from my original post:

I'm sure some guy on here has some worst-case, niche scenario where losing email for a day means the end of the world - but for most of us, it just means people who REALLY need us use a phone. If your email is so important, you need some kind of redundancy that a single provider is just not going to ever give you. You'd better have a hell of a data center with redundant connections through multiple providers.

That's you :)

Re:The benefits of cloud computing

[MightyYar]

You don't know what you are talking about:

Google and its licensors shall own all right, title and interest, including without limitation all Intellectual Property Rights (as defined below) relating to the Service (and any derivative works or enhancements thereof), including but not limited to, all software, technology, information, content, materials, guidelines, and documentation, except that Google does not own Customer Content, or any End User or third-party content and/or information used as a part of the Service, including the content of communications appearing as part of the Service.

Re:The benefits of cloud computing

[syousef]

.. however if I were a big-shot with an important news source, I'd have my secretary call all of the relevant people on my contacts list to let them know that the email was down.

You're clearly a sensible person. Not everyone the CEO must deal with can be trusted to be sensible.

Re:The benefits of cloud computing

[MightyYar]

You're clearly a sensible person.

No, no, no... I'm a foe, remember :)

Re:The benefits of cloud computing

[Bandman]

*sigh* I know ;-)

Re:The benefits of cloud computing

[Belial6]

No, YOU are misunderstanding. Adding one extra server is not "throwing money at the problem". It is trivial to add a second server that is set as a cluster so that any email delivered to either server is instantly replicated the other server. This means no data loss and dramatically higher up time than Google. Understand. All data delivered is stored on two separate machines in real time so that even if you have a hardware failure, there is no data loss. This is trivial to install, and maintain. It has been trivial to install an maintain for over a decade.

Re:The benefits of cloud computing

[Bryansix]

What you describe is not trivial to install or maintain. It is possible to do but MOST small to medium businesses DON'T do it.

Re:The benefits of cloud computing

[Belial6]

I call clicking through a windows install wizard, selecting the "cluster box" and typing in the name of the other server trivial to install. I call adding users by clicking on the "Tools->People->Register" then filling out 3 or 4 fields with things link "Name", "Email Address" and "Password" to be trivial to maintain. Of course, if you hate Windows, you can do the same thing on Linux, or your big iron.

This is a problem that was solved a decade ago. I guess this is part of the bad side of a MS world. Trivial problems that were solved a decade ago seem like magic to people until someone as big as Google decides to hock their wares as if it is something that needs a giant data center to accomplish something so trivial.

Re:The benefits of cloud computing

[Bryansix]

http://technet.microsoft.com/en-us/library/aa997253(EXCHG.65).aspx Everything I'm reading says there must be some sort of shared storage. That makes it a little more then checking a box. It also makes software update take one more step. Ya, it is easy but it's not trivial.

Re:The benefits of cloud computing

[Belial6]

Well, your first problem is using MS Exchange. Switch to Lotus Domino and it is just a check box. It has been since at least Version 4 which was released in '95 or '96. The same is true of things like off-line web apps. Lotus Domino has had it since at least 2001. It is secure and has good replication by default. Take a look outside of MS once in a while and you will find that there is a great big world of software waiting for you.

I still remember going to MS seminars back in '97 when MS was telling us that they were going to compete head to head with Notes/Domino in groupware. They were claiming that they were going to make Outlook and Exchange more feature rich. A year later, it all just got swept under the rug, and they turned their attention to changing the definition of 'groupware'.

Offline mail access?

[Anonymous Coward]

Yes, Gears is the answer. That way I can get my mail messages without any connectivity. Now THAT is innovation.

Re:Offline mail access?

[larry bagina]

Does it include a reverse bayesian filter to generate stock market and h3rb@l \/i4gr@ spam?

Outage Outrage

[Pantero Blanco]

It's a risk you take any time you let someone else handle something for you.

Re:Outage Outrage

[Xandar01]

This is what you get when you pick the cheaper option eh?

Re:Outage Outrage

[mshannon78660]

It's a risk you take any time you let someone else handle something for you.

Specifically, it's a risk you take anytime you use a free service for something critical. You can't have an enforceable service level agreement for a free service - in order to be binding, a contract has to involve consideration from both sides.

Re:Outage Outrage

[jeffmeden]

That's just it, google DOES charge for the hosted apps version of Gmail. See http://www.google.com/apps/intl/en/business/index.html for more info.

Re:Outage Outrage

[AvitarX]

Have you not read what they are able to do with your private communications?

I would call that consideration.

More to the point though, I would say that if you can't afford downtime, pay for the SLA ($50/user/month).

Of course there credits are hardly an incentive to keep things working.

Re:Outage Outrage

[mshannon78660]

If that's what they're using, then they do have recourse:

Google Apps for Business

Claims right on that page that it provides 99.9% uptime.

Re:Outage Outrage

[Vellmont]

It's a risk you take any time you let someone else handle something for you.

It's a risk you take, period. You're trying to tell me that you can guarantee no unplanned downtime if you were to handle it yourself?

Re:Outage Outrage

[jellomizer]

It's a risk you take if you do it yourself. However the Outrage will be dished to you...

Re:Outage Outrage

[Goodgerster]

They aren't charging my company for the hosted apps version of it. Furthermore, it's working perfectly.

Re:Outage Outrage

[Medievalist]

It's a risk you take any time you let someone else handle something for you.

Specifically, it's a risk you take anytime you use a free service for something critical. You can't have an enforceable service level agreement for a free service - in order to be binding, a contract has to involve consideration from both sides.

Having an "enforceable service level agreement" does not make things magically unbreakable.

One of the great business fallacies of our time...

A really smart provider will not sign a contract promising 100% uptime, but a stupid one will. Which one will deliver the better service? In practice, your real guarantee of reliability is quality work, and the best way to get quality work is hire the best and treat them well. Making them sign pieces of paper promising doom for failure does little or nothing useful.

I've been using Gmail all day, incidentally. Works fine for me - I've been corresponding with other gmail users no problem.

Re:Outage Outrage

[Anonymous Coward]

It's a risk you take any time you let someone else handle something for you.

It's a risk you take, period. You're trying to tell me that you can guarantee no unplanned downtime if you were to handle it yourself?

No, but you get more control over the process if you take care of it yourself.

For some things, that's not practical, and you haveto take the risk (for example, power and utilities), but any mid-sized company should be able to handle email on their own.

Re:Outage Outrage

[Bryansix]

You only pay when you have over a certain number of users.

Re:Outage Outrage

[Anonymous Coward]

This is not completely true. I use google for domains and I get FREE email for up to 300 accounts. Again, for free. I lose some of the higher end admin features, but basic gmail for my domain is fa fa free. Sweeeet.

Re:Outage Outrage

[petermgreen]

And then it says for more details I should look at the terms of service. When I go there and search for uptime my browser finds no results.

For a SLA to have teeth it needs to have significant penalties for failing to provide the agreed level of service. Of course you are very unlikely get that kind of SLA at the price google are charging.

Re:Outage Outrage

[Bandman]

Since I've been shopping for email hosting, I've encountered a few misguided sales people who try to convince me that they actually provide 100% uptime.

The most amusing case was the company who offered two hosting levels. One, which guaranteed 99.9% uptime for their shared hosting, kept their shared customers' accounts on a clustered set of Exchange hosts, with data stored on an enterprise SAN. The 100% uptime was reserved for those people who paid an extra $500 a month. Those lucky customers had an exchange server running on an off-the-shelf Dell box that wasn't connected to the SAN for backup at all.

I really did laugh at the engineer and salesweenie as I thanked them for their time and hung up.

Re:Outage Outrage

[omkhar]

With proper SLAs you out source responsibility and liability. Those who haven't adequately protected themselves with SLAs and the relevant UCs are playing with the guards off.

By your statement we should all generate our own power too.

Re:Outage Outrage

[Anonymous Coward]

By your statement we should all generate our own power too.

Saying there is risk involved in something and saying that the risk is never worth it are two different things.

"including high-ranking executives"

[Anonymous Coward]

Uh oh. Beware the wrath of a herd of angry PHBs.

Stallman is laughing

[Rinisari]

Is that the sound of cloud computing advocates crying, or the sound of Richard Stallman laughing?

Re:Stallman is laughing

[runlevelfour]

Not to mention it is a "free" service, no one has a right to demand it to be up anyway. You are storing your e-mail on google's servers and you pay nothing for it. If they need access to their crucial time-sensitive data then they should keep it themselves or pay for a service which will guarantee availability.

Re:Stallman is laughing

[petermgreen]

According to TFA this is also affecting paying customers.

Re:Stallman is laughing

[Etrias]

Keep in mind what they are probably talking about is the paid service you can get through Gmail Apps. For most people, yes, gmail is free with a personal account, but there are whole organizations who have their email hosted with Gmail. Think of it as hosted Exchange without Exchange.

If it's a free service, then yes, caveat emptor. But if someone is paying for this, then there should be a higher level of service or at least explanations of what happened or why it went wrong.

Re:Stallman is laughing

[Pantero Blanco]

Not to mention it is a "free" service, no one has a right to demand it to be up anyway. You are storing your e-mail on google's servers and you pay nothing for it. If they need access to their crucial time-sensitive data then they should keep it themselves or pay for a service which will guarantee availability.

Some of these companies are paying customers. The headline only mentions Gmail, but this is also about Google Apps as a whole.

Re:Stallman is laughing

[itp]

That's not exactly true. The articles in question make it sound like these are users of Google Apps, a paid service that provides standard Google Apps (like GMail, Docs, etc.) customized for your organization/domain, at a cost per user.

Re:Stallman is laughing

[Anonymous Coward]

I think you're mistaken there. The idea that anyone would pay for gmail is laughable. It's free dude! check it out.

Re:Stallman is laughing

[Etrias]

Have you really looked at it? I mean actually go to the Google Apps page at looked at it, or did you just hit reply without knowing. Oh, who am I kidding, this is /. and I'm lucky you read any of my message.

Seriously, here's a link to the Google Apps business page. Look around. This isn't free stuff. I'm not sure why you scoff at this and not other business webmail applications. There...do you see that...do you see how Gmail isn't always free?

My point remains...if they paid for it, Google owes them an explanation.

Re:Stallman is laughing

[That's Unpossible!]

No, it's the sound of one hand clapping.

What I mean to say is, what admin HASN'T had an outage like this?

Shit happens. I'd rather get email that works 99% of the time, and when there's a problem, google engineers are dealing with it, leaving me time to work on more important things.

Re:Stallman is laughing

[Tony Hoyle]

No. Google Apps is free if you just want the standard stuff like mail. I have several domains in it and have never paid a penny.

You can pay for it to get extra services and a faster response time, but for the most part it's free.

Re:Stallman is laughing

[Anonymous Coward]

They didn't pay for it though.

Re:Stallman is laughing

[syousef]

I'm not sure why you scoff at this and not other business webmail applications.

I'm not the GP but Lack of a SLA perhaps? In any case I scoff at the idea of trusting a third party with any important data. If you can't afford to lose it, and if you can't afford the service outage, you shouldn't be using a 3rd party if at all possible.

My point remains...if they paid for it, Google owes them an explanation.

They agreed to pay Google for a service that explicitly does not include support. Google owes them nothing.

Re:Stallman is laughing

[omkhar]

28 hrs of downtime != 99% uptime

Better check your backups...

[rarel]

After all, it's "only a beta", eh?

Re:Better check your backups...

[EncryptedSoldier]

lmao yeah really...i know google gives all the excuses in the world as to why they still consider it a beta, but c'mon. It's just an excuse so when shit like this happens they can point their fingers at the subscript and say "well it's beta"!

Re:Better check your backups...

[moosesocks]

Um. I can't find where they made this excuse...

Re:Better check your backups...

[Anonymous Coward]

http://www.networkworld.com/community/node/33131

Re:Better check your backups...

[EncryptedSoldier]

you got it! i read that article too!

If you need something done right do it yourself!

[EncryptedSoldier]

You can't count on Google to run your IT...sorry buddy. Using Google may be cost effective, but the obvious trade off is that someone else is really doing your job, and if that person drops the ball, then you really screwed the pooch, at least that's what your boss will think.

Give it to someone else and lose control

[1s44c]

If you give something to someone else you are no longer in control of it. Email is a critical business tool, I'll not be giving mine away any time soon.

Re:Give it to someone else and lose control

[Anonymous Coward]

Curious if they are "improving" their email "reader".

Re:Give it to someone else and lose control

[pfarber]

I concur. At least you will be able to work around the issue if its on site and within your control. Posting your cries to a support forum is not my idea of administering a system.

Now would be a great time to convince the CEO to bring it in house... and show how valuable your skills are next round of raises.

If its just a font end issue can't you access mail via POP3 of other way? The mail isn't bouncing is it?

These guys need a brain transplant...

[Khyber]

"Admin Bill W. posted a desperate message on the forum Thursday morning, saying his company's CEO is steaming about being locked out of his e-mail account"

Run your own damned mail server if it's THAT IMPORTANT. Seriously, it's not hard to set one up, and you've obviously got the money to do it.

Once again, it's a case of rich people with more money than brains having the problems. Nothing important here, nothing of value lost.

Re:These guys need a brain transplant...

[Anonymous Coward]

Run your own damned mail server if it's THAT IMPORTANT. Seriously, it's not hard to set one up, and you've obviously got the money to do it.

Yeah, that's the first thing I thought of.

The second thing was, what if my server goes down (and it will). Who has more and better trained support staff, me or Google?

Re:These guys need a brain transplant...

[Anonymous Coward]

Can't agree more. You can even do it for almost nothing, except the cost of the hardware and the internet connection. I've put friend's businesses on a small Linux box with Postfix and shared all my configurations with them, and it's been bulletproof ever since.

I put together a small box for my 40 person company for around $800, and the only time we have downtime is either when we have a power outage that lasts longer then our UPS's, hardware failures (bad stick of ram), and kernel updates.

In other words, my uptime so far, has been longer then Google's. I point that out during my annual reviews. :)

Re:These guys need a brain transplant...

[Rary]

Run your own damned mail server if it's THAT IMPORTANT. Seriously, it's not hard to set one up, and you've obviously got the money to do it.

Right. Because some nerdy 20-something admin with a copy of "Sendmail for Dummies" can do a much better job than all the engineers at Google.

This is a paid service offered by one of the largest and most knowledgeable technology companies around. They should be able to do a much better job than any internal IT department. There are arguments in favour of doing it yourself, but there are definitely arguments in favour of outsourcing to a competent provider, which Google should be.

This is a PR disaster for Google.

Re:These guys need a brain transplant...

[iamhigh]

Who is running a single mail server that should be easily recoverable in a few hours? Who is running a service used my millions with complications I probably can't even fathom?

They might be "better" at running world class stuff than you, but you can be better at running simple stuff. Veritas, Acronis, etc... take your pick and you are back up and running. That probably isn't true for Google.

Re:These guys need a brain transplant...

[TooMuchToDo]

doing a much better job than any internal IT dept. != never fails

Re:These guys need a brain transplant...

[Builder]

How do you figure it's not that hard to run your own mailserver? Please tell me how to achieve the following, simply.

1. POP and IMAP access for all users
2. Simple ability to delegate administrative rights for the domain including creation of new user rights
3. Ability for users to manage aliases
4. Ability for users to manage mail lists
5. Spam prevention with similar success rates (including very low false positives) to Gmail or any other mass mail site
6. Webmail
7. Simple password changing for users
8. Authenticated SMTP
9. Simple SPF

I've spent a lot of time and effort, and have a good smtp / imap / pop / webmail solution, but it's still cobbled together from loads of stuff and it's still missing solutions to allow users to change passwords and delegate administration control of a domain.

Re:These guys need a brain transplant...

[Anonymous Coward]

Right. Because some nerdy 20-something admin with a copy of "Sendmail for Dummies" can do a much better job than all the engineers at Google.

I'm basically a one man show and my qmail server has been in service for longer than gmail has been around. It has also had less downtime. Granted I don't have near the volume gmail does, but I'd rather rely on my own service than outsource it so I can have more control over it when things go wrong. There are less moving parts with my setup and therefore its easier to fix...

Re:These guys need a brain transplant...

[slimjim8094]

Right. Because some nerdy 20-something admin with a copy of "Sendmail for Dummies" can do a much better job than all the engineers at Google.

Well, actually, there's a decent chance. These technologies are extremely mature, and pretty simple. For example, I ran a 25-user email server for well over two years with a 99.8% uptime... between age 15 and 17, and it handled at least 500 emails a day.

The difference is, I was dealing with tens of gigabytes of data and backing that up (external HD + gpg + rsync + my house's firesafe). Google is dealing with thousands of terabytes (including, frankly, a couple of mine - I use their free domain service)

I'm sure as hell not as smart, and I'm NOT criticizing Google here. But when you're dealing with MUCH less data and don't need any load-balancing or fancy DNS tricks, it's pretty easy to get right.

But they are doing load balancing, clustering, immediate failover with synchronization, and all those other fancy tricks, because they need more than one server, have a lot of data to back up, and need to keep it all in sync. That is Fucking Hard to get right. A server in my client's basement on a T1 realistically only needs to worry about fire and trees.

Re:These guys need a brain transplant...

[SaDan]

The second thing was, what if my server goes down (and it will). Who has more and better trained support staff, me or Google?

In my case, I do.

But that's just it... If you don't have a competent staff, you are hosed either way. Run stuff in-house and depend on a consultant and his hours/workload, or farm it out to Google.

People who can't maintain something in-house are screwed either way. Companies that have IT staff in-house need to make use of their resources better, and/or get rid of the dead weight in IT so you can maintain your critical services in-house.

Re:These guys need a brain transplant...

[SaDan]

http://www.zimbra.com/

Look at the Open Source Edition to get started for $0.00 for unlimited users (pending hardware capabilities).

Enjoy.

Re:These guys need a brain transplant...

[magamiako1]

I have to agree with you here. I've been trying to get a mailserver running, and part of the problem is everyone has their own "way" of doing it, and there's no concrete method other than figuring it out, which can be a pain in the behind.

The sendmail/postfix guys have their "one way" of doing things. that is, the way it was always setup was send mail to users on the system and to other users on other systems. Pretty simple, but in the newer environments of hosted email (rather than users on the system) there are problems.

The postfix guys aren't going to help you with courier or dovecot. The Ubuntu guys can't help the Redhat users. And even inbetween distributions, different people have entirely different (and often times incompatible) methods.

What you stated above is extremely difficult to do elegantly.

Re:These guys need a brain transplant...

[Rary]

doing a much better job than any internal IT dept. != never fails

True. However, they guarantee 99.9% uptime. Anything more than about 8.75 hours per year of downtime would be breaking that guarantee. So far they've got customers who have been out for more than 24 hours, and that's not even counting the 3 significant outages they had in August.

Re:These guys need a brain transplant...

[Rary]

I'm not saying it can't be done. As I said in my previous post, there are arguments in favour of keeping that kind of work in-house.

However, as much as we all love to blame clueless PHBs for every decision that appears "bad" with the benefit of hindsight, can you really fault someone for buying services from a company as knowledgeable as Google, with their 99.9% uptime guarantee (which means the ability to be compensated if that guarantee isn't met), instead of doing it internally, where the only recourse if something goes wrong is yell at someone and maybe fire them -- which doesn't actually solve the problem, or provide any kind of compensation?

Re:These guys need a brain transplant...

[Ant P.]

99.9% means on average you can get up to 1 hour of downtime a year. You can also drown in a stream with an average depth of 6 inches...

Re:These guys need a brain transplant...

[mortonda]

Run your own damned mail server if it's THAT IMPORTANT. Seriously, it's not hard to set one up,

Yes and no, unfortunately. Sure it seems easy to set up a mail server, but there are a lot of misconfigured mail servers out there that are open relays, or spew backscatter spam, or simply get hacked and turned into zombies for spammers.

That doesn't begin to touch the issues of administration of users, anti spam software, uptime and redundancy planning, etc.

I'd say, it's not that hard *for a competent system admin* to set up a mail server.

Re:These guys need a brain transplant...

[slimjim8094]

No, I think I agree with you. It's great to be able to shift the blame, but I was talking about the tone of your post.

Your tone (to me) seemed like 'don't criticize google because you probably couldn't do as good a job'. All I meant was 'well, actually you probably can, but because you're not as big an organization and don't have to deal with as much'

It's quadratic. As a business increases in size, it starts to have too much data to manage with just one server, and backing up becomes a pain. As it grows larger, you need multiple physical connections, clusters, and the synchronizing things that keep it all working.

Google has all that to worry about, while most 2000 person businesses really only need one server. Which keeps the whole thing pretty simple.

Re:These guys need a brain transplant...

[Sentry21]

Sorry - you're saying that if we pay Google for a service, we shouldn't be complaining when that service completely fails to work for days on end, or works intermittently at best? If this were Microsoft everyone would be screaming to the rafters about how pathetic they are, about how their software is junk and they can't even keep an e-mail service running.

Oh, but it's Google! So because it's Google, they're obviously in the right, and if you have problems with them it's your fault.

Well if Google can't run a service reliably, they shouldn't be selling it. Our company signing up with them was a mistake, and it's one we won't repeat.

Re:These guys need a brain transplant...

[Builder]

Have a look at my doc at http://www.penguinpowered.org/documentation/email_virtualhosting.html

It covers most stuff, but there is still quite a bit missing. The main thing is the benefit of being on google mail for spam because they have so much more traffic to work with.

The above solution keeps mail accounts separate to system accounts and gets most things right.

Re:These guys need a brain transplant...

[NeoSkandranon]

If it's "that important" you aren't hiring nerdy 20-something admins.

If you are, then you have (much as when using Google's services) zero room to complain.

Re:These guys need a brain transplant...

[Walter Carver]

Right. Because some nerdy 20-something admin with a copy of "Sendmail for Dummies" can do a much better job than all the engineers at Google.

I don't know about Bill W., but is that your idea of an administrator? It differs a lot with mine.

This is a paid service offered by one of the largest and most knowledgeable technology companies around.

Don't you think you are putting a lot of trust on them?

Re:These guys need a brain transplant...

[Anonymous Coward]

Right. Because some nerdy 20-something admin with a copy of "Sendmail for Dummies" can do a much better job than all the engineers at Google.

How do you know that the engineers at Google aren't 20-something admins with a copy of "Sendmail for Dummies"? I doubt that the guy manning the switch in the middle of the night is the uber genius that everyone seems to project as the typical Google employee. Google has f*ckups too.

SaaS

[Anonymous Coward]

This is just a test of the Software-as-a-Service model.

Google provides the service , free of charge, and lets see what kind of uptime users expect.

Want 99.99 uptime? pay 9.99 a month say.

Want 99.999 uptime, pay 99.99 a month say.

This will be the future of computing.

Let someone else host your data, for a small monthly fee.

Let someone else host your email, for a small monthly fee.

"Lease" a computer, for a "small monthly fee".

use an operating system for a small monthly fee.

It will work just like your cellphone. You will be allowed to "bundle" services saving you say 15% on your 'small monthly fees'.

Say goodbye to the personal computer, say hello to your 'network terminal'.

Re:SaaS

[jellomizer]

It sounds good to me...
$99.99 a month for 99.999% uptime is actually a good deal. I can't beat that, with running my own server at work for that cost I only have enough money to charge a guy to check out the server for 15 minutes a day. I might if I am lucky get 99.9% uptime from that.

This doesn't prevent you from getting your own computer and doing whatever you want however for most people who are willing and able to play and pay by the rules they are probably paying less, even in the long run.

If you are unwilling or unable to follow by the rules for this model, they will still sell computers, you can still get your own OS, if not Windows and OS X Linux will still be around.

Saas is actually a good model. It lets you focus on your business not the IT.

WTF? I see no outage.

[anomalous cohort]

Huh? I was logged in to gmail all day yesterday and am logged in today. I am receiving and sending plenty of email from that account. What's the problem?

Re:WTF? I see no outage.

[squisher]

Huh? I was logged in to gmail all day yesterday and am logged in today. I am receiving and sending plenty of email from that account. What's the problem?

Uhm, please read at least the freakin' summary! It says right there that not all users are affected...

Re:WTF? I see no outage.

[mapsjanhere]

Is that you, brenda@viagra.com?

Re:WTF? I see no outage.

[An ominous Cow art]

I use (free) Gmail through Thunderbird and IMAP, and have been getting timeouts and spurious authentication failures for months.

Sometimes you do get what you paid for, but it was pretty smooth for a while.

Re:WTF? I see no outage.

[slimjim8094]

Same here; I was predisposed to assume that it was Thunderbird...

Re:WTF? I see no outage.

[ceoyoyo]

I've never had Google's IMAP work reliably.

I have four IMAP accounts that are checked by Mail on my Mac. It helpfully puts warning icons next to the ones that are unreachable. The two hosted at a local server? Very rarely a problem. Mobile Me? Since they actually got it working, always there. GMail IMAP? Seems to go out at least a couple times a week.

Re:WTF? I see no outage.

[Anonymous Coward]

So if 0.1% of the users are affected 1% of the time. How does it then compare to running your own servers? I don't think gmail ever had an outage that affected the majority of the users at the same time. If gmail have a problem meaning a thousand users cannot access their email for a few hours, it makes it to the headlines on slashdot. If a company with a thousand employees had their mailservers down for a few hours, it would not get nearly as much coverage.

Betcha

[iminplaya]

Nobody saw this coming, huh? He he he...sorry man... ha ha ha... really I am... HAHAHA... No, I'm serious... BWAHAHAHA!!!

Re:Betcha

[iminplaya]

Another troll...*sigh*

The thing is, you're built too low. The fast ones go over your head.

Remember, gmail is still beta!

[Anonymous Coward]

I bet you that "Gmail is still in beta" will be their excuse, after all is said and done.

POP/IMAP

[superphreak]

Why no Gears for offline Gmail access at very least, Google?
I believe it's called POP/IMAP access, and it's been around a long time. Oh, downside - you might need a program called Outlook/Express or Thunderbird. Free download available.

Beta means never having to say you're sorry.

[Orgasmatron]

Who puts important mailboxes on a beta service? Sheesh.

Re:Beta means never having to say you're sorry.

[I'm not really here]

The same idiots that pay to use a BETA product.

Re:Beta means never having to say you're sorry.

[Zarel]

The same idiots that pay to use a BETA product.

Too many people make this mistake, calling Google Apps a beta product.

First off: Gmail is only called "beta" because it is constantly being improved. If you use the version of Gmail within Google Apps, which is stable and not beta, you will find that its features are usually a few months behind Gmail. The same goes for all of Google Apps - it is not a beta product (although it's possible to opt-in to beta features).

You get what you pay for!

[itsybitsy]

Complaining about a FREE service makes no sense (assuming the CEO isn't paying Google that is). It's says a lot more about the steaming CEO than it says about Google reporting and working on the problem.

It seems that CEO's really are spoiled brats, at least in this case. Maybe he'll take his golden parachute while he's steaming and leave the minions in his company to get to the real work of the company.

Depending on outside companies for your email is like depending on string while mountain climbing. It's bound to break. In fact I'm surprised that there are not more visible problems with Google's services. They've seemed to be able to hide their inevitable glitches quite well.

Every system you use and depend on can fail. Prepare for it. Choose which systems you support yourself and which you rely on outsiders to deliver. While no man is an island it doesn't make any sense to have too many critical path dependencies on others either. Balance.

Re:You get what you pay for!

[AKAImBatman]

Doesn't anyone RTFA?

Google Apps is a suite of hosted collaboration and communication software and services designed for workplace use. Its Premier edition costs US$50 per user per year and includes a 99.9% uptime guarantee for the Gmail service.

In August, Gmail had three significant outages that affected not only individual consumers of the free Webmail service but also paying Google Apps Premier customers. As a result, Google decided to extend a credit to all Apps Premier customers and vowed to improve its problem-notification methods.

$50/yr for each user is not "free". Nor is it in the domain of "you get what you pay for". $50 per user is actually a rather significant sum when we're talking about 100+ user companies.

Re:You get what you pay for!

[mapsjanhere]

I see you skipped on RTFA in best /. tradition; the whole problem is not with free users being locked out but PAYING premium google app customers being left in the dark.

Re:You get what you pay for!

[Maxo-Texas]

However,

The maximum hit is refunding you $50 per user.

And the likely maximum hit is refunding you only for the downtime.

We are sorry your email was down... here is your ($50/365 ) * 2 = 27 cent refund.

Re:You get what you pay for!

[AKAImBatman]

That's better than when your small business email server blows a RAID controller* and it takes all day to build a new one from backups. In that case you're looking at additional costs as opposed to a refund.

* The whole point of RAID is to make machines more reliable, right? So how come RAID controllers have become such unreliable parts? Yay for Dell-quality equipment, I guess.

Re:You get what you pay for!

[Chyeld]

$50/yr for each user is not "free". Nor is it in the domain of "you get what you pay for". $50 per user is actually a rather significant sum when we're talking about 100+ user companies.

I'm not defending OP's lack of RTFA skills. But really, $50 per user per year is a drop in a bucket for any company actually making money. Consider most companies that employ blackberries are paying ~$50 a user a month. And most companies that provide company paid cell phones are paying at least ~$20 a phone a month.

Nor is this a huge disaster for Google.

RIM has had some very spectacular outages in the past, yet people still want their 'Crackberry'.

The company I work for just got hit in the past two weeks with a huge issue, our data center is located off site and we appearently lost the link to it mid-day Friday and it was still out mid-morning on Monday. After that, AT&T went belly up and no one in the corporate office could dial out. This is a company with over a thousand locations nation-wide.

And yet, no one is screaming for us to switch to a new data center, or to drop AT&T. Because these things only happen once in a blue moon. Just as Google outages do. Yes, while things are down, you get some very nasty feedback, and you learn the difference between type a and type b personalities. But once everything is back up, unless someone went out of their way to ensure that the decision makers are too pissed to forget about it, they do. Until the next time.

The question isn't "How long will Google be down" but "How often will Google be down". So far the answer is "Seldom". And that'll work for 90% of the folk out there using them.

Re:You get what you pay for!

[Anonymous Coward]

$50/year/user for 100 users is considerably cheaper than a server (amortised over 4 years) and an admin's wages...

Re:You get what you pay for!

[itsybitsy]

As I said in the parent post...

Re:You get what you pay for!

[Anonymous Coward]

At $50 per person for a 100 person company you're only paying $5,000 to handle your email for a year. That is extraordinarily cheap compared to a home grown solution, where $5k gets you next to nothing in hardware and backup ability, or about 2 weeks of admin support tops.

Re:You get what you pay for!

[Bryansix]

More like $27 for a 100 user company. But ya, still a small condolence.

Re:You get what you pay for!

[Mista2]

What you need to do it yourself:
2 SMTP mail gateways, with SPAM and security filtering built in. 2 Internet connections, 2 routers set in a failover condition.
Atleast 2 backend mailbox servers, preferably in a cluster or other high availability arrangement. This will require expensive shared storage. These maybe over two sites for physical seperation, requiring data synchronisation between your mailbox stores.
You will also need a high availability authentication system, multiple copies of your user database (active Directory, eDirectory etc) and your own public facing DNS servers x2 for redundancy.
Don't forget backup and archiving!
All interconnecting switches and servers will require dual paths, dual power supplies etc to avoid comms failures. This is to allow planned downtime on all equipment for patching and maintenance.
Even then the unexpected can happen - a software fault in our SAN required a complete server room shutdown for us to reboot the SAN. Later in January we have to move the server room. Our only sitewide outage this year 8)

To get nine-nines is really expensive. Most medium sized organisations will settle for Business hours uptime for a fraction of the cost. Blackberries and remote access from home is driving up the availability requirement though. We are finding it increasingly difficult to get outage windows for maintenance.

Re:You get what you pay for!

[foobarbazquux]

$50/yr for each user is not "free". Nor is it in the domain of "you get what you pay for". $50 per user is actually a rather significant sum when we're talking about 100+ user companies.

Is it? Is it *really*? What decent office software can your corporate (read, Windows) users buy for $50 a year? Compared with a world-class mail, calendar and collaboration suite?

Surely, $50 a head to save your IT department from having to provide:
  * Mail servers (training, architecture, backing store, backups)
  * Calendar server (as above)
  * Shared document access
is cost-effective.

As an example, I worked out how much it cost my university of 20,000 users to migrate to and use MS Exchange over five years. I might add, the worst of it was that they went for Exchange and now students are mandated to use Outlook Web Access for their email. It came to about $2M in staff costs alone. Factor in hardware costs (they were talking 25 high-end servers and a pair of beefy NetApps storage devices), power, space and aircon -- suddenly $50 p.a. starts to look really well worth it.

Re:You get what you pay for!

[definate]

I deal with this sort of infrastructure for my company. $50 per user is cheap.

Think about Internet costs, Power costs, Server costs, Cooling costs, Licenses, Backup costs, Redundancy costs, Rent costs, and Administration costs (Salary(-ies)).

The salary an "ok" sysadmin commands in Australia is around $60,000 a year. Which means based on that cost alone using Google Service I could support 1,200 users. This isn't even including the other costs.

For a business $50 per user is REALLY cheap.

Re:You get what you pay for!

[Salgat]

5 grand a year for +100 employees for all the things Google Apps provies? Sounds cheap to me, especially when you're paying at least $3,000,000 for those employees to work for you.

That's SaaS for you...

[scsirob]

This is the main reason not to turn to Software as a Service. Sure, it's nice to just rent some functionality, but you are not in control of your own destiny. What if Google decides that GMail no longer fits their business model? Poof...

Re:That's SaaS for you...

[tshak]

If you ran your mail server off a single $499 dell workstation you'd probably have availability problems as well. This is not a problem inherent with SaaS, it's a problem with using a consumer grade mail service for a corporate mail.

Re:That's SaaS for you...

[truesaer]

The problem here is not that they've outsourced an IT function, but that they outsourced it without a service level agreement.

It might be worth it for some companies to accept a bit of downtime in order to have Google handle this for me. If companies really need to guarantee a certain level of uptime then they really ought to be using a contractor or service that contractually provides it. If it is damaging their business rather than just being a really annoying inconvenience then I put at least half the blame on bad management for not having an appropriate contract in place.

Re:That's SaaS for you...

[jellomizer]

So if you were in control of your destiny then your email servers may still be down however you are the one the needs to fix the problem...

Also there is a case of being smart with you SaaS models, You can often get it so you can get your own data back when you request it. If GMail no longer fits their business model. Well move the data to an other SaaS. Email is kinda easy to move around.

You sound like a guy who is afraid to Fly. Even though you may be statically better off flying then driving however if you don't have the control in your hands you are more nervous.

Re:That's SaaS for you...

[Todd Knarr]

Actually they did have an SLA. The users complaining were using Google Apps, for which they pay and which includes a Service Level Agreement. However, the users are learning 2 important life lessons:

1. Down is down, SLA nonwithstanding. All the SLA means is that you may get limited compensation when the service is down. It doesn't get your service up and running one second sooner.
2. In this case, the compensation is pretty poor. Google won't compensate for damage to their businesses due to e-mail being down. All Google has to do is provide 15 more days service at the end of the contract period. But then, what did you expect for $50/year?

Re:That's SaaS for you...

[ceoyoyo]

Try again. This is their corporate grade apps package (not just e-mail) that's down, and this isn't the first time.

Re:That's SaaS for you...

[TooMuchToDo]

In this case, the compensation is pretty poor. Google won't compensate for damage to their businesses due to e-mail being down. All Google has to do is provide 15 more days service at the end of the contract period. But then, what did you expect for $50/year?

No one, I mean no one will compensate you for damages to your business if the service they're providing fails (unless you're paying millions of dollars a month, and it's written into your contract). Read your agreements with any service provider. So why would Google be any different?

Re:That's SaaS for you...

[Anonymous Coward]

Google can call it "corporate grade" all they want. It's still consumer grade simply based on its availability. Google is giving robust SaaS a bad name. Until they demonstrate 5 9's it's not even close to being reliable enough to call "corporate grade".

Re:That's SaaS for you...

[truesaer]

Down is down, SLA nonwithstanding. All the SLA means is that you may get limited compensation when the service is down. It doesn't get your service up and running one second sooner.

This is true, but an SLA also implies that the provider is taking steps necessary to attempt to comply with the SLA. For example, lets say I run a business running mail servers. I could just take an old desktop, install Linux on it, get an internet connection and start selling accounts. But if I start signing SLAs I'm going to need to take steps to make sure I can meet them. I may need to implement a backup system, ensure I keep a full slate of spare parts on hand for any occurrences of hardware failure, perhaps put together some kind of hot swappable RAID to handle a single failed disk, perhaps invest in multiple internet connections from multiple backbone providers, perhaps figure out a system that is redundant enough that an entire data center could be destroyed without interrupting service. I might hire 24 hour staff to monitor the systems.

It all depends what what level of service I've agreed to provide. I don't think Google really guarantees a particularly high level of service, although one would expect that a system as heavily used as GMail/Google Apps would have pretty effective staff and infrastructure in place anyway.

Re:If you need something done right do it yourself

[whisper_jeff]

The thing is someone will always drop the ball. In this case, the CEO can't chew out the guy in IT who pooched the email server and is working frantically trying to get it back up and running because that guy works for a different company. Or do people honestly think that an internally-run email server never has problems?... Just because it's Google does not mean it's infallible.

at its very worst

[Anonymous Coward]

Google is still about 10x better than most servers run by the U. S. Gov't. Which is why so many USG people use it :-)

including this anon coward.

it can go both ways

[Trepidity]

I've been at plenty of places that run their own mailservers where uptime is considerably worse than Gmail's, so it'd be an improvement to offload it. The biggest problem seems to be at medium-sized shops: big enough for there to be problems, but not so big that you have some sort of massively redundant setup with transparent failover and 24/7 staffing. The ideal of the cloud-computing style of outsourcing is that you'd outsource to someone who was big enough to have a massively redundant setup with transparent failover and 24/7 staffing. However Google seems not to have delivered on that ideal.

Re:it can go both ways

[guruevi]

There must be some bad mail (server) admin then or a bad stick of RAM in the server. I maintain a very high uptime throughout a typical business day and I only have a single server. The most the server has been down during business is 2 hours and in that time I had to rebuild half the machine. Even Exchange which I utterly hate doesn't have that much downtime.

In my philosophy, if you can't recover your e-mail system from a disaster within 4 business hours you're doing something wrong (in my opinion) and you either have to fire your admin, spend some money on a backup system, split it up in some fashion (per department, per site etc.) or you have to replace your e-mail system. The problem with giving your e-mail to a hosting provider without any control is that you can't guarantee that to your customers.

These days setting up Postfix with a synchronized MySQL database (or other database) and a parallel storage backend is not a big deal anymore and those type of systems are so flexible there is really no excuse to have a large amount of downtime if you can afford a backup system. I will do it for you under $20,000 including hardware and you should be able to support up to 2000 users with it.

Re:it can go both ways

[Anonymous Coward]

We are a small business (4 servers, 50 employees.) and all our "downtime" is less than 1 hour per year, all scheduled and after 12 a.m. on a weekend, and has been that was since 2000 when I started.

Re:it can go both ways

[orangenerd]

There must be some bad mail (server) admin then or a bad stick of RAM in the server.

(...)

Usually, first choice.

Re:it can go both ways

[ToadMan8]

We have a fairly massive organization with a large Exchange cluster and 24/7 staffing, etc. Our issue is that, if we host our e-mail solution, our users believe we can (and ask us to) do anything and everything. If we had a hosted solution, it would be much easier for the support group to say "nah, our e-mail provider can't do that; sorry!".

The things users ask for are usually not critical to the business, but put the solution in jeopardy from the necessary futzing to accomplish it.

We are a medium-sized University, btw, and the users asking for things are Faculty, thus why the top doesn't just say "hush; get back to work".

It's beta for a reason.

[Anonymous Coward]

In fact isn't nearly every google app beta.

Re:If you need something done right do it yourself

[samkass]

So you think most companies have better IT departments than Google? I agree that using a free beta software to run mission-critical software is probably unwise, but there are other providers that offer way more uptime than probably most internal IT departments could manage. Pair Networks, etc. It will cost money, though.

why no offline?

[nimbius]

because offline access

means you aren't using google. and we dont want that now do we?

tell the CEO to stuff it. it was probably one of his mid level jerkoffs who decided outsourcing a critical business application to a 3rd party vendor with little accountability and no SLA whatsoever would be 'good for initech'

I for one have no bad feelings about the outage.

There is an SLA for paying customers

[Que_Ball]

Quote from article: So, will this one prompt calls for a service-level agreement for paying customers?

Paying customers of the apps Premium account level DO have a service level agreement.

Free customers do not however which is probably what they were trying to say.

Revised quote: So, will this one prompt calls for a service-level agreement for free customers in addition to paying customers?

From the terms of service for Premier account edition:
http://www.google.com/apps/intl/en/terms/premier_terms.html

1.9. *Service Level Agreement*, or *SLA* means the Service Level Agreement located at the following URL: http://www.google.com/a/help/intl/en/admins/sla.html

Downtime period is a period of ten consecutive minutes of Downtime

Service Credit is
three days of service added to the end of your term at no charge for monthly uptime percentage between 99.0 and 99.9
seven days for between 99.0 and 95.0
fifteen days for worse than 95.0 uptime percentage.

You must request your service credit. It is not automatic.

Re:There is an SLA for paying customers

[DragonWriter]

Free customers do not however which is probably what they were trying to say.

Uh, an SLA for free customers? So, the people who aren't paying anything could get a refund if Google's uptime in a month is below some floor.

What good would that do?

Re:There is an SLA for paying customers

[jimicus]

three days of service added to the end of your term at no charge for monthly uptime percentage between 99.0 and 99.9
seven days for between 99.0 and 95.0
fifteen days for worse than 95.0 uptime percentage.

But with Google, you don't get to negotiate scheduled downtime (which wouldn't normally count towards the percentage downtime calculation). It happens when it happens and if that impacts you, tough.

And 7 days free service for a system that's only up 95% of the time? Big deal. In 30 days, that's a 36 hour downtime period. If you're lucky, the 36 hours is spread out across the month entirely out of your business hours and nobody really notices it. If not....

Re:There is an SLA for paying customers

[Anonymous Coward]

Service Credit is
three days of service added to the end of your term at no charge for monthly uptime percentage between 99.0 and 99.9
seven days for between 99.0 and 95.0
fifteen days for worse than 95.0 uptime percentage.

You must request your service credit. It is not automatic.

Lame. So, 1% down is 3.65 days... and they give back 3 days... 5% down is 18.25 days... and they give back 15... They don't even give back an equitable number of days! How does anyone sign into that snakeoil contract?

Go outside...

[Anonymous Coward]

Go outside, see more sunshine. 4'get'bout'IT email a day.

Bill W.

[Anonymous Coward]

I guess we know what drove him to drink, now.

Google SLA

[WPIDalamar]

http://www.google.com/apps/intl/en/terms/sla.html

There you go, the SLA for Google Apps. It's listed at 99.9%

But... the remedies for them failing that suck, only up to 15 days worth of service per month will be credited.

Also, it costs $50 per user per year

How long before we se "cloud appliances"?

[fuzzyfuzzyfungus]

If I were Microsoft, I'd look at a situation like this and see an opportunity to use my existing software products as an asset in pushing my own "cloud computing" or "software as a service" or whatever it is these days initiative.

The rough outline of the scheme would look something like this: put together a lightly modified variant of Exchange, call it "Cloudspare Server" and make it available in either binary or appliance form, at low cost, to enterprise customers who purchase outsourced email services from you. The "Cloudspare server" box would sit there, synchronizing with MS's email servers from time to time, and would be available as an emergency backup should the "cloud" services be unavailable.

Since the expected service of the local server would be only a few hours a year, at worst, it could be relatively conservatively specced, and thus cheap, while still offering something rather than nothing in the event of downtime.

Google, or anybody else, could obviously do something similar, as they also have experience with packaging certain of their services as appliances; but MS has the advantage of already having an accepted product that could be modified to suit.

Re:How long before we se "cloud appliances"?

[somethingwicked]

True, too bad they haven't thought of it yet

http://www.microsoft.com/online/exchange-hosted-services/learn.mspx

Okay, so they just need to promote it more.

Re:How long before we se "cloud appliances"?

[fuzzyfuzzyfungus]

Well, on the plus side, I guessed correctly. On the minus side, I ought to have googled harder first. ;)

Re:How long before we se "cloud appliances"?

[somethingwicked]

BTW, before I get bombed for this, I included the link for their hosted service, but the subset that is damn close to what you described is Microsoft Exchange Hosted Continuity

http://www.microsoft.com/online/exchange-hosted-services/continuity.mspx

And yes, I know there are likely tons of other non-evil alternatives to MS for this, just mentioning MS has this since the parent said they should...

No problem logging into Gmail here

[kindbud]

WTF. I logged in just now and read/replied to mails without a problem. Check the Windows Firewall on the CEO's computer, maybe?

Re:No problem logging into Gmail here

[pugugly]

I work nights - used it at work yesterday, checked it today when I got into work, haven't had an issue?

Pug

der

[Anonymous Coward]

That's some smart CEO, using an unsecured gmail account for his executive correspondence. I wish the article had mentioned the company so I could sell all of my shares immediately..

'a small number of users,'

[a2wflc]

Every gmail bug that affects 'a small number of users,' has affected me. Is my account that special or is their definition of 'small' different than mine? Maybe because 99% uses the number 99 which is small compared to 4294967296.

Re:If you need something done right do it yourself

[LWATCDR]

Because every company can afford redundant internet connections, back up generators, a fall over mail server, and a 24/7 IT staff and I don't mean some poor guy with a cell phone and no life.

Hey my account work...

[Dwarth]

Why should I care about a CEO account.... if it's not my company CEO (of course)

T-Mobile G1?

[Anonymous Coward]

Does anyone know how GMail outages affect the newly released G1 cell phone? I know you have to have an account, but what if you can't log in? Can you still make calls?

No google gears because...

[blahplusplus]

... in gmail is most likely because of the ads! I see ads in my newsfeedbar and on the side when I'm reading email. That is why they are not going to google gear gmail in my opinion, whether this will change or not I am uncertain but they certainly gain a hell of a lot of insight into what a user wants by being able to use gmail to comb the emails for context relative ads. Most people don't give a flying f-ck what comes in and goes out of their email, they use it like any other communication device. Many personal things are said by email partially for convenience and partially because of the stupidity of the majority of end users. As always ignorance and convenience trumps privacy when it comes to the internet.

Re:No google gears because...

[afidel]

Ads can (and SHOULD) be turned off for google apps customers, it's an admin preference. Therefore the lack of gears for paying customers is nonsense. Now, the real answer is just use IMAP/POP for offline backup. I prefer IMAP because it retains folders, but both work fine.

Re:If you need something done right do it yourself

[jellomizer]

Umm. No.
You blame Google your boss will blame google... Unless he just doesn't like you and he will yell at you for anything. I bet you will be in a Lot more trouble if your Email server went down because he can more directly blame you.

Outsourced Monoculture

[Doc Ruby]

Outsourcing links in your essential service chains is risky enough. Outsourcing them to a single point of failure is too risky. So many independent places all outsourcing something so central to so many service chains is unacceptably risky.

I would never rely on GMail without a local cache of all the content GMail holds, or without a truly alternate server to serve my messages when GMail goes down, as it clearly does some percentage of the time.

Re:Outsourced Monoculture

[Number6.2]

Why do you hate Corporate America. Look at all it's done for you! :D

Re:Outsourced Monoculture

[fortapocalypse]

GMail is not really a single point of failure, since they have redundant servers, routers, databases, etc. etc. Although this outage definitely exposed something that wasn't fault tolerant enough. Only a small percentage of companies use more than one mail provider. Most use their own internal one, and more often than not, it is not setup as well as GMail. On top of all of that my organization recently had a full day's email outage, and we'd be having issues for weeks because of a migration. I know of another company locally that lost a good bit of their email recently. Problems like that are normal, and you can't just say people shouldn't be using GMail.

Re:Outsourced Monoculture

[Doc Ruby]

No, GMail is clearly a single failurepoint. As is now clear all of GMail can indeed go down at once, or at least in a way that people cannot "route around" by using an alternate server during the outage. Realizing that one's network schema has such a single failurepoint takes an insightful network admin, for reasons just like this one. It might not be obvious that those different boxes in the schema can make an impassable barrier if they go down together, and that such an event is entirely possible, but that's what a good network admin gets paid the big bucks for.

If I thought that more than a few minutes more than a couple times a year were an acceptable downtime for, say, any of the investment banks I worked for, they'd have a hot spare replacement for me spun up and working overtime before I even knew it. Some orgs tolerate failure. The ones that count do not, and do not tolerate admins who do.

Re:Outsourced Monoculture

[fortapocalypse]

Good point, but anything can fail really. You can add layer upon layer of redundancy and fault tolerance, and then someone explode a nuclear bomb that takes out the right spots in the world's network that totally disables your services. Still, as you say, fault tolerance on a macroscale cannot be ignored. And I'm not saying that GMail would be a great way to outsource email service for a large critical enterprise, but it certainly is fine for the most part for some some large organizations, even if this particular failure is obviously not acceptable and something needs to be done to avoid it in the future. Critical services go down, and people get fired. Organizations learn and grow. Life goes on.

Re:Outsourced Monoculture

[Doc Ruby]

You have it backwards. Multiple nukes taking out multiple failurepoints is a very low probability. What's more, in that event, the overall context of worldwide war means that most apps won't matter at all if they go down (eg. your coupon service, your appliance sales customer contact database), so the benefit of mitigating that very low risk is very small at a huge cost. Which means you don't do it.

However, GMail going down is now proven to have a relatively high probability. The impact doesn't render any services irrelevant. The cost of mitigating it by using something else, or adding local caching and server redundancy is relatively low, though just depending on a more distributed set of alternatives is probably cheaper with the same mitigating bottom line. So the cost:benefit*risk evaluation is clearly a legitimate subject for small to large IT departments to address.

Critical services go down and people get fired when they rationalize inaction or expensive waste like that, and the inevitable crisis (like bankruptcy) shows the weak link was in the IT department, not in the IT systems per se. That's how orgs grow and learn, and how life goes on.

Re:Outsourced Monoculture

[fortapocalypse]

I'm definitely not trying to rationalize inaction. I've become quite tired of being accustomed lately to others' failure, disorganization, and lack of accountability, which might be summed up as "inaction".

And I agree with you. Backing up data, whether it be for email or any type of service/application, is critical for the organization. Caching can be very important too, but just so I understand...

Doesn't a cached mail solution also present its own fault-tolerance issues?

For example, lets assume that you always send users to the name of the mail service provider (myfinancialinstitution.com) backed by multiple names pointing to geographically distributed mail servers that cache and back up GMail. Now your macro point of failure is your own service. You could have DNS issues/network issues, server issues, DB issues, etc. that will still come back to bite you.

That is the problem in my mind. While trying to setup additional fault tolerance, you're conceivably setting up other lines of dominoes to the side of the other GMail lines of dominoes that could fall. So hopefully when your dominoes go down en masse (as GMail did, and as each service/network almost inevitably will at some point), you can easily repoint that name (hopefully still reachable) so that the 100,000 users can be rerouted to GMail (assuming you're using the same ports, settings, etc.). And hopefully the organization was smart enough to allow and suggest their users be able to work remotely, in case the network outage affects the workplace. Also we'd assume the organization hasn't tied their users so tightly to the internal network/services that they can work when almost completely isolated from it. :)

Anyway, you are right. And apologize, because this is all just in good fun for me.

Re:Outsourced Monoculture

[Doc Ruby]

But the local mail cache is itself just a failsafe. If it goes down but the main remote mail store is up, it doesn't matter. Only if it goes down while the remote is down does it matter. But it hasn't decreased your remote's uptime; its downtime is only a small fraction of the extra uptime it supplies. Even if the local/remote stores' roles are reversed in priority, this is still true.

Redundancy doesn't reduce uptime if the two redundant routes are truly redundant, and not only partly redundant (which would thereby create a new, nonredundant, failurepoint). That's why the simple formula of Main-Uptime-Percentage + (Alternate-Uptime-Percentage * Main-Downtime-Percentage) works: 99.9% uptime and 99.9% alternate uptime means 99.9999% uptime over the redundant system.

This is fun for me, too. Seeing how these conventional network wisdoms scale to any architecture, even one that counts GMail as a whole as a failurepoint, is not my current job. But I'm reassured that there's nothing new under the Sun, even when the datacenters use other CPUs in massive parallelism :).

Re:If you need something done right do it yourself

[Anonymous Coward]

Hey, I'm a poor guy with a cell phone and no life, you insensitive clod.

Re:If you need something done right do it yourself

[EncryptedSoldier]

Well that's the point. You can either do it yourself, outsource to another company, or use a beta product (i guess that's an option of its own). The unfortunate fact is: When the boss man gets mad, he's going to get mad at a tangible, human being, whom he holds somewhat responsible. He's not going to get mad at the idea of a hard working IT department in another state. In this world, most people like to point fingers, and especially if it is your idea to outsource your email and such, you need to expect fingers being pointed at you when there is a problem.

Re:If you need something done right do it yourself

[snspdaarf]

The thing is, if I am going to take a screamin' reamin' from the boss, I prefer it be for something that is either my fault or that I can do something about. While a normal human can grasp these issues, some admin-types seem to think that if they throw a big enough shit hemorrhage that it will force the IT people to fix the problem. Tough to do when it is outside of their control.

"Hi, Bill!"

[Anonymous Coward]

Admin Bill W. posted a desperate message on the forum Thursday morning, saying his company's CEO is steaming about being locked out of his e-mail account...

Bill W. then followed up with a simple method of fixing Gmail in just 12 steps.

Re:If you need something done right do it yourself

[sjwest]

Email can be a pain, but it is worthwhile have your servers. A 'Management' problem for moving it to google and not the admins problem i suspect.

Friends!

[pete-classic]

Bill W.? He has more friends than Tom!

-Peter

PS: As always, remember that there is no "I don't get it." moderation option.

Semi-Offtopic : Hey! Google Home just changed!

[JoshDM]

Totally different appearance all of a sudden. Tabs are on the left now. Did I miss something or did this just happen?

RTFBlurb

[aardwolf64]

Read the blurb for crying out loud. It says it only affects a small number of users. You aren't one of them, so relax...

Re:RTFBlurb

[kindbud]

Oh. I see. So this isn't really stuff that matters.

Next article: My Mom can't find her address book.

Hey, we'll outsouce it to the "cloud" and save!

[Animats]

Considering that Google charges $50/yr/user for business email, they have far too many long outages. At that price point, they should be fully redundant across multiple data centers.

Welcome to "cloud computing". It's not your cloud.

"Hey! You! Get off of my cloud Don't hang around 'cause two's a crowd On my cloud." - some '60s band.

You are an Idiot

[Anonymous Coward]

and need to be fired if you complemented Google apps as you companies primary way of working.

Hook it up to outlook

[Twillerror]

I have my Google apps hooked up to Outlook via IMAP. If their servers are down I can at least get to my older emails. Forward the emails to another box as well if you're really worried about it.

There are a lot of comments out there saying that Exchange has really bad uptime. I've used it a lot with pretty good results. Yah it might go down, but it might only effect a relatively small amount of people and maybe even smaller if you have your mailboxes spread out. Also, there are solutions out there that have real time failover and other options so having business hour uptime is probably pretty dang good. Sure you have to take it offline on a weekend or something for an upgrade, but a company I do work for (knock on wood) has had essentially 100% business hour Exchange uptime for years now. Plus if the internet is down you can still send interoffice emails, you own the data, you can archive it for legal reasons, you can snoop if you need to, and lots of other good things that having an in house solution is great for.

The point is that Exchange or other internal email solutions are not as evil as you'd like to believe. For a small startup company I think Google apps is great. My company has 3 employees right now...so it doesn't make sense to have Exchange. When we get to 20+ we will start to seriously consider going back to an internal system.

And I would also point out that GMail is a massive system. The more complicated it gets the more chances are it will fail. Having a variety of simpler setups does have some advantage.

Seriously?

[Anonymous Coward]

I'm constantly amazed by people who whine about some Absolutely Vital Mission-Critical widget well outside of their control that they haven't bothered to do even the most basic of "What if" redundancy planning for. If something like e-mail is *this* vital to your company then you should have plans in place for how to do e-mail if Google goes away.

Additionally, the submitter or sub's boss must have presented multiple options to the CEO for "How We Should Do E-mail". Hopefully one such option was "Running Our Own Mail Server". It's then up to the C*O to determine the direction, while weighing the costs, risks, and benefits of each. Someone as smart as a CEO would certainly know enough to understand the costs, benefits, and risks involved in these options. So it's obviously the CEO's fault that he/she chose Google for e-mail.

Of course, if you didn't present the CEO with multiple options and just said "do it through google cuz that's where I have my mail and teh googles never goes down" then I guess you probably shouldn't be working in IT.

And don't even begin to claim "not everyone knows enough to run their own mail server". If someone has a business model that depends on e-mail this much then they need to have or hire that expertise. Alternately, paying a company whose entire business model is e-mail hosting would be another option. Yes, these options are likely more expensive than Google. But remember, this is an Absolutely Vital Mission-Critical thing we're talking about.

Gmail's working fine for me, all day now.

[Medievalist]

It's a risk you take any time you let someone else handle something for you.
It's a risk you take, period. You're trying to tell me that you can guarantee no unplanned downtime if you were to handle it yourself?

I want 24 hours notice prior to any unscheduled downtime! And don't give me any of that technical mumbo jumbo, I have a SERVICE LEVEL AGREEMENT!

Not at all

[Anonymous Coward]

I run my own mail server off a 300 MHz ancient Pentium. I'm going to upgrade it to a 500 MHz Soekris board.

I get thousands (3-5K) email messages per day (I'm on a lot of mailling lists). Each one is spam filtered, and I might get 3 spam messages per week (which I could eliminate, but I'm too lazy).

The uptime is superb (I have it on a real UPS), and it draws little power. The only availability issues I have is when the power to the whole house goes off for a long time.

Sure, the CPU is pretty busy. But everything works superbly.

Upgrading to a modern $499 PC would be way overkill. It would suck down too much power. The Soekris upgrade will be quice nice, thank you very much.

The point is, if you know what you're doing, you can set things up reliably and cheaply.

But of course, these basic UNIX skills are way beyond most modern IT admins.

Still, my uptime seems better than Googles. And a heck of a lot cheaper as well.

Re:If you need something done right do it yourself

[garett_spencley]

Don't underestimate the appeal of being able to chew someone out for a problem. Even if it is not effective at solving the problem ... it sure feels a hell of a lot better.

Speaking of which ... my MySQL server is having problems and I'm the admin ... so I need someone to yell at. Got a minute ?

Re:If you need something done right do it yourself

[ajs]

You can't count on the USPS to deliver your mail...sorry buddy.

You can't count on Verizon to run your telecommunications...sorry buddy.

Every service you use was, at one point, decentralized and every large corporation ran it themselves. Then someone did a better job and companies slowly released the reins. Does Verizon's phone service go down? Yep. Does the USPS lose mail? Yep. Goes Google mail go down? Yep. But, in the end we've decided that we'd rather rely on these external services than continue to try to run increasingly large services with ever-diminishing returns for the individual business.

Is Gmail Still Free?

[jweller13]

Is gmail still free. If so, you get what you pay for. I don't understand how a business and make itself dependent upon free third-party email.

Re:Is Gmail Still Free?

[rob1980]

Not only did you not RTFA, you also didn't read the eleventy-seven comments on this thread all saying there is a SLA for paid Gmail hosting. Congrats.

Re:If you need something done right do it yourself

[ajs]

Interestingly, phone service and physical mail have both gone through several iterations of increasing and decreasing scope and centralization within organizations, so my above examples are a bit simplistic, but overall I think they hold up. We're at the start of what will be a century-plus period of understanding the role of computer-based communications in the business world, and as that grows and changes, Google will continue to grow and change and others will compete with them in interesting and perhaps successful ways.

I'm not waving a Google flag, here, just reacting to the idea that a single outage makes for a useless corporate service (which, if true, would have every company in the world running their own fleet of planes and drilling for their own oil).

Re:Semi-Offtopic : Hey! Google Home just changed!

[RayMarron]

I noticed that, too. Since I have only one tab, I now have a nice, big column of wasted space on the left. Way to Go, Google!

the "small IT shops are worse than SaaS" BS

[SuperBanana]

Yes. In your organization how many times have your servers went down or had a problem... Compare that to Google Mail... You will probably find that there is a lot less downtime.

Bullshit- this is an often-repeated myth that small or medium-sized IT shops can't offer competitive uptimes. It's simply not true- I'm a sole sysadmin, and my server (~200 users) has only had one time when we had an outage , and it took us all of about 15 minutes to fix. We have a number of people who choose to use GMail, and I'm constantly reminding them that they should not be relying on Gmail so much.

The problem is not downtime- it's lack of any way to mitigate the problems, and a complete and total lack of any customer service from Google. There is NOBODY you can call when there's a problem. PERIOD.

Compare and contrast. Google:

• If Google hoses someone's account, they're completely fucked. Google will shrug and say "meh, whaddya gonna do?", and point to their user agreement.
• If someone breaks into their account or changes the password, they're completely fucked. Google won't block access, can't prove who is who, getting logs will be a slow fight to the death, etc.
• If the user deletes a bunch of mail (or someone else does) or there's a bug with their email client (ie if they're using IMAP or POP access), they're completely fucked. Google won't do a restore. Their backups (if they even have any) are for "oh shit" system-wide fuckups (like, I'm guessing, the current one- I bet the accounts got deleted and they're restoring from backups.)

Me:

• If we hose someone's account, they need only wait about 15 minutes for the tapes in the jukebox to shuffle and we've got their entire account back from less than 24 hours ago. If I refuse or cannot, I'm pretty much out of a job.
• If someone breaks into their account or changes the password, I can lock the account in seconds, and I've got logs for forensics I can hand over immediately to the university police. Again, my paycheck is close to these people.
• If they delete a bunch of mail (or someone else does) or there's a bug with their email client (ie if they're using IMAP or POP access), again, they need only wait for the jukebox to shuffle tapes around. It's a few minutes of my time and perhaps a trip to the server room to feed the jukebox some tapes.
• If we have a crash, or a hacker breaks in, etc- we tell people what happened and we get the hairy eyeball from the administration. If Google hoses your account, you're told they had a 'service outage' or 'technical problem', and that's that.

The building I (and the server) are in in could burn to the ground, and I could have us back up in less time than this stupid outage at Google (I'm factoring the time to find/buy two commodity PCs, find/buy compatible tape drive/SCSI card, do an OS install, install the backup server, and fetch the off-site backups from across campus.)

If Google's datacenter burns to the ground, how long do you think you'll be without your GMail account?

Re:the "small IT shops are worse than SaaS" BS

[Anonymous Coward]

It should take google nill time to get back up after a data centre burns down because they have more than one. The failure point of your system is the fact that it is centralised - but being on a university you have little choice over that - and little reason to distribute your datacenters geographically - so you don't have failover.

Re:the "small IT shops are worse than SaaS" BS

[Terrasque]

Yeah, got a rude awakening there myself. My gmail account was closed a month ago, no prior warning. No way to get to support. No way to actually contact a human.

I still have absolutely no clue why it was closed. I also lost blog, gallery, docs, calendar, site stats, rss reader, notebook... You get the idea. Luckily I didn't have any serious data there, but still. It's a lot of things I used daily just suddenly gone.

And google's ToS says "We can close an account for any reason. Absolutely any reason. Like, we didn't like the color of the sky today is a good reason." Which isn't helping the issue much either.

Re:the "small IT shops are worse than SaaS" BS

[Anonymous Coward]

The building I (and the server) are in in could burn to the ground, and I could have us back up in less time than this stupid outage at Google

How? You'd be dead, or probably in a burn unit, or something...

Re:the "small IT shops are worse than SaaS" BS

[Anonymous Coward]

The failure point of your system is the fact that it is centralised - but being on a university you have little choice over that - and little reason to distribute your datacenters geographically - so you don't have failover.

And there's why you should STFU.

We have three large state universities who have off-site agreements with one another for all mission critical systems (and some not so mission critical systems).

Re:the "small IT shops are worse than SaaS" BS

[Bandman]

If Google's datacenter burns to the ground, how long do you think you'll be without your GMail account?

But that's one of the strengths of using gmail.

The loss of a complete datacenter for them would be a minor issue which would cause slow access. Accidental removal of one part of their infrastructure doesn't cause much of a problem. It's the administrative issues that get replicated across the network that cause major issues.

Re:the "small IT shops are worse than SaaS" BS

[jellomizer]

You work as the administrator. Your job is probably administering the system... Take You salary and add say 30% for your benefits. 35k - 90k depnding on your location vs. Say you use a professional SaaS shop and pay say $2000 (not just google free service) a month for the same services, that is 24k a year. With the $2000 a year service you are probably a good customer thus they make sure they are up. If not they will work just as hard as you to keep it running. This shop may have 10 - 20 people working on those servers if there is a problem they will go and fix it.

Dont confuse the Free Gmail as a true SaaS.

Re:the "small IT shops are worse than SaaS" BS

[Zebra_X]

The reality is that e-mail is a fraction of the time that this individual spends his/her time working. There are location dependent tasks such as File Servers Maint., Router Config, LAN Confg., End User Calls, etc. that this person also services. So there is really no "savings" because you still need that person around. Chances are that person is going to provide a level of support above and beyond what any SaaS could offer, especially if the organization hires the right person. So keep the e-mail server and the quality uptime.

Re:the "small IT shops are worse than SaaS" BS

[Anonymous Coward]

Now substitute "you got hit by a car" for "the building burns to the ground". Everything you say still true? No? Then STFU.

Re:the "small IT shops are worse than SaaS" BS

[mollymoo]

Some people pay $50 per user, per month for Google Apps Premier and this outage affected their GMail too. Does the paid-for GMail count as a true SaaS?

Re:the "small IT shops are worse than SaaS" BS

[Anonymous Coward]

All for ~200 accounts?

Re:the "small IT shops are worse than SaaS" BS

[joshuaobrien]

Your strategy is to build the disaster recovery system after the disaster? What if the disaster is that when the building burns down with you in it, your systems lose their sole admin? Who do you call then?

Re:the "small IT shops are worse than SaaS" BS

[TheLink]

In some places that person also gives advice on tech stuff to the boss. Like "Hey Boss, you shouldn't bet the entire company's email on gmail". :)

Re:the "small IT shops are worse than SaaS" BS

[Anonymous Coward]

Why did you put something in quotes when it's not actually a quote. That's horribly misleading.

Re:the "small IT shops are worse than SaaS" BS

[hawks5999]

No, that's $50/user/year not $50/user/month. That's $4.17/user/month. For 200 people paying $833/mo for mail and up to 5 TB of offsite storage is pretty good. The SLA on Premiere allows for about 8.75 hours of downtime per year. All in all, that's hard to beat.

Re:the "small IT shops are worse than SaaS" BS

[Askmum]

Why can't we mod up to 6? Parent deserves it.

Face it: Google's useragreement basically says "do as you're told and don't complain" to the user. Whereas a company's useragreemant says "do as you're told and don't complain" to the system administrator.
What a difference that makes. Sure, Google is free, so the price-performance ratio can not be beaten by any corporate solution, but if that is your only concern than you have no business complaining to Google. You use their product on their terms. Live with it.

Re:the "small IT shops are worse than SaaS" BS

[Anonymous Coward]

If Google's datacenter burns to the ground, how long do you think you'll be without your GMail account?

Oh, crap... how will I search for the GMail website?!?!

Re:the "small IT shops are worse than SaaS" BS

[mollymoo]

Gah, I knew it was per year, a simple brain fart. The SLA is worked out monthly though, so it's ~40 minutes per month rather than 8.75 hours per year. 99.9% uptime for $50 per year would be good, but Gmail doesn't have 99.9% uptime, they have a 99.9% SLA. My domain host has no SLA, but have given me less downtime than GMail.

Re:Semi-Offtopic : Hey! Google Home just changed!

[dwarg]

I don't know if they are unrelated. Gmail has now been integrated with Google Homepage (which is pretty cool) so the two may very well be connected.

It looks like not all the style information is correct yet but it looks like they expanded the widget architecture to embed more complex apps. My /. RSS feed is now expanded as well so I can rank articles and preview the summaries better from within the homepage.

Google's stuff is breaking all over

[RayMarron]

Google Groups search - Usenet & private newsgroup archives (that was formly done so well for so long by deja.com) has been utterly broken for months.

I think it's a staffing problem. They have all these big-brain, creative types that are super handy when you want to bring something new, exciting and difficult online. But when it comes time to maintain stuff and slog through the bug database, those people suddenly are working on their "personal projects" or going on sabattical or something. To coin a bad analogy, they're staffed with artists, not tradesmen.

Re:Google's stuff is breaking all over

[mollymoo]

I also get the impression nobody at Google can be bothered with the boring stuff. It's not a staffing problem, it's a management problem. Google is famous for its lack of management and it's starting to show in the quality of their products.

One Answer

[Intrinsic]

Dont rely on external web mail for critical operations, it must stay in house so you have control over the problem. Hope this makes people aware of the dangers now, I have been warning people for years.

Gears?

[cmacb]

I know people with multiple gigs worth of e-mail already. I think Gears would be a mismatch for this level of storage.

If it were set up so that only the last (for example) hundred meg or so were being retained locally, that might be OK.

But in any event, simply running ANY POP reader every few days would make an accumulating local backup of your mail, if that is what you want to do.

Last time Google was said to be down (right here as a matter of fact) I found that my POP client worked fine and I could still both send and receive with it. What was down in that case was simply the web interface.

Sorry, I don't think I've ever used an e-mail system of any kind that NEVER became unavailable at on time or another. My entire Internet connection goes away about once a year and there is absolutely no way to send or recieve e-mail until Verizon gets its act together. Should I expect the Gmail team to come slap Verizon around for me? I wouldn't mind that.

Can anyone tell me the difference...

[Spy der Mann]

between the GMail outage and *ANY* IMAP server's outage?

For practical purposes, it's just the same. Regardless of whether this is a "software as a service" curse, what can be done about having your e-mails stored in a remote server? Having my emails on a server was the very reason for me to choose web-based e-mail, so I can check and answer my mails while at home or at work. Also, it's not ISP-dependant, so I can keep the address and mails even if I have to switch ISPs.

Re:Can anyone tell me the difference...

[Todd Knarr]

The big difference is the affected company can do something about the problem. The CEO can come down and tell the admin he's not going home until this is fixed. They can call in any extra people they need. They can, if it's really that critical, have someone physically go and buy a new server and get enough software installed on it to get mail back up and running until the main system can be fixed. Expensive proposition there, but the company gets the option of deciding whether it's worth it or not.

Compare this to the situation Bill W.'s company is in. Their e-mail is down. All they can do is wait until it comes back up. No matter how crucial service is, no matter how much money they're losing because it's down, they've got absolutely no control over how fast the problem gets fixed. That'll be determined by how important to Google restoring service is. And the cost equation to Google is the cost of having staff working overtime all night to fix the problem vs. the cost of giving Bill W.'s company 15 days more service (about $2.06 at the $50/year rate for Google Apps).

Re:Can anyone tell me the difference...

[Mista2]

Also remember eMail is not a high availability or ensured delivery service. If it is important enough that is HAS to get there, use UPS.

Lol Mailtrust Ad

[genner]

I wonder how much rackspace is racking in having their mailtrust ad slapped on top of this story.

Gmail Haiku

[Anonymous Coward]

Spanish for "Deep Shit"
Caca Profundamente
For choosing GMAIL

Banner ads to the contrary

[HalInc]

In poignant irony, the banner ad I see above the story is a google ad that reads:

"So why not switch to Google Apps?
We maintain our hosted software 24/7 so you can sleep at night."

Strange Help Desk Phone Calls

[The Moof]

Employee: Hi, Help desk?
Help Desk: Yea, how can I help you?
Employee: I can't get my e-mail.
Help Desk: Hmm... I see. Yea, there's an issue. Hold on while I call the help desk.
Employee: Sure, no pro...wait, what?

Opportunity for software developers?

[RomulusNR]

It's too bad no one has created email servers that companies could install and run themselves.

Re: Sig

[JesseMcDonald]

Evolution is a state-sponsored, state-protected religion.

Evolution is a scientific model which explains objective observations about the physical world. It is no more a "religion" than calculus, the periodic table, the laws of thermodynamics, or the Theory of Relativity.

perspective

[desierto]

I was just watching that airline crash show, and one of the crash investigators was quoted as saying something like: until something goes wrong, it's hard to fix it so it doesn't happen again. Having some gerkwad CEO not be able to get his email for a few hours, seems like a small price to pay for google to learn their lesson.

Re:If you need something done right do it yourself

[SaDan]

I don't have redundant anything and I have better uptime at home than Google has had this year.

That being said, if my house burns to the ground, it would take me about 12 hours to get a new server with restored data running in a new location. Wait, that's still faster than Google...

Re:If you need something done right do it yourself

[SaDan]

Those who can do. Those who can't outsource.

SLAs aren't a cure-all

[Knara]

Yeah, okay, you get an SLA by paying for the business version, but anyone doing their homework and a few thought experiments will realize that SLAs are only potentially helpful after the fact.

$50/yr/user isn't going to get you 100% uptime, I don't care who is running it.

This strikes me kinda similar to the folks who try to run their businesses off Dreamhost shared web hosting servers (which don't even HAVE SLAs) and go ballistic when something breaks.

So...

[symbolset]

Over a day and a half of downtime in a given month makes less than 95% uptime so you get the compensation of two free weeks of service - if you ask for it. At a rate of $50/yr that's like, uh, $2 worth.

Sounds fair. Will they turn it off on purpose for a fee too? Some of us could use a break from email.

Its called IMAP support

[gravis777]

And a more immediate question: Why no Gears for offline Gmail access at very least, Google?

I enabled imap support in my gmail, and added the account to my Outlook, and enabled Cached mode. Bingo, problem solved!

http://mail.google.com/support/bin/answer.py?answer=77689&topic=12814

Re:If you need something done right do it yourself

[evilviper]

Because every company can afford redundant internet connections,

Dial-up is, what, $10/mo? Plenty of (Business) DSL providers offer ~20 hours of free dial-up every month. 56kpbs is more than enough to handle e-mails (provided you don't constantly send huge attachments like legal documents).

back up generators,

If your entire building doesn't have power, you're probably not going to care that your employees, sitting in the dark, can't send and receive e-mail. It's not like a web server... Any e-mail sent to you during that time will be queued up on the sending server for days, until you come back online.

Still, I can walk into the nearest auto parts store and pick up a small, portable 1500W gasoline generator for $200 in 10 minutes. If you're a big enough company that you can't handle e-mail being down for a day, that's a pretty modest price for fairly reliable service. And when the shit hits the fan, just let the IT guy steal a couple janitors or other marginal employees, and have them alternate between refueling the generator.

For slightly longer outages (eg. natural disasters), heavy equipment shops rent massive and hassle-free diesel generator trailers, which can run an entire medium-sized office, nonstop. They hold upwards of 80 gal and run at run at high loads for a full couple days between refueling.

a fall over mail server,

You'd probably get off cheaper and more reliable with TWO low-end servers rather than just ONE high end server.

and a 24/7 IT staff and I don't mean some poor guy with a cell phone and no life.

Why not? If somebody's going to come in within an hour, and be able to take care of it (one way or another) in an couple more, your business probably isn't going to grind to a halt.

In case it isn't clear, a few years ago I WAS that IT guy (with no life... Hmmm) in a small office when the electrical panel at work caught fire, and took a couple weeks to replace. With a pickup truck and plenty of cash on hand, you can handle any situation. Keep all your receipts.

It is Beta...

[flyingfsck]

The advantage of cloud computing is that when it goes poof, your competitors have gone poof too...

It's per slogan!

[David Gerard]

"Don't Be Evil."

Incompetent, now ...

Re: Sig

[Bryansix]

It is to those who defend it religiously. Look up the definition of the word religious. Nevermind, I'll do it for you.

3 a: scrupulously and conscientiously faithful b: fervent , zealous

Re: Sunny!

[TaoPhoenix]

Icarus, is that you falling?

Beta?

[tuituiman]

My business pay for use of Gmail. And when it goes down so do we. That's not such a good thing considering we're a Funeral Palour, and without e-mail we can't embalm, prepare, run a post mortom etc as our paperwork comes through the system. I realize Gmail's in Beta, but they have defined their own definition for "Beta", which basically frames Gmail as a full stable system with more stable functions to come. Shame on you Google. And now to lighten the mood... Why did the chicken cross the road 12 times? His suspenders were stuck to the lamp post. Not funny? so sue me. lol.

Re:Beta?

[Tony Hoyle]

You're paying less than $1 a week per user. If email was important to you you'd be paying a *lot* more than that. Stop being a cheapskate.
 

Re:Beta?

[tuituiman]

Actually it's considerably more. Our whole domain is hosted through Google.

five nives is not cheap.

[ChrisA90278]

To all those who say "Set up your own email server", I ask, "Have yu ever tried it". It is easy to set up just one server but it you want 100% uptime then yu are going to have to set up at least three MX records in your DNS and have each point to a different geographical location servered by a different ISP. Then at each location you need a pair of cross strapped router/switches that control fail out or do round robbin. Then you need a set of servers. Each of those would be dual power supplies, RAID and so on. OK so you CAN do this but at what cost? You are looking at about $100K to tand up something like this if you don't count your time.

Yes yu can set up a Linux box running sendmail but will you have 100% uptime. What if your Internet connection goes down? or the building catches fire? No onw can do 100% but "five nines" can be done (that 99.999%) but five nines is not cheap.

Re:five nives is not cheap.

[FatLittleMonkey]

It's a false dichotomy to compare google and "five nines". Google promises only three, has delivered two, and will give you 15 days credit(*) if they fall to one and a half.

The question is, can you do better than 95% uptime for about $48 per user-account? If you can, you are a 'tard for going with google.


(*) about $2 value and only if you apply for it.

Re: usb drives, clouds...

[TaoPhoenix]

C. Other.

The online copies are backups. When your laptop went AWOL, you go to some new computer, download them, then do your thing.

Go redundant. When your laptop isn't available, these new phones can sometimes process your actual documents. We're one generation short of proper usability on this front. That will be fixed in about 2 years.

Phone not an option? Get a "disposable desktop". You know, some piece of junk for $100. There's a huge influx of machines due to hit this maturity stage within the same next two years when HeavyOS drives upgrades.

My USB drive is my watch. It's strapped to my arm. So unless I'm a twit and take it off, it's essentially unloseable. Oh look, I lost it. Here's one on my car key chain. Awww, I got mugged. Maybe in 10 years they'll be doing subcutaneous mods. (Gee. My beer belly holds 4 terabytes.)

Marshmallows!!

[mujadaddy]

If Google's datacenter burns to the ground

Is that a suggestion?

Re:If you need something done right do it yourself

[suggsjc]

my MySQL server is having problems

I can fix your problem in 10 steps...

outage

[Anonymous Coward]

Hmm, With only a 1/2 day outage and based on the price for all they offer it still seems to be a good deal. When your paying for it, you can bitch and moan.

Beta

[Itchyeyes]

Guess we now all know why it's still in Beta.

imap.gmail.com

[VGPowerlord]

imap.gmail.com is still operational. Perhaps people should use an IMAP client instead of the web interface.

Re:If you need something done right do it yourself

[Iron E]

If the company does have a competent IT department, the boss will at least know what the problem is, that it is being worked on and get an estimate when the service is expected to be back. Afterwards he'll get a report why it happened and what is being done to minimize the chance of a similar outage in the future.

I'm not saying all IT stuff should be done in-house, but if you go with a giant like Google (for whom Gmail and GoogleApps isn't their main business), this negligible transparency seems the most you can expect.

You'll have to clarify

[Anonymous Coward]

Google has a different definition of beta...

that's what

[Chutulu]

you get by using Linux servers ;)

Hmm...

[oljanx]

It appears that during this reported outage the (my) iGoogle page layout has changed, and the GMail notifier is not functioning although I have no problem accessing my GMail account. It looks like there's change in the wind, and those changes didn't roll out as seamlessly as someone hoped.

Re:Hmm...

[man_ls]

Gmail Notifier stopped working on any of my systems about 3 months ago. I'm not sure why. "The service is temporarily unavailable" is all it tells me.

Still more reliable than Exchange

[DustoneGT]

I'll gladly put up with the occasional outage...it's better than the almost weekly MS Exchange bugs on a crappy MS Windows Server system.

Re: Sig

[JesseMcDonald]

Not everything defended "religiously" is a religion. Definitions of religion. The first two definitions given (from wordnet.princeton.edu) sum it up nicely (emphasis mine):

• S: (n) religion, faith, religious belief (a strong belief in a supernatural power or powers that control human destiny) "he lost his faith but not his morality"
• S: (n) religion, faith, organized religion (an institution to express belief in a divine power) "he was raised in the Baptist religion"; "a member of his own faith contradicted him"

Not All Clouds Are White and Fluffy

[Amazing1]

Not all clouds are white and fluffy, some can get quite stormy. This one seems to have a hurricane blowing by it at the moment. The only thing you trust a cloud to do is to rain on your parade.

Too bad, so sad, serves them right!

[billcopc]

That's what they get for not having an SLA.

In today's bastardized world, if you don't have something in writing, you have absolutely nothing.

If a small fry were to screw up this bad, they would be afraid of their phone thanks to the many passive-aggressive office drones complaining repeatedly. Not that it helps in any way, but I'm pretty sure Google's techs aren't being harassed with phone calls every 5 seconds. I hate to enable the lusers, but when you've got paying clients breathing down your neck, you tend to take whatever measures necessary to fix the problem asap.

If anything, this should send the message that Google, contrary to popular belief, is not invincible. They mess up just like everyone else, which means maybe they're no better than anyone else.

Well . . .

[The Second Horseman]

We had under two hours of downtime planned and unplanned - on our enterprise email system in the last two years. Including moving all the data from one SAN array to another without the benefit of an array-based replication solution. Our unplanned downtime has been minutes. And not with a very expensive system. Four modest 1U servers, clustered, and a mid-range fibre channel array. In terms of staff time, it's a fraction of an FTE (not even half) to manage it. Oh, yeah, and I know the backups are good.

Re:Well . . .

[bbn]

Good for you. I guess you have not had to use that backup from a total SAN failure yet. Likely you never will as I assume you have good equipment.

If you where to scale up your setup to gmail size, that would have happened. And you would not have been able to claim only two hours downtime.

Re:If you need something done right do it yourself

[TheQuantumShift]

"You can't count on Outsourcing to run your IT...sorry buddy. Using Outsourcing may be cost effective, but the obvious trade off is that someone else is really doing your job, and if that person drops the ball, then you really screwed the pooch, at least that's what your boss will think."

Fixed that for you.

It is?

[alisson]

Is it just the site? I'm still getting gmail through POP access...

email - not 'mission critical'

[Anonymous Coward]

Unless you provide email services, email is not mission critical. You might want to check your company 'Mission Statement'.

I work for a hospital, our mission is to save lives. With over 8000 users, you can imagine the email volume. But work does not stop if email goes down.

If your company makes nails, when the nail making machines stop making nails, you have a mission critical issue.

Just ask your auditors next time they come around. They are not worried about your email servers, trust me, they are not 'critical'.

What cavalier and unprofessional attitude.

[jotaeleemeese]

My business suffers, then I sit down and I let some unreachable, unaudited Engineers solve the problems for us.

I will tell you something. If my company did not have email due to an outsourced service not working, my most important job would be to get in the phone with somebody that would tell me what the heck is going on.

Next day I would be asking some very serious questions and awaiting the right answers.

When an internal mail srver fails .....

[jotaeleemeese]

.... the only thing that counts are the priorities of your own company. You decide how much work you put on those servers and how fast you need them back.

When an outsourced service fails, the outsourcing company queues your concerns in a macabre balancing game in which they are wearing 1000 hats to please 1000 different costumers.

Each company should decide if this is acceptable, the mistake is to defend Google (or any other outsourced service) as the absolute best solution for everybody.

Blaming? Blaming is not the point

[jotaeleemeese]

When a serious company has problems, blame is the last thing everybody is thinking about.

The task would be to be back on business, with Google you have a third party with its own interests at stake.

I that is acceptable to your organization or not only you know, but if your main concern is blame you are truly starting in the wrong place.

Oh fucking please.

[jotaeleemeese]

The Engineers at Google are not re-enacting the 2nd coming of Jesus the Christ.

There are plenty of people out there that understand email systems as well or even better than Google's Engineers.

It is not like Google invented the smtp protocol yesterday ...

No company that needs that will use Google.

[jotaeleemeese]

Most likely they are regulated and an outsourced service would be almost surely out of the question.

Re:If you need something done right do it yourself

[Sentry21]

It's not a question of dropping the ball - Google's services are vastly inferior to any other service I've used (or run myself).

Every day someone will come up to me with a problem. Often they won't be able to get their e-mail for an hour or two. Sometimes the server will say there is no 'INBOX' folder. Sometimes mail won't be delivered for hours, or even until the next day. Sometimes it won't show up at all. In every case, I've checked and re-checked their settings, but when we're sending with their SMTP servers and fetching from their IMAP servers, how is mail getting lost or delayed for hours?

Lately, they've delivered all mail destined to me from our ISP to someone else in the company, and can't figure out what's happening.

Labs has a new feature where you can exclude 'folders' from showing up in IMAP, or you can tell it to delete them when you 'delete' them in IMAP (instead of just hiding them but they still show up in All Mail). Except you can't use Google Maps on Google Apps accounts, so all of our Mac users are relegated to downloading every message twice, or changing their INBOX prefix.

Hell, half of our users forward their work e-mail to a personal account and deal with everything there. Some of our employees don't even use their work e-mail, and just give everyone their personal e-mail address.

When I get a phone call at 8 AM from someone saying 'Outlook isn't working, can you fix it?' and I have to say 'no', there's something horribly wrong. We're switching away from Google, and I'll be glad when they're far, far behind us.

Exactly - read these other recent headlines

[walterbyrd]

Google's "unhealthy dominance" could harm search market

CIO - Could deploying Google Apps be a career-limiting move?

Google Apps Hits Snag in the Enterprise

Google Apps is risky business

Are Your Privacy And Security At Risk With Google Apps

Google Chrome Considered Harmful

And so on. There is no mistaking the stench of a msft fud storm.

Have you noticed the msft shills all over slashdot? Happens every time there is a hot-button issue like this.

Re:Exactly - read these other recent headlines

[Anonymous Coward]

Have you noticed the msft shills all over slashdot?

Your next post WILL be one of the following:

1. A list of these "msft shills", complete with concrete, NON-speculatory, incontrovertible evidence that they are "msft shills".
2. An abject, unconditional confession that your "shill" accusations are NOTHING but juvenile, incompetent lies designed to support a childish false dichotomy where anyone who isn't in complete agreement with you is one of the "bad guys".

Those are your ONLY possible choices.

Beta and Free?

[us7892]

It's free, and Beta. So you lose email for a few days.

Re:If you need something done right do it yourself

[gandhi_2]

Very well put.

In the Army, we say "you can delegate authority, not responsibility."

The lesson here:

[denttford]

Admins need alcohol to function.

That will be the day ...

[justinlee37]

What if Google decides that GMail no longer fits their business model?

That will be the day that a flying pig successfully chases an asbestos cat through hell.

100x$50 is...

[SmallFurryCreature]

Wait a minute, I can do this. Don't help. 100x$50 lets see. eh, $5000. Did I do it right.

For a 100+ person company 5k should be peanuts. To get email for everyone in that company? You can't even buy a decent server for that. An admin for your own server would costs at least that per month and that gets you a guy who only works 40 hours a week.

If 5k is a siginificant amount of money in your 100+ company, then you need to look for a new job.

you're also 50x more expensive

[Anonymous Coward]

Google Mail/Apps is priced like a commodity. There will be clients who need this level of service... but not that many.

expectations

[stefancaunter]

the customer with no mail has unrealistic expectations. when this is getting set up, it needs to be pointed out that it isn't controlled internally. free, but comes with risks. when i deploy google apps, i stress the privacy and advertising angle more than the reliability as far as potential issues to consider. the reliability speaks for itself, it is what it is, and what it will be. places need to assess if they want to plow dollars into their own infrastructure, or use the dollars on better people. for some places, the risk is worth it to get free email for their domain. since apps recently tied domain mail to outlook and other clients, a reason to not choose it was removed. an honest evaluation of the service needs to be given before a decision is made. here's what they do, here's what you get, here's what it would cost for me to do this in house.

The CEO... got what he paid for

[UnixUnix]

(all right, just kidding... )

To get a service go to a service company

[pdeschutter]

What people havn't understood yet is that Google is a product company and not a service company. Despite all the nice talk about consumerisation of IT, corporate cusomers want a service company. A company that picks up the phone, engages itself with an SLA and actually puts the Customer First. Google is miles away from such a corporate culture. This does not mean that the SAAS market does not have a very bright future, just that the winners will probably be focused companies considering this their core busines. Check out companies such as http://www.contactoffice.com/ http://www.37signals.com/

Re:clouded computing

[SenseiLeNoir]

the 7/7 terrorism killed a lot of people. However, it still doesn't compare to the amount killed on the trains by Railtrack/British Rail/ et all due to incompetence, and poor maintenance.

And both those two figures don't compare to the number of people killed due to drink driving by idiots who think they can still drive after a few pints, or other forms of negligent driving.

Yet i dont see as much of a crackdown on these idiots...

Why, because when a terrorist does something, even once in a blue moon, it makes big news.

When a small child is killed by a drink driving "terror", it rarely makes a blip.

ITs the same with your this topic. When you computer breaks down, it does so pretty often, yet hardly makes news. When an office server goes down (less frequent) the effect, and inferences are higher. when gmail goes down (much less rarely, considering) it makes big news.

All we need is perspective.

Re:clouded computing

[xaxa]

"Train crashes at 90 miles an hour, old woman killed" plus the associated pictures sells newspapers much better than "Drunk driver kills pedestrian". Maybe when the media starts printing headlines like "3000th road death this year" something more might happen.

Also, "Road death toll hits record low" seems to be news http://news.bbc.co.uk/1/hi/uk/4636913.stm
But "Railways safer than ever" isn't.

Re:If you need something done right do it yourself

[ptbarnett]

56kpbs is more than enough to handle e-mails (provided you don't constantly send huge attachments like legal documents).

I don't send huge attachments. But, other people in my company don't think twice about sending huge attachments, because it has no discernible effect on the office LAN and they had no consideration for the folks in the field.

I was repeatedly frustrated by the inability to read the rest of my email because I was waiting for a multi-megabyte attachment to download. Older versions of Outlook didn't handle partial downloads correctly, or didn't even support message caching.

I finally solved it with a combination of things, not the least of which was avoiding dial-up altogether. I haven't used a dial-up modem in years: I now use Sprint Broadband in the field.

Tell the CEO...

[NateTech]

Welcome to out-sourcing, ass-hat.

You knew the risks of trusting someone else to run your mail system, going in.

And if you didn't... you're a moron.

Ask him if he's been enjoying not having to pay for servers, power, admins, etc.

You don't need Gears

[Anonymous Coward]

Why no Gears for offline Gmail access at very least, Google?

Well, that's because Gmail includes free POP3 access that you can turn on. So if you want offline email access for redundancy, just use your email client of choice (Thunderbird, Outlook, etc. all work just fine).

Re:If you need something done right do it yourself

[evilviper]

I was repeatedly frustrated by the inability to read the rest of my email because I was waiting for a multi-megabyte attachment to download.

Yes, it's annoying when you have to sit there and wait for it, but that's not a problem with the server on-site... Delivery just happens in the background, no hassles or waiting as far as you're concerned.

Even with non-stop huge attachments, a dial-up line will still keep your e-mail flowing, it'll just be delayed by a couple hours at most. It should all get through by some time in the middle of the night, once everyone has gone home and stopped sending. And the next morning the queue will be empty, and delivery will again be almost immediate.

We're not talking about recommended daily procedures here, just a way to get through a rare, occasional outage.