Slashdot Mirror

← Back to Stories (view on slashdot.org)

'Why You Should Not Use Google Cloud' (medium.com)

Posted by BeauHD on from the time-is-money dept.

A user on Medium named "Punch a Server" says you should not use Google Cloud due to the "'no-warnings-given, abrupt way' they pull the plug on your entire system if they (or the machines) believe something is wrong." The user has a project running in production on Google Cloud (GCP) that is used to monitor hundreds of wind turbines and scores of solar plants scattered across 8 countries. When their project goes down, money is lost. An anonymous Slashdot reader shares the report: Early today morning (June 28, 2018) I receive an alert from Uptime Robot telling me my entire site is down. I receive a barrage of emails from Google saying there is some "potential suspicious activity" and all my systems have been turned off. EVERYTHING IS OFF. THE MACHINE HAS PULLED THE PLUG WITH NO WARNING. The site is down, app engine, databases are unreachable, multiple Firebases say I've been downgraded and therefore exceeded limits.

Customer service chat is off. There's no phone to call. I have an email asking me to fill in a form and upload a picture of the credit card and a government issued photo id of the card holder. Great, let's wake up the CFO who happens to be the card holder. What if the card holder is on leave and is unreachable for three days? We would have lost everything -- years of work -- millions of dollars in lost revenue. I fill in the form with the details and thankfully within 20 minutes all the services started coming alive. The first time this happened, we were down for a few hours. In all we lost everything for about an hour. An automated email arrives apologizing for "inconvenience" caused. Unfortunately The Machine has no understanding of the "quantum of inconvenience" caused.

9 of 508 comments (clear)

  1. Sorry, but... by Known+Nutter · · Score: 5, Insightful

    If millions of dollars are on the line, you should be running your own systems. Seriously. I'm not an IT expert, data infrastructure guy or anything. I'm just a dumb nerd, and I know that. Never trust your data to a third party when millions are at stake -- let alone critical infrastructure reliability.

    --
    Beware of the Leopard.
    1. Re:Sorry, but... by Nutria · · Score: 5, Insightful

      As if servers doing down can't happen if you host it yourself.

      But then you're in control, instead of having to rely on some amorphous, anonymous monster that only allows communication via automated email.

      --
      "I don't know, therefore Aliens" Wafflebox1
    2. Re:Sorry, but... by Anonymous Coward · · Score: 5, Insightful

      That's why you also configure a failover at a secondary location. This server punching guy sounds like a damn idiot. With millions on the line why is there no disaster recovery plan outside of "call the CFO for his credit card?"

    3. Re:Sorry, but... by AmiMoJo · · Score: 5, Insightful

      Google's cloud services are enterprise grade if you pay enterprise prices for them.

      If you pay for it on a credit card assigned to the CFO then you are not an enterprise and you are not paying enterprise prices.

      They chose a cheap, no-SLA no-support service, probably because it was cheap. Then they get upset that they aren't receiving the support they didn't pay for.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    4. Re:Sorry, but... by Anonymous Coward · · Score: 5, Insightful

      As if servers doing down can't happen if you host it yourself.

      But then you're in control, instead of having to rely on some amorphous, anonymous monster that only allows communication via automated email.

      Exactly right. If your servers go down, YOU can fix them immediately and you don't have the problem of trying to get in touch with some support person who may or may not respond quickly, or may not respond at all if you can't even figure out how to get in touch with them.

      That said, this was a case of a company trying to be cheap, and instead of an enterprise account they tried to run critical applications on what is supposed to be a consumer-level account, which is why they started getting e-mails from Google about "suspicious activity".

  2. Wait by Anonymous Coward · · Score: 5, Insightful

    The first time this happened,...

    Why was there a second time?

  3. Re:Amazon's cloud s no better by fahrbot-bot · · Score: 5, Insightful

    ... but Amazon is a disaster for a company that isn't fully fault tolerant and has critical servers that can't go down.

    If your company has "critical servers that can't go down" (wherever they are) and you're not fully fault-tolerant, you're the disaster, not Amazon.

    Not here to pick a fight, just sayin'. (one finger pointing at someone else is also three fingers pointing at yourself)

    --
    It must have been something you assimilated. . . .
  4. Yeah, big warning signs on the user side here by Phil+Urich · · Score: 5, Insightful

    The first time this happened,...

    Why was there a second time?

    So many of the problems here (ex. paying with a credit card and one that has only a single person's name on it? Having no fallback that can be spun up elsewhere?) are foolish if this has never happened before, and utterly, mind-bogglingly idiotic if this in fact has already happened before. It's one thing to be blind of something you should know could be a problem, it's quite another to be blind and wholly unprepared for a problem you've personally experienced! Something seems fundamentally wrong at this company.

    Also, if your entire business can die because it takes an unexpected few days off, then perhaps your business is running a bit too raggedly and doesn't have enough meat on the bones . . .

    --
    I remember sigs. Oh, a simpler time!
  5. Cheap service, cheap results by fyngyrz · · Score: 5, Insightful

    The company thought they could get away with paying less for server infrastructure. They can. But they get less. This is one of the "less" things they get.

    If you value your data, host it yourself, preferably in multiple locations. If you want to go cheap, then you can expect to lose things.

    Like your data, or access to it, or availability of it.

    It's not such a smart thing to cheap out on the important stuff.

    Of course, convincing the bean counters of future risk inherent in what appears to them to be current savings... good luck with that.

    Well, best to get rid of your bean counters. :)

    Here's a maxim of mine I like to drop on the table during discussions like these:

    If you can't afford to do it well, you almost certainly shouldn't be doing it at all.

    --
    I've fallen off your lawn, and I can't get up.