Businesses Moving From Amazon's Cloud To Build Their Own

itwbennett writes "There are rumblings around this week's OpenStack conference that companies are moving away from AWS, ready to ditch their training wheels and build their own private clouds. Inbound marketing services company HubSpot is the latest to announce that it's shifting workloads off AWS, citing problems with 'zombie servers,' unused servers that the company was paying for. Others that are leaving point to 'business issues,' like tightening the reins on developers who turned to the cloud without permission."

Comments

Nor surprising and won't matter.

[serviscope_minor]

It doesn't surprise me and I don't think it will matter much.

Amazon is not particularly cheap. If you host your own, even with power, cooling and hardware, the payback time is about 4 to 6 months.

If you have a lot of load then it is going to be cheaper to host it yourself, so it's worth doing for big companies.

With Amazon of course you can start as a one man band and still have potential to grow without it getting painful from an administrative point of view.

Re:Nor surprising and won't matter.

[CastrTroy]

The only case where it really made sense was when you had extremely variable load. It's nice for scientists that need to rent 100 computers for use with one project, but if you're going to be using the same resources on a day-to-day basis, then it makes much more financial sense to just own your own hardware, and rent space in an existing data center. It also makes sense if you use less than a whole server in resources, but VPS was already filling that need quite well before Amazon came along.

Re:Nor surprising and won't matter.

[serviscope_minor]

The only case where it really made sense was when you had extremely variable load.

Indeed, or if you're expecting to scale. The thing is, as you scale up, you can always move the baseload to dedicated servers and just do the variable part on Amazon.

Re:Nor surprising and won't matter.

[thereitis]

If you're just using Amazon for compute power then perhaps, but then you've got no geographic redundancy with that single data center. Whether it's worth rolling your own solution really depends on your needs (lead time, uptime requirements, budget, IT skill/availability, etc).

Re:Nor surprising and won't matter.

[Dancindan84]

The thing is, when a company reaches a certain size they likely have a enough computer infrastructure to have an IT department anyway, even if they aren't an IT company. With your example of Ford, they have offices for managers, sales etc. All of those people likely have desktop computers, so they likely have dedicated desktop support. Additionally they probably have some kind of centralized authentication like active directory, which means they'll need a server and some sort of sys admin/IT infrastructure already. They likely wouldn't be adding an IT division in order to host their own email, they'd be adding an email server/management to the load of the existing IT department, which is obviously not as big an upfront overhead cost, making it more attractive.

Re:Nor surprising and won't matter.

[serviscope_minor]

Take for instance a large company like Ford (picked because they aren't a computer/technology/web based, but large company). Their expertise has nothing to do with computers.

Are you sure about that?

A large company must have many many areas of expertise. Obviously their goal is to make cars. But have expertise in cars, large scale manugacturing, logistics, marketing, engineering, anything required to support engineering including simulation running on supercomputers, human resources and probably a whole bunch I haven't thought of.

The point is that many of them will involve computers to a large degree, so although a company like Ford makes no money with computers per-se every area of their operation will involve computer systems. As a result they will have a huge amount of computer expertise.

Re:Nor surprising and won't matter.

[drinkypoo]

The defining factor is whether you can keep more than one IT guy busy full time. If you can, then you hire at least two, one senior and one junior to at least fight fires when he's sick. If you're keeping at least two IT buys busy full time, you're going to be paying for them whether they work for you or not, but if they do work for you then you can fire them, so you have some control over what they do. If they'll just be placed with someone else if you don't like them, they're not going to work as hard for you. You need as much control as you can get over your own IT department. It's daft to contract out anything so critical when you're only adding to the likelihood of leaks and malfeasance.

Re:Nor surprising and won't matter.

[7213]

Regarding Ford specifically.

You'd be surprised at the scale of their IT organization (as someone who once worked in Ford's datacenter).

They already have their own 'internal cloud' and have for some time (before 'cloud' was a 'thing'). The only thing different here is internal provisioning processes vs. Amazons credit card & go plan.

The cost of Amazon doesn't make sense, when you already have a pair of tier 1 datacenters and an IT organization more then capable of maintaining it.

Ford already HAS servers that won't be 'clouded' any time soon, so they have every bit as much justification to keep on doing things internally as Amazon would. And doing things for themselves gives them more control & likely better costs.

Re:Nor surprising and won't matter.

[hawguy]

neither does amazon unless you pay them a lot more $$$

Depending on your needs, setting up geographical redundancy with Amazon can be extremely cheap -- if you just want a cold or warm site to fail over to, you don't need to keep your entire infrastructure running at the secondary site, just replicate the data, and then spin up the servers over there when you need to fail over.

That's what my company does - we have about a dozen servers to run our website, but the secondary site has only a couple micro instances to receive data. When we need to failover, we just tell one of those servers to wake up the rest of the infrastructure and update the databases from the snapshots that have already been transferred over, including repointing DNS to the backup site. We could make the failover fully automatic, but are afraid of "split brain syndrome" leading to the failover site taking over when the primary is still fine so it's still a manual process. Our backup site is never more than 15 minutes out of date from production.

This has worked well in testing - we've done some "live" late-night failovers and it's relatively seamless -- since it's so cheap to set up the backup site (essentially we just pay for the cost of storage at the backup site), we're going to set up another region overseas for extra redundancy.

Re:Nor surprising and won't matter.

[hawguy]

Depending on your needs, setting up geographical redundancy with Amazon can be extremely cheap

And history has shown that you pay for what you get.

Right, if you cheap out and pay for a single availability zone in a single region, when that AZ or Region goes down, your site is down.

If you pay for multi-AZ and Multi-region deployments you get much better availability.

Just like Amazon says.

Over the past 2 years, Amazon has been more reliable than the coloc we moved away from, mostly due to the triple (!) disk failure that took out our SAN RAID array - one disk failed, and while we were waiting for the replacement, another disk went down, after we replaced those two, another disk went down while rebuilding the RAID-6 array.

With AWS, an entire region can go offline and we can bring up the backup site on the other side of the country (or, starting next month, we could bring up our Ireland region).

All this for less than half the cost we were paying for the coloc + equipment maintenance.

The obvious next step...

[lxs]

...will be to give every user their own personal cloud housed in a box under their desk.
At which point the cycle will begin again.

Re:The obvious next step...

[benf_2004]

...will be to give every user their own personal cloud housed in a box under their desk. At which point the cycle will begin again.

That sounds like a great idea! We can call it a Personal Cloud, or PC for short.

Re:The obvious next step...

[Richard_at_work]

Why is there *always* a snarky comment along these lines whenever someone talks about not using a "public" cloud provider - cloud when talked about in these ways does not mean "someone else owns the hardware", it means "an infrastructure setup which means I do not have to care about the infrastructure when deploying applications", whether that be owned by someone else or an internally provided solution.

The old manner of inhouse application infrastructure involved one or more application server, one or more database server, and the related network and service architecture specifically required to handle redundancy and failover - but the point is, you had to care about that service architecture when dealing with your app! Which server had spare resource to act as a failover for another application (which invariably meant you ended up with two servers allocated for the job anyway, the main and a dedicated backup or two servers which took requests on a round robin manner), which server was not to be used for these purposes, which applications do not live together etc etc.

Today, the goal is to have a "large number of essentially commodity hardware servers" acting on a level which you can forget about for most solutions (there are always going to be situations where heavily tailored hardware solutions will still exist) - where you can treat the hardware as what it should be, a resource to be used and allocated as required.

Virtualisation was the first step (in modern terms, not talking about mainframes here), and cloud takes the aspect of virtualisation several steps down the road.

This has got sod all to do with the "cycle", and everything to do with "computing as a resource".

business forecast: cloudy

[OffTheLip]

Businesses don't want to miss the next big thing but like most decisions, time will tell. "I've looked at clouds from both sides now, From up and down, and still somehow It's cloud illusions i recall. I really don't know clouds at all"

Re:business forecast: cloudy

[dkleinsc]

On the upside, it makes it now possible for a business to say "Hey (hey), you (you), get offa my cloud!"

Maybe not completely

[Gripp]

I work at one such company. We recently setup openstack and plan to eventually use it for our production environment. But ec2 will still stay in the picture. Both for services were the end user needs more direct access to the machine and for failover purposes. I just don't know that openstack means the end of ec2.

Tightening reins on developers?

[cryfreedomlove]

From this article: "like tightening the reins on developers who turned to the cloud without permission"

Let me state this in other words: "Insecure IT guys are afraid for their own jobs if they can't lord it over developers". Seriously, developers working in an API driven cloud just don't need a classic IT organization around to manage servers for them. Cloud is a disruptive threat to classic IT orgs.

Re:Tightening reins on developers?

[SimplyGeek]

That's not always the case. Look at workplaces that fall under HIPAA regulations. That last thing IT wants is for developers to start up their own app projects in the cloud, whereby their apps start accessing PHI/PII. The moment that PHI goes from the local intranet and those bits go onto a 3rd party cloud service, the company will shit a brick because the developer's just violated regulations. There's a reason IT and security requires oversite of app development.

Different needs for different scales

[slim]

How hard is it to understand that the cost/benefit depends on your size?

Car analogy: If you're an individual who needs a car a couple of times a year, you rent one on those occasions. If you drive almost every day, you buy a car and you get it insured. If you're a small company, you give your travelling staff a car allowance. If you're a big company, you buy a company car scheme and insure all the cars under one policy. If you're a gigantic company, you self-insure all your staff's company cars.

Draw a graph of the cost vs scale of a third-party cloud, versus your own datacentre. At some point the graphs will cross. That's where you switch.

cowboys like you

[onyxruby]

I've reined in cowboys like you for years, from one fortune 500 to another. Arrogant jackasses that can't be bothered with change management, best practices, version control, documentation, pesky things like policies, regulations and laws. Self righteous developers that can't see past their own nose too see how thier actions or inactions affect those around them.

Every single time they think they are above these things and that they know better than the industry around them. They never realize why something that works in their special environment works perfectly fine where they have the rights of a God but has all kinds of mysterious errors in production where there they are brought back down to earth. They then chafe when their development environment is set up identical to production, yet it is amazing how quickly previous mysterious bugs that plagued production and caused incredible operational costs suddenly get fixed. They of course never have to clean up multi-million dollar messes, talk to regulatory agencies, sit down with lawyers to plan how to mitigate their mess or have a face to face with an angry Attorney General.

I've only won this argument and helped companies save millions by reining in the cowboys like yourself a couple dozen times. Probably something to do.with cleaning up large multi-million dollar messes more than once.

Random pricing

[EmperorOfCanada]

One thing that has kept me away from Amazon's cloud is the unknowns with its pricing. I have visions of a DDOS either clearing out my bank account or using up my monthly budget in the first 2 days of the month. Plus if I mis-click on something I might get an awesome setup that cleans me out. I am not a large corporation so one good bill and I am out of business. But even larger companies don't like surprises. So regardless of the potential savings I am willing to spend more if the price is fixed in stone instead of chancing being wiped out. I like sleeping through the night.

Plus as a human I really like being able to reach out and touch my machines, even if I have to fly 5 hours to do it. So the flexibility of the cloud sounds really cool where the pricing is not so flexible. It would be nice to spool up an instance of a machine that isn't going to do much most of the time that doesn't actually use up a whole machine. But then when one machine starts to get pounded to give it some more juice. Plus upgrading your hardware would be much more of a dream. You move your most demanding servers to your hottest hardware and slide the idle servers over to the older crap. Plus restores and redundancy are a dream.

Then you still have the option to fully dedicate a machine in "realspace" to a demanding process. While VM does not have much overhead it does have some. So taking a server(s) that is being pushed to the maximum and sliding it onto bare metal will then allow your hardware to be used to maximum efficiency.

Then by having no real cost overhead to having more near idle machines spool up your developers can play interesting games. Maybe they want to see what your software will do with 20 MongoDB servers running instead of the current 3; or 200.

This all said, I am a fan of Linode; where I can predict my pricing very well.