Slashdot Mirror

← Back to Stories (view on slashdot.org)

Businesses Moving From Amazon's Cloud To Build Their Own

Posted by samzenpus on from the on-my-own dept.

itwbennett writes "There are rumblings around this week's OpenStack conference that companies are moving away from AWS, ready to ditch their training wheels and build their own private clouds. Inbound marketing services company HubSpot is the latest to announce that it's shifting workloads off AWS, citing problems with 'zombie servers,' unused servers that the company was paying for. Others that are leaving point to 'business issues,' like tightening the reins on developers who turned to the cloud without permission."

39 of 121 comments (clear)

  1. Nor surprising and won't matter. by serviscope_minor · · Score: 4, Insightful

    It doesn't surprise me and I don't think it will matter much.

    Amazon is not particularly cheap. If you host your own, even with power, cooling and hardware, the payback time is about 4 to 6 months.

    If you have a lot of load then it is going to be cheaper to host it yourself, so it's worth doing for big companies.

    With Amazon of course you can start as a one man band and still have potential to grow without it getting painful from an administrative point of view.

    --
    SJW n. One who posts facts.
    1. Re:Nor surprising and won't matter. by CastrTroy · · Score: 4, Interesting

      The only case where it really made sense was when you had extremely variable load. It's nice for scientists that need to rent 100 computers for use with one project, but if you're going to be using the same resources on a day-to-day basis, then it makes much more financial sense to just own your own hardware, and rent space in an existing data center. It also makes sense if you use less than a whole server in resources, but VPS was already filling that need quite well before Amazon came along.

      --

      Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
    2. Re:Nor surprising and won't matter. by serviscope_minor · · Score: 4, Insightful

      The only case where it really made sense was when you had extremely variable load.

      Indeed, or if you're expecting to scale. The thing is, as you scale up, you can always move the baseload to dedicated servers and just do the variable part on Amazon.

      --
      SJW n. One who posts facts.
    3. Re:Nor surprising and won't matter. by thereitis · · Score: 4, Insightful

      If you're just using Amazon for compute power then perhaps, but then you've got no geographic redundancy with that single data center. Whether it's worth rolling your own solution really depends on your needs (lead time, uptime requirements, budget, IT skill/availability, etc).

    4. Re:Nor surprising and won't matter. by CastrTroy · · Score: 2

      I guess it really depends on what business you are in though. Take for instance a large company like Ford (picked because they aren't a computer/technology/web based, but large company). Their expertise has nothing to do with computers. Now, the question becomes, would it be cheaper for an organization of this size to host their own email? Most likely it would. But the real question is, do they want to devote any corporate time to even dealing with this kind of thing. Basically they would have to have a whole new division added on to their company to handle IT management, and they'd have all the fun stuff that goes along with it. It makes much more sense for someone like Amazon to host their own email because they already have a bunch of people managing servers anyway, so adding a few more servers isn't getting into uncharted territory. I mean, Ford probably uses enough keyboards that it might make sense to have their own keyboard factory. But that doesn't mean its a good business decision or that it's something they want their company to be doing.

      --

      Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
    5. Re:Nor surprising and won't matter. by Anonymous Coward · · Score: 2, Interesting

      we use Azure for a lot of things in my particular department because it helps us bypass our IT department. Sometimes we need to set stuff up really fast and only have it last for a short amount of time. It takes our IT department about a week to open ports on our firewall or map a machine to an IP....when we have 2 days to get something working this doesn't work. As far as cost goes...it isn't all that much more expensive than handling the hardware ourselves. I can also, on the fly, scale things up as I need to. It's a lot easier than buying ram...shutting down the server...getting someone to put the RAM in since I don't have access to NOC. Works perfectly for our needs. I can also run a ton of test web sites for free with Azure and then move them to production as I need to. If they stay under a certain barrier then I don't get charged for them at all since the first like 5 gig of traffic is free once I move it to a dedicated resource. Trying to do this traditionally wouldn't work for us at all...and would make things even worse.

      Your mileage may vary.

    6. Re:Nor surprising and won't matter. by Dancindan84 · · Score: 4, Insightful

      The thing is, when a company reaches a certain size they likely have a enough computer infrastructure to have an IT department anyway, even if they aren't an IT company. With your example of Ford, they have offices for managers, sales etc. All of those people likely have desktop computers, so they likely have dedicated desktop support. Additionally they probably have some kind of centralized authentication like active directory, which means they'll need a server and some sort of sys admin/IT infrastructure already. They likely wouldn't be adding an IT division in order to host their own email, they'd be adding an email server/management to the load of the existing IT department, which is obviously not as big an upfront overhead cost, making it more attractive.

      --
      "Always forgive your enemies; nothing annoys them so much." - Oscar Wilde
    7. Re:Nor surprising and won't matter. by serviscope_minor · · Score: 4, Insightful

      Take for instance a large company like Ford (picked because they aren't a computer/technology/web based, but large company). Their expertise has nothing to do with computers.

      Are you sure about that?

      A large company must have many many areas of expertise. Obviously their goal is to make cars. But have expertise in cars, large scale manugacturing, logistics, marketing, engineering, anything required to support engineering including simulation running on supercomputers, human resources and probably a whole bunch I haven't thought of.

      The point is that many of them will involve computers to a large degree, so although a company like Ford makes no money with computers per-se every area of their operation will involve computer systems. As a result they will have a huge amount of computer expertise.

      --
      SJW n. One who posts facts.
    8. Re:Nor surprising and won't matter. by Jawnn · · Score: 2

      It If you host your own, even with power, cooling and hardware, the payback time is about 4 to 6 months.

      That depends a great deal on the scale and availability demands placed upon your infrastructure. One can deploy a "private cloud" on one or two cast-off PC's, but that will be little more than a toy. If you want to support a serious deployment (dozens or hundreds of nodes) with anything approaching usable performance, you're going to be investing in some serious network and shared storage hardware, not to mention host servers. Want HA? Still more (bigger) bucks. Still, it doesn't take much to make those investments pay. I just think that 4-6 months is a bit too optimistic in all but the most trivial installations. YMMV, of course, but 12-36 months is more like it.

    9. Re:Nor surprising and won't matter. by alen · · Score: 2

      neither does amazon unless you pay them a lot more $$$

    10. Re:Nor surprising and won't matter. by drinkypoo · · Score: 3, Interesting

      The defining factor is whether you can keep more than one IT guy busy full time. If you can, then you hire at least two, one senior and one junior to at least fight fires when he's sick. If you're keeping at least two IT buys busy full time, you're going to be paying for them whether they work for you or not, but if they do work for you then you can fire them, so you have some control over what they do. If they'll just be placed with someone else if you don't like them, they're not going to work as hard for you. You need as much control as you can get over your own IT department. It's daft to contract out anything so critical when you're only adding to the likelihood of leaks and malfeasance.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    11. Re:Nor surprising and won't matter. by AlphaWolf_HK · · Score: 2

      It's often the case that you'll just need one, plus a support contract. The support contract will handle any issues that the IT guy might not be able to do on their own, such as speedy hardware replacement.

      Flexpod is pretty neat in that regard; it has automatic monitoring that will notify the vendor in the event of a perceived imminent hardware failure. They'll begin the process of sending a technician out with the replacement part in hand often times before the admin is even aware that anything is wrong. Doesn't cost anything when it happens, and because of the HA features you have zero downtime while the hardware is being replaced.

      Flexpod (or its competitor Vblock) aint cheap, but you do get what you pay for if you want an in-house HA cloud service.

      --
      Careful with names containing L slashdot.org/~AiphaWolf_HK slashdot.org/~AlphaWoif_HK slashdot.org/~AiphaWoif_HK
    12. Re:Nor surprising and won't matter. by ebno-10db · · Score: 2

      Additionally there is the matter of control. You have a lot more of it if you do your own stuff. However, business often mindlessly follows the latest fashions (this explains why the execs get the big bucks - like fashion groupies). For years the trend has been to move everything out-of-house and buy "services". There recently seems to be some trend away from this, because it turns out the people who warned of the problems were often right (what a surprise). It doesn't help that many companies that offer "services" have become little more than overpriced front companies. For example, much of IBM's US operations have become little more than an overpriced front and unnecessary for India Business Machines.

    13. Re:Nor surprising and won't matter. by 7213 · · Score: 3, Informative

      Regarding Ford specifically.

      You'd be surprised at the scale of their IT organization (as someone who once worked in Ford's datacenter).

      They already have their own 'internal cloud' and have for some time (before 'cloud' was a 'thing'). The only thing different here is internal provisioning processes vs. Amazons credit card & go plan.

      The cost of Amazon doesn't make sense, when you already have a pair of tier 1 datacenters and an IT organization more then capable of maintaining it.

      Ford already HAS servers that won't be 'clouded' any time soon, so they have every bit as much justification to keep on doing things internally as Amazon would. And doing things for themselves gives them more control & likely better costs.

    14. Re:Nor surprising and won't matter. by hawguy · · Score: 4, Interesting

      neither does amazon unless you pay them a lot more $$$

      Depending on your needs, setting up geographical redundancy with Amazon can be extremely cheap -- if you just want a cold or warm site to fail over to, you don't need to keep your entire infrastructure running at the secondary site, just replicate the data, and then spin up the servers over there when you need to fail over.

      That's what my company does - we have about a dozen servers to run our website, but the secondary site has only a couple micro instances to receive data. When we need to failover, we just tell one of those servers to wake up the rest of the infrastructure and update the databases from the snapshots that have already been transferred over, including repointing DNS to the backup site. We could make the failover fully automatic, but are afraid of "split brain syndrome" leading to the failover site taking over when the primary is still fine so it's still a manual process. Our backup site is never more than 15 minutes out of date from production.

      This has worked well in testing - we've done some "live" late-night failovers and it's relatively seamless -- since it's so cheap to set up the backup site (essentially we just pay for the cost of storage at the backup site), we're going to set up another region overseas for extra redundancy.

    15. Re:Nor surprising and won't matter. by hawguy · · Score: 4, Interesting

      Depending on your needs, setting up geographical redundancy with Amazon can be extremely cheap

      And history has shown that you pay for what you get.

      Right, if you cheap out and pay for a single availability zone in a single region, when that AZ or Region goes down, your site is down.

      If you pay for multi-AZ and Multi-region deployments you get much better availability.

      Just like Amazon says.

      Over the past 2 years, Amazon has been more reliable than the coloc we moved away from, mostly due to the triple (!) disk failure that took out our SAN RAID array - one disk failed, and while we were waiting for the replacement, another disk went down, after we replaced those two, another disk went down while rebuilding the RAID-6 array.

      With AWS, an entire region can go offline and we can bring up the backup site on the other side of the country (or, starting next month, we could bring up our Ireland region).

      All this for less than half the cost we were paying for the coloc + equipment maintenance.

    16. Re:Nor surprising and won't matter. by Demonantis · · Score: 2

      Your example is so true though. I have seen several professors use the service to complete a simulation over the weekend when they are on a deadline. Its a tool. You use it when it makes sense. The migration away is from inept companies that didn't do a real business case for the procurement, but have woken up now to the real cost.

  2. The obvious next step... by lxs · · Score: 5, Funny

    ...will be to give every user their own personal cloud housed in a box under their desk.
    At which point the cycle will begin again.

    1. Re:The obvious next step... by benf_2004 · · Score: 5, Funny

      ...will be to give every user their own personal cloud housed in a box under their desk. At which point the cycle will begin again.

      That sounds like a great idea! We can call it a Personal Cloud, or PC for short.

    2. Re:The obvious next step... by Richard_at_work · · Score: 3, Interesting

      Why is there *always* a snarky comment along these lines whenever someone talks about not using a "public" cloud provider - cloud when talked about in these ways does not mean "someone else owns the hardware", it means "an infrastructure setup which means I do not have to care about the infrastructure when deploying applications", whether that be owned by someone else or an internally provided solution.

      The old manner of inhouse application infrastructure involved one or more application server, one or more database server, and the related network and service architecture specifically required to handle redundancy and failover - but the point is, you had to care about that service architecture when dealing with your app! Which server had spare resource to act as a failover for another application (which invariably meant you ended up with two servers allocated for the job anyway, the main and a dedicated backup or two servers which took requests on a round robin manner), which server was not to be used for these purposes, which applications do not live together etc etc.

      Today, the goal is to have a "large number of essentially commodity hardware servers" acting on a level which you can forget about for most solutions (there are always going to be situations where heavily tailored hardware solutions will still exist) - where you can treat the hardware as what it should be, a resource to be used and allocated as required.

      Virtualisation was the first step (in modern terms, not talking about mainframes here), and cloud takes the aspect of virtualisation several steps down the road.

      This has got sod all to do with the "cycle", and everything to do with "computing as a resource".

    3. Re:The obvious next step... by CastrTroy · · Score: 2

      With the specs of some of the desktops coming out, they almost are a personal cloud. For about $2000, you can get a machine with 64 GB of RAM, 6 cores (12 if you count hyperthreading), dual SSDs for some speed with redundancy, + 2x1 TB hard drive for large capacity storage, and a pretty decent video card. I remember when $2000 would buy a modest computer, and I'm not even that old.

      --

      Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
    4. Re:The obvious next step... by lxs · · Score: 2

      I think somebody needs a hug.

    5. Re:The obvious next step... by drinkypoo · · Score: 2

      If you drop the 64GB of RAM to 16GB you can get all that for about $800. That's still loads of headroom.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    6. Re:The obvious next step... by Attila+Dimedici · · Score: 2

      The thing is that the term "cloud computing" derived from the way that network architectures were drawn out for visualization purposes. You drew out a diagram that consisted of boxes of different descriptions representing devices on your company's network and how they connected to one another. Then you had a line that went out to the Internet, which was represented by a cloud because you had no idea what devices your communication passed through and you did not care. I have never really liked that metaphor for putting virtualized computer services on someone else's hardware, but I understand it. If I am hosting virtual machines on your hardware, I don't care how they communicate because if I run into a problem at that level it is your responsibility to fix it. On the other hand, if I am running virtual machines on my hardware and I run into a communication problem because of a hardware level problem I need to know how the various virtual machines connect to each other as well as how the actual hardware connects together if I am going to successfully troubleshoot the problem. That means, if it is running on my hardware it is not running on some vague "cloud" but on specific hardware which is connected in a specific manner. What makes the cloud metaphor useful is that nobody in my organization knows how those machines connect and we don't really care.

      --
      The truth is that all men having power ought to be mistrusted. James Madison
  3. business forecast: cloudy by OffTheLip · · Score: 4, Funny

    Businesses don't want to miss the next big thing but like most decisions, time will tell. "I've looked at clouds from both sides now, From up and down, and still somehow It's cloud illusions i recall. I really don't know clouds at all"

    1. Re:business forecast: cloudy by dkleinsc · · Score: 4, Funny

      On the upside, it makes it now possible for a business to say "Hey (hey), you (you), get offa my cloud!"

      --
      I am officially gone from /. Long live http://www.soylentnews.com/
  4. Maybe not completely by Gripp · · Score: 3, Informative

    I work at one such company. We recently setup openstack and plan to eventually use it for our production environment. But ec2 will still stay in the picture. Both for services were the end user needs more direct access to the machine and for failover purposes. I just don't know that openstack means the end of ec2.

  5. Tightening reins on developers? by cryfreedomlove · · Score: 4, Insightful

    From this article: "like tightening the reins on developers who turned to the cloud without permission"

    Let me state this in other words: "Insecure IT guys are afraid for their own jobs if they can't lord it over developers". Seriously, developers working in an API driven cloud just don't need a classic IT organization around to manage servers for them. Cloud is a disruptive threat to classic IT orgs.

    1. Re:Tightening reins on developers? by h4rr4r · · Score: 2

      Let me state this in other words: "Developers know jack and shit about security and business requirements, they will now be able to not meet either of those even faster". Developers are afraid that if the cloud thing does not replace all classic IT they might still have to explain to someone in a meeting why their code falls over all the damn time and admit that maybe more hardware is not always the best answer.

      Cloud is what traditional IT orgs manage for you slick. You think it is just developers all the way down?

    2. Re:Tightening reins on developers? by Guido+von+Guido+II · · Score: 2

      Let me rerephrase that for you. "Developers don't care jack shit..." Show me a developer who is incapable of being a successful sys. admin and I will know you a terrible developer. It's all about time and interests.

      Absolutely. I know plenty of people who've been both good sysadmins and good developers at various points in their career.

      Like you say, developers aren't necessarily interested in the things that make for good administration, though. The ability to create virtual machines at will, for instance, means that developers can create more virtual machines than are needed, which results in greater administrative overhead and greater costs. They can also sidestep normal administrative procedures.

      We used to have enough problems with this back in the day when everything ran on physical machines. A developer would slip a box into the data center and not tell anybody or document anything. Later we'd have to clean up the mess when we discovered that nobody had been patching the machine and it had a security problem, or they hadn't configured something properly so that it didn't start up properly on reboot. My favorite was the time the developer (who was in my opinion very otherwise very good) forgot to configure the IP addresses of a machine with anything more than ifconfig. The machine came back up after its first reboot without IP addresses and a good chunk of the customer's site was down for an extended period of time.

      With virtual machines, this is so much easier.

    3. Re:Tightening reins on developers? by SimplyGeek · · Score: 3, Insightful

      That's not always the case. Look at workplaces that fall under HIPAA regulations. That last thing IT wants is for developers to start up their own app projects in the cloud, whereby their apps start accessing PHI/PII. The moment that PHI goes from the local intranet and those bits go onto a 3rd party cloud service, the company will shit a brick because the developer's just violated regulations. There's a reason IT and security requires oversite of app development.

  6. Different needs for different scales by slim · · Score: 5, Insightful

    How hard is it to understand that the cost/benefit depends on your size?

    Car analogy: If you're an individual who needs a car a couple of times a year, you rent one on those occasions. If you drive almost every day, you buy a car and you get it insured. If you're a small company, you give your travelling staff a car allowance. If you're a big company, you buy a company car scheme and insure all the cars under one policy. If you're a gigantic company, you self-insure all your staff's company cars.

    Draw a graph of the cost vs scale of a third-party cloud, versus your own datacentre. At some point the graphs will cross. That's where you switch.

  7. Regarding zombie servers by jmak · · Score: 2

    There are tools to deal with them, and were even recently featured on Slashdot: http://news.slashdot.org/story/13/01/07/1551231/netflix-open-sources-janitor-monkey-aws-cleanup-tool

  8. cowboys like you by onyxruby · · Score: 4, Interesting

    I've reined in cowboys like you for years, from one fortune 500 to another. Arrogant jackasses that can't be bothered with change management, best practices, version control, documentation, pesky things like policies, regulations and laws. Self righteous developers that can't see past their own nose too see how thier actions or inactions affect those around them.

    Every single time they think they are above these things and that they know better than the industry around them. They never realize why something that works in their special environment works perfectly fine where they have the rights of a God but has all kinds of mysterious errors in production where there they are brought back down to earth. They then chafe when their development environment is set up identical to production, yet it is amazing how quickly previous mysterious bugs that plagued production and caused incredible operational costs suddenly get fixed. They of course never have to clean up multi-million dollar messes, talk to regulatory agencies, sit down with lawyers to plan how to mitigate their mess or have a face to face with an angry Attorney General.

    I've only won this argument and helped companies save millions by reining in the cowboys like yourself a couple dozen times. Probably something to do.with cleaning up large multi-million dollar messes more than once.

    1. Re:cowboys like you by ebno-10db · · Score: 2

      I've only won this argument and helped companies save millions by reining in the cowboys like yourself a couple dozen times.

      Sounds like you should get paid pretty well for that. So instead of complaining, you should thank the OP and his ilk for helping to provide your paycheck. Next cops will complain about there being crooks. Some people don't understand where their bread is buttered.

  9. Ahhh, ha ha ha. last square on buzzword bingo! by swschrad · · Score: 2

    get off the cloud, build our own cloud. also known as bringing the server room back into your own hands.

    also known as BOFH never dies.

    --
    if this is supposed to be a new economy, how come they still want my old fashioned money?
  10. Random pricing by EmperorOfCanada · · Score: 3, Interesting

    One thing that has kept me away from Amazon's cloud is the unknowns with its pricing. I have visions of a DDOS either clearing out my bank account or using up my monthly budget in the first 2 days of the month. Plus if I mis-click on something I might get an awesome setup that cleans me out. I am not a large corporation so one good bill and I am out of business. But even larger companies don't like surprises. So regardless of the potential savings I am willing to spend more if the price is fixed in stone instead of chancing being wiped out. I like sleeping through the night.

    Plus as a human I really like being able to reach out and touch my machines, even if I have to fly 5 hours to do it. So the flexibility of the cloud sounds really cool where the pricing is not so flexible. It would be nice to spool up an instance of a machine that isn't going to do much most of the time that doesn't actually use up a whole machine. But then when one machine starts to get pounded to give it some more juice. Plus upgrading your hardware would be much more of a dream. You move your most demanding servers to your hottest hardware and slide the idle servers over to the older crap. Plus restores and redundancy are a dream.

    Then you still have the option to fully dedicate a machine in "realspace" to a demanding process. While VM does not have much overhead it does have some. So taking a server(s) that is being pushed to the maximum and sliding it onto bare metal will then allow your hardware to be used to maximum efficiency.

    Then by having no real cost overhead to having more near idle machines spool up your developers can play interesting games. Maybe they want to see what your software will do with 20 MongoDB servers running instead of the current 3; or 200.

    This all said, I am a fan of Linode; where I can predict my pricing very well.

    1. Re:Random pricing by bored · · Score: 2

      While VM does not have much overhead it does have some.

      This is the common view, but its a gross generalization. For some problems you won't be able to measure much of a difference. These problems tend to be problems that are low thread count (1-2 loaded threads) and have a high cache/TLB hit rate with few kernel interactions. On the other hand, applications pegging out more than 8 CPUs, or doing a lot of cacheline ping-ponging, etc tend to take a noticeable hit. Furthermore, applications that are doing a crapload of IO through adapters that are neither pass-through, nor have SR-IOV or similar high speed IO support and appropriate hypervisor support will be hit with orders of magnitude performance problems. Even with support, picking the wrong set of adapter vendor/hypervisor/OS can have pretty serious performance impacts.

      So as you said, put it on the bare metal if your having performance problems. But this is sort of my problem with many of the modern developers mindset. The lets just throw another VM at the problem attitude is so wrong its amazing. I think its founded on the fact that a huge number of people in the IT industry don't understand the difference between throughput and latency when it comes to performance problems. I see it all the time with the "we just need another VM" mindset. Half the time the problem is that the attached disk subsystem is doing 300IOPS cause its running on some internal RAID6 with 4 disks and each IO is doing a read/modify/write cycle. Plug in a SSD and reduce the physical server count from 10 machines to 1. Or for that matter move a table out of the database to a memory mapped file, and return the results directly from the mapped page rather than bouncing it through a socket, and then copying it a couple times through some interpreted scripting language before before the ethernet adapter finally DMA's it out. Its amazing when you show someone that the 1.5k piece of data they are returning was copied 30 times around in memory 20-50 times and that is what their CPU is spending all its time on.

  11. Legal issues as well by davidwr · · Score: 2

    It's usually "cleaner" if you either don't out-source sensitive data or if you out-source it in a way that is either 100% encrypted and you hold the only keys or if it's stored in an "identifiable" physical place ("it's on THAT set of hard drives, and it's being processed on THAT set of CPUs" etc.) that isn't shared with other users.

    --
    Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.