Slashdot Mirror
Ask Slashdot: Do Any Development Shops Build-Test-Deploy On A Cloud Service?
bellwould (11363) writes "Our CTO has asked us to move our entire dev/test platform off of shared, off-site, hardware onto Amazon, Savvis or the like. Because we don't know enough about this, we're nervous about the costs like CPU: Jenkins tasks checks-out 1M lines of source, then builds, tests and test-deploys 23 product modules 24/7; as well, several Glassfish and Tomcat instances run integration and UI tests 24/7. Disk: large databases instances packed with test and simulation data. Of course, it's all backed up too. So before we start an in-depth review of what's available, what experiences are dev shops having doing stuff like this in the cloud?"
Atlassian is already trying to push their customers in this direction. Their Bamboo OnDemand offering spins up AWS instance as needed for builds. In this case, you could still host a local Bamboo instance and use elastic remote agents.
One thing I do like about this sort of setup is that it keeps you honest about deployment. Your build environment stands up a new instance every time your remote agent goes stale and is recovered to reduce costs.
We're not at your scale, but we do everything with AWS and have found that it works well.
One thing you might want to do is reexamine your mentality around 24/7...you need to evaluate what really does need to run 24/7 and what needs to be available 24/7 (i.e. something that can tolerate the time it takes to spin up from an AMI).
For example, your Jenkins server could be configured with a master/slave arrangement that allows the main Jenkins server to be a small or medium instance that runs 24/7 and then when a build needs to happen, spin up a beefier slave to rip through it as fast as possible and then shutdown when done. Each build then has a fixed cost, regardless of whether it runs serially or in parallel.
Our main reasons for choosing to use the cloud were:
- We have remote workers, both permanent and a WFH policy...cloud makes it not matter where you're working from.
- Less maintenance...stuff mostly just works and most things are scripted rather than configured.
- We like the mentality of thinking of computing as a resource, not a collection of discreet machines. Running 5 builds in parallel is expensive when you think machines but costs the same as 5 serial builds when you follow the spin up, build, spin down philosophy.
Amazon has a detailed AWS cost estimator:
http://calculator.s3.amazonaws...
When we migrated to the cloud, our actual costs were within 15% of the estimated costs.
But really, the easiest thing to do is just build a test environment and try it -- you only pay for the time you use.
When we migrated to AWS we knocked 70% off our colocation bill (we had more space at the coloc than we needed, but it's hard to move production hardware to a smaller space without downtime, plus we had significant savings in equipment leases and maintenance contract costs).
Our dev/test hardware was aging and becoming unreliable (and no longer matched production since we moved to AWS), so we moved that up to AWS as well, but even after that migration our total AWS bill less than half what we paid at the colocation center. We only run the dev/test hardware during business hours, or on-demand as needed -- we set up a simple web interface that lets developers spin up test instances as needed. AWS keeps dropping prices, so we're even as we've grown, our costs have remained relatively constant.
AWS has some of the lowest cloud prices I've found anywhere. You can get AWS instances for under $3/month reserved according to what you need. 'Small' Linux instances cost about $15/month reserved last I checked. In fact they'll even give you a Micro instance free for a year without spending anything as part of their 'free tier'.
How did you come to the conclusion AWS was expensive?