Slashdot Mirror
Who Owns Deployments - Dev or IT?
txpenguin asks: "I am IT manager for a small software company. We host several generations of our applications in a fairly complex environment. Our systems are very much inter-dependent (clustering, replication, heavily loaded, and so forth), and bad changes tend to have a domino effect. Additionally, it seems that there are always those who need to be 'in the loop', but aren't aware of changes which affect them. There is a constant battle between IT and Development regarding who should handle the deployment of new code releases and database schema changes to production systems. Dev doesn't understand the systems, and IT does not know the code well. How do you handle this at your company? What protocols seem to work best? Can there be a middle ground?"
These kinds of things where there are two opposing sides always have the same answer. Unless one side is teh debil or something.
You have to compromise. That's it. Middle ground. There are no other solutions to or ways around this problem. As you describe it, each side has access to and knowledge of half the problem. Half plus half is whole!
So, meet with the guys in Dev. If you want to be beaureaucratic and official about it, create a "deployment team" consisting of an equal number of members from each side that will sit down, discuss, and supervise all necessary changes to production systems. Hell, send someone to a project management class if you need to.
Now, the obstacle you're likely to hit is office politics. People won't want to listen to others and/or won't want to give up their turf or allow others on it. Too bad. To place how serious this issue is in overcoming the political terms: everyone in both departments needs to be cooperating or unemployed.
So there you go. Just like any other relationship, business or otherwise: sit down and talk it over. Problems solved!
ACs are modded -6. I don't read you, I don't mod you, I don't see you. Don't like it? Don't be a coward.
IT should own the deployments.
Assuming the dev department does their job well, a deployment would not require any of the dev department's skills.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
In art classes they teach students to draw the space around the objects they are trying to depict. It's a useful skill in many areas.
Rather than imagining that there is this atomic transition point that one side or the other must own, look more closely at what happens when changes are put into production, zooming in until you have enough detail that every piece naturally belongs to one team or the other.
Then look at how this would play out in the real world, to find the "frothy" or "tangled" parts (well, IT should do this, then Dev should do that, then IT should do two more things, then it's Dev's turn again). These parts should be sorted out by requiring documentation (or scripting) to flow one way or the other, so that the process can be performed by one group without the direct involvement of the other.
In short, the problem here is the granularity of your question.
--MarkusQ
Plenty of other posters have pointed out that you sound a like an operation that is a bit small for the full Software Development Process. However if you're asking I suspect you're a growing company, in which case you need to get a Process in place, and soon or you will experience the full agony of a chaotic IT environment. (NB That's where I work now - I've worked sane places too) Fairly typical Process: 1. Dev receive Requirements and Defects from the Business, and code to them, unit testing their code (). 2. Code is delivered to Operations with a 'Release Note' or equivalent covering how to deploy the code to Environments 3. Operations deliver (deploy) the code to test environment(s). Link and Acceptance testing is performed - does it meet Requirements? are key defects resolved? Plus regression testing - does it break the existing system? Test sign it off if it clears these tests. 4. Operations deploy the code to the Production system on sign off. You inevitably end up with tensions in the Business vs IT, plus the divisions between the priorities of Dev, Test and Operations. Sounds like you are at the stage of not having any well-defined roles/teams for these responsibilities. I'll detail the Operations breakdown too. Operations: As others pointed out this breaks down into various teams. DBAs, Sysadmins, Change Management, Release Management, Operators, depending on your site. Operations are responsible for the stability and smooth running of the Production system - they must accept change, but control it. Since I work there, and you specifically address the subject, I'll detail Release Management too Release/Change Management Usually end up the Gatekeepers on changes. They'll need to be familiar with the whole system, and resistant to the pressure they'll receive from all sides. They need to know what versions of code are where, and be able to reject bad code when it turns up, but be flexible enough to make sure Test have something to test. They need to be experts on everything your IT does. No jobsworths here, and good generalists are rare. Since you'll inevitably go through a period of chaos if you are growing I'll mention that staff turnover here is very high - unless you get in contractors, and pay highly for them. The Change Management role, sometimes covering the Release role too, is to track changes and know who they impact, and be able to prioritise changes. If Release and Change roles are separate, CM is closest to the business. Hope that helps.
In my company QA is the bridge between development and production. I'm a team lead (dev) in a company which has a suite of web applications. Each application has a lead assigned to it, who handles the development and documentation of a product through their team. We do several deployments of software each week, and if our leads had to hand-hold through each of them we'd be hamstrung for time and working more night hours than we'd like.
When we have a RC I'll branch the trunk, and request that QA perform a Pre-Production build. Developers will work with ops to get this running properly on the pre hardware, as this can be done outside of maint hours. We'll then do several builds of the branch until it's gold, and then tag off the branch as X.vv.zz.
While a major release is in QA the lead focuses on creating/updating the operations document which addresses day-to-day maintainence issues and tells operations how to troubleshoot the app in the case of a problem. They also produce an implementation plan which identifies the groups/persons needed to deploy the application, and the steps needed to be taken, using what they've learned from the initial pre deployment. Once this is done, and QA has promoted the app, a dry-run is performed to try to catch any missing steps. The implementation plan is handed to QA, who coordinates with IT/Ops to resolve any conflicts and schedule the deployment. Ops/DBA's then physically performs the deployment following the steps given in the plan. In a major release situation, you may have a team lead or platform manager coordinating the actual steps on a conference bridge. But for minor releases we've been able to just have our operations teams do the full deployment with verification by QA and the product's customer service group.
We also have a twice weekly meeting where any upcoming production changes are discussed between IT/Ops, QA and Dev. Release documents are put on a calendar, so if an issue comes up on another product we can go to this and see what may have caused it.
Dev and QA also meet weekly to discuss the progression of products through or into QA. Any issues with testing or problems with builds not being stable can be addressed.
It took us a while to get to this point. We had previously been in a situation where dev would handle the build and deployment process, and it was had for many of the leads to let their projects go, but now we can see the benefit, not only for the company, but also in the fact we don't have to be doing releases at 12AM on tuesdays anymore. It takes a lot of work across departments, and definitely is a long road, but something that needs to be done.