The Cult of DevOps

packetrat writes "I was at OmniTI's Surge conference today, which turned out to be, among other things, a meeting of the cult of DevOps. Ars Technica covered the keynote and some of the presentations, but some of the best stuff is in the comments. Google CIO Ben Fried told the tale of a really poorly engineered trading application at Morgan Stanley that he was associated with, and how the way IT was structured there contributed to that engineering and to its spectacular failure, costing the bank untold millions in stock trade processing fees from its institutional customers. He said what he learned from cleaning up the mess has informed how Google runs its IT operations, and a culture that promotes generalist skills. A lot of how he describes Google's approach sounds like the DevOps kool-aid a lot of the other speakers were serving, but it also sounds like common sense — are most IT organizations really that poorly run that developers are totally unaware their software is sending messages that are generating network storms, or network engineers are clueless enough about QoS to route leased lines into their data center through their public-facing Internet?"

Comments

Cult of DevOps?

[merlinokos]

I'm not sure I can take anybody who calls an attempt to make IT and Development more aware of each other a cult, seriously.
The traditional way of doing things didn't work for 30 years. Why is it that when people are trying to make (and apparently making) a difference to how companies work, they're regularly denigrated by a large subset of the very people whose working lives they're trying to improve.
Haters gonna hate, I guess.

Re:Cult of DevOps?

[wisty]

I'm guessing the main haters are sysadmins, who see threats to their importance and way of working.

It's no longer desirable to have a gradually grown system. Everything should be built on virtual machines from scripts. You no longer need a wise old guy with a beard to change anything - who cares if your modification might trash the system? You can spin up a test instance, and dump it if it doesn't work right. If you have two apps that don't play nice with each-other, just put them on difference machines. The philosophy of a DevOps guy is very different to that of a sysadmin - you don't do brain surgery on a horse while it's in the middle of a race. Just shoot the damn thing, and send in a healthy clone. (Or would you prefer a car analogy?)

There was a world of difference between "works on the dev box", and "works in the production environment, without screwing up anything else". That gap is getting bridged by DevOps practices. OK, you still need people with sysadmin skills, but the way they work (and the things they have to learn) is becoming a lot more like development than sysadmin work, and sysadmins are not bred to like change.

Re:Cult of DevOps?

[gmack]

And how do you know the new VM won't have the same problem? If you never know what went wrong and fix the actual problem you will just end up restarting VMs constantly.

The curse of 'enterprise'

[Junta]

I've been around enough to know that this 'new' 'DevOps' philosophy is just the way it always has been done at many successful companies making extensive use of technology.

I have come to associate the phrase 'enterprise IT' with those who *don't* work that way, and make their lives needlessly complicated. Of course, every last party to the mess will generally recognize it and know what could hypothetically be done about it, but only bitch in private about it and rarely ever push for meaningful change. The reason is simple, so long as it is a complicated mess, it requires a great deal of human care and feeding, meaning job security. Management can't force things to change without huge risk as everyone has sufficiently entrenched themselves.

Re:Recently

[Junta]

Also known as the first code then think approach

No, it's think *as* you code. You do think some before starting to code, a vague rough picture of the pieces, but you don't invest a lot of time in that 'pure' design phase because the more detailed you plan without proving it out, the more you *should* throw away as you start implementation and realize how ill-conceived your design was or that maybe your design was adequate, but a better way makes itself apparent when actually implementing. Generally when I see a development team incapable of operating at all in this manner and fail to achieve any measure of 'complete', they only 'complete' under other strategies of development on a technicality and produce low quality product. Some management people think that methodology can be used to make piss-poor (cheap) developers serviceable and avoid having to reward/retain good talent.

A lot of people wrote books on patterns (design and otherwise), but in the end if no one follows these patterns the problems remain.

I think there are two aspects to this. One is that most development an organization is predestined to operate in a specifc way depending on who comprises it, regardless of what name they pick to describe it. I know organizations that did 'waterfall' and 'changed' to 'Agile', but really just renamed things they did, acted the same, and called it 'Agile'. However, I don't know if this is a bad thing. I think getting too hung up on a specific 'pattern' others preach in conferences is bad, and organically feeling your way for a process that works for your team is better. Awareness is good, but getting locked in is bad.

However, I have found in the sea of Agile and Waterfall and all sorts of buzzwords a methodology that really resonates with me:
http://programming-motherfucker.com/

Re:It is always IT's fault

[bobaferret]

I think this is the whole point of the devops stuff. It's not IT's fault, it's not the developer's fault, nor is it QA's fault. The larger your organization, the larger communications gap between these 3 areas. The devops folks are there to coordinate and communicate between the three. They may review/write a little code, they may consult on infrastructure, they may make sure things get tested appropriately and meet the customers expectations. For small shops/teams this fairly straight forward. Big shops have to have someone who's job is to know everything about the project. There is a similar position in commercial building. It's the "site manager" they make sure that the contractor is building what the architect asked for. From the number of bathroom stalls, to the depth of the foundation. Which at the same time working with the architect to fix problems that arise from misplaced city sewers lines, an unavailability italian terrazzo tiles in rural Illinois. All the while making sure that the customer is getting what they paid for and understanding and approving changes. Sorry I didn't have a good car analogy.....

WTF are you talking about? Devops is nothing new

[Colin Smith]

Historically a sysadmin has been able to and does write, fix software, and administering large scale systems has always meant templating and then deploying from updated templates... which means having a build system which can build to a template usually automatically (it's boring) which apparently is now called continuous integration. You also need an infrastructure designed to easily deploy changes. VM or physical is almost totally irrelevant, VMs make life a little easier.

Lets see... Infrastructures.org for example was definitely there mid nineties (and still is, cool.). That's almost a whole generation ago, how old are you?

I have no idea what makes people think this devops stuff is new. Is it just the fact that there are now thousands of newbies trying to make names rediscovering good practices? Is it just that we now have more large scale systems, or is it that we now tend to have highly distributed vs centralised systems?

And it's not development. It's engineering. The difference is maths.

Go on, get off my lawn!