Slashdot Mirror
Agile Methods in System Administration?
sta asks: "Agile methodologies focusing on development techniques and approaches abound, many of which at the very least give food for thought. I have been in constant discussion with our dev manager about agile approaches and their extension to the systems administration world. Can agile methods and the agile mind set be applied to the systems administration world? Does systems administration rely on policy and procedure for it's integrity and reliability or is that just an ingrained habit? There appear to be a number of administrative tools that claim agility but are there any established agile methodologies?"
as someone who has developed software under a couple of different processes, i actually find this a compelling question...
if it came to a choice or not, i'd not be very inclined to following any porcess. my favourite programming tools are a clipboard and a mechanical pencil -- and frankly after i've drawn up a UML i end up wanting to be an Agile programmer almost by default. it depends on how you're defining Agile.
for that matter, IBM claims that its Rational Unified Process is inherently flexible, but having had to program in that process a few times, i'll argue that it's not.
let's argue that one of the key tenets of the Agile Method is to encourage the talented and motivated to do a good job and lead through their individualism. in a small or even medium sized business, that is a good way to run things (and this is true of fields other than IT). it encourages random improvements and magic features that might not have been planned in the roadmap since version 0.1.1. and if you are running a particularly small group, it is brilliant way to do things.
by comparison, if i was managing the development of a prodcut like M$ Word, i would live in dread of a rogue coder doing something brilliant but failing to properly check that every object and variable behaved like it was supposed to. i'm reminded of a blog by a microsoft programmer with a bug that caused havoc with the mac version
bringing it back to the sysadmin question: i thik there is a lot of potentilal in using Agile Methods. flexible and fast are what the cutsomers want. but i think there needs to be a further defining of what the "official" Agile Method is because i can certainly see problems with it.
as someone who sysadmins some very strange situations, i can imagine horrible problems if one of my junior admins decided to go all rogue elephant on a DNS server using some OS that is incompatible in a weird and unknown way
i love and encourage people to find their magic solutions to scertain questions, but yes i must admit that if you do something that means a batch of our users can no longer find the network then i'm a little worried.
to my future junior sysadmins: yes we can already run the directory and network with any damn OS you choose, they all fit. magic improvements are highly welcomed and i look forward to them. but please, follow the damn chart, there are a couple things that are in place and already work.
No, not really. Did you read my disclaimers?
I did mean that starting from scratch probably isn't doable. I didn't bother looking for something that already existed since, well, frankly I didn't care. Advice gathered from Slashdot is worth every penny.
It still doesn't look to me like Nagios solves everything off-the-shelf that I've encountered in my life, but that sure does go a long way towards it.
So I'll revise my statement to say you're insane not to use something like Nagios, and write whatever plugins you need if they don't exist already, which they probably won't for custom in-house apps.
To me the word "agile" in programming has come to mean "a renaissance of best practices". In other words, an excellent programmer is already agile, but a mearly"good" programmer might benefit from having it spelled out.
/etc configs (if you're using dispatch-conf + RCS on your gentoo box, you're doing this already .. just start checking YOUR changes in the RCS files too.. this saves my butt at least once a week)
.. does your machine need a certain config? Write a script that verifies that the config is present. Run it, it will fail. Fix the config, run it again, watch it succeed. Now put the script in cron and forget about it (you might want to build a framework for this if you have a lot of these little scripts puttering around). This too has saved my butt: I have scripts that make sure ports are CLOSED on certain machines.. when the firewall config was accidentally erased I detected it right away.
.. your scripts will be shorter and more robust. Use programs like djb's daemontools to run daemons, instead of this bizarre thing where they put themselves in the background and you need all this scaffolding to track them down to start/stop/watchdog (wtf is up with that anyway).
.. use publicfile instead of Apache .. etc.. just do the simplest thing that can work.
But by giving it this new name, there's a risk of backlash. I guess that's happened already, judging by the utterly vapid responses to this post!
To me, agile means the following things:
* don't do work until you need to
* do everything in the simplest way you can
* write tests for everything
* make your work relocatable and reusable
* use version control
* communicate with customers
* release your work to the customer as rapidly as practical
Now, I happen to wear both a sysadmin and a programmer hat depending on the time of day. Mostly sysadmin, but since discovering this philosophy of programming, I've started doing more coding. It's a pretty amazing feeling to have 100% code test coverage with everything tucked away in version control, with single-command deployment and distribution to client machines. They never taught me this stuff in school.
So, if you're a syadmin, and you're not as cynical as the other slashdotters here today, can you see anything in that list that might help you?
Here's some things I do as a sysadmin:
* use version control for
* practice "test-first sysadmin"
* do everything in as simple a way as possible: for your next script, use environment variables and the presence/absence of files in a directory to configure it, instead of inventing Yet Another Config File format
Use the filesystem instead of MySQL
Don't ever install things by hand. Build a custom package. If your OS doesn't let you do this easily, switch to one that does (gentoo rocks for this as well).
* document everything: when you do something, make a note of it on a wiki or a text file. Give the other sysadmins access and chide them when they don't keep the file up to date. lead by example.
* automate: don't do anything 3 times. anything you do a third time, automate it. (you will appreciate the simplicity of using the filesystem as your config file when you do this, by the way). need to watch something on a box? create an rss feed. you don't need a web server, just build the rss feed and scp it from your laptop every hour. do whatever it takes for you to work as little as possible yet meet your goals.
Yeah, you probably do a lot of this already, but you'd never think of calling it "agile" or anything else other than "a good job". Fine, whatever. Agile is more of a mindset than anything else, based around simplicity and being ready for change.