Evaluating the Performance of an IT Department?

Daniele Pagano asks: "I have just been promoted from code monkey to manager of the IT department of a construction subcontractor. As far as this industry goes, we are relatively high-tech (and getting better), but like many IT departments we are often considered a necessary money 'black hole' (i.e. they just notice the outages). So one question that arises is: how do we actually value our work? That is, how much money are we actually saving the company, and how do we demonstrate it to them? How do we value the contributions of each IT staff member (say, for a bonus or raise) in an objective, quantifiable manner? I know there is no one correct answer, and I have many ideas, but I thought we could pull our thoughts together for the betterment of small IT departments everywhere."

Comments

Suggestion, if you have the time

[mswope]

Tom Demarco, et. al, covered this in, "Peopleware"

You need to figure out what benefits you bring to your company vs. what costs
your company would bear w/o you.

If you have the time, read "Peopleware." (it's not a very long read) If not, figure out what it would cost to outsource you. Keep in mind that a lot of outsourced support ends up under "capital or recurring expenditures" rather than "personnel costs."

In our industry, it seems that on the average of 6-8 years, some bean-counter in the company says, "we're an XYZ-company, not a communications/high-tech/software/ technology company." Then, cut-backs start, out-sourcing starts, costs soar and after a very painful 4-6 years, they start hiring people back to run the soft underside of the company.

If you do it right...

[clambake]

...noone will be quite sure you've done anything at all. Good IT, I mean really good IT, will be a whole lot of nothing. You can value an IT department by just how little they have to do, because if they have done it right, there will very very little, beyond patching now and then, for them to actually do durning the day. Of course this isn't true when you upgrade systems, but in general, you should measure them not by how much they did, but by how much they didn't do.

So, instead of saying "Joe fixed three server meltdowns this week, good job, here is a raise!" go with "Joe's machines haven't needed maintenence for three years, here is your raise!"

Re:If you do it right...

[puppyfox]

This is definitevely good as far as IT goes, but as a construction company only a fraction of our business is in the office (450 field people, 50 office, 3 IT), the rest is guys digging trenches and pouring concrete. How can you relate money saved on graders or hours of guys driving in a truck to good IT (both system and software development, which we outsource the development of, but I architect myself with our management team)?

For example, one of our great successes last year was not getting better servers and dramatically increase uptime and all kinds of good IT things, but was spending a couple of days writing a small Access app to import budgets from one system to another via ODBC so we can tell if we are losing money on the field or not. That's what really matters in the end! But how do you quantify such things?

Re:If you do it right...

[clambake]

This is definitevely good as far as IT goes, but as a construction company only a fraction of our business is in the office (450 field people, 50 office, 3 IT), the rest is guys digging trenches and pouring concrete. How can you relate money saved on graders or hours of guys driving in a truck to good IT (both system and software development, which we outsource the development of, but I architect myself with our management team)?

For example, one of our great successes last year was not getting better servers and dramatically increase uptime and all kinds of good IT things, but was spending a couple of days writing a small Access app to import budgets from one system to another via ODBC so we can tell if we are losing money on the field or not. That's what really matters in the end! But how do you quantify such things?

Well, one thing ot remember is that you should only quantify the ends, not the means. Meaning, you should measure the quality of the final product, and not spend too much time attpting to measure the ingredients that went into that. Of course, if the final product is crap, then you will want to start measuring on finer and finer grained levels, but if you are being successful, then be VERY careful how deeply you look into your process.

Look too deep and you will feel the natural inclination to "fiddle" with things, only to find that it's a game of Jenga, and every thing you touch has the potential of bringing your entire development process down around your head in a way that you won't be able to recover from.

Not all "inefficiency" is as bad as some management classes/books will lead you to believe. In fact, more often than not, the percieved inefficiency is actually a vital component that you just can't see as such because you failed to notice some subtle element of the complete system.

Read studies on the incredible problems, for example, that plauged the park services in the early 1900's as a good analogy of what happens when you fiddle with carefully balanced systems. Maybe people wanted to protect the elk from becoming extinct in the park, so they killed most of the wolves that hunted them... so the elk population exploded, ate the forest clean and ended up extincting themselves, but not before extincting the beavers who also needed the same shrubs that they ate, which caused the salmon to die, which caused the river-grasses to overgrow, which caused... etc, on and on. Each step of the way the rangers would tweak one bit, then another then another, all trying to bring everything back into balance, and eventually the whole thing would collapse around them, taking decades to recover naturally.

Make sure to outsource

[toddbu]

Nothing damages the credibility of an IT department more than insisting that everything be done through them. We're an ASP, and we often get calls from users who are doing an end-run around their IT departments so that they don't have to wait forever to get the job done. If an IT department takes some work and outsources other work based on experience and cost, they'll be respected and even sought out for their knowledge. If they get in the way and do everything they can to protect their jobs, then they become the "black hole" that you refer to.

If you want my advice (and why wouldn't you? ;-), I'd do a poll of other departments to see what they think of IT. Talk to everyone from the office grunt to the VP of department X, and take the temperature of your department. Don't be afraid to get beat up, and then go back and do some soul searching on the stuff that people don't like about you. Then go back out to these same folks and let them know what you're doing to solve their problems, and be sure to point out that you know your mission is to help them get their jobs done.

Re:Make sure to outsource

[blincoln]

We're an ASP, and we often get calls from users who are doing an end-run around their IT departments so that they don't have to wait forever to get the job done... If they get in the way and do everything they can to protect their jobs, then they become the "black hole" that you refer to.

I don't intend you any personal offence, but this kind of mentality is one of the main causes of headaches for me - someone who works in the IT department at a fairly large corporation.

Obviously there are some exceptions, but if we make someone "wait forever," it's usually because it will take "forever" to do it in a way that is supportable in the long term. As a contractor, you don't have to worry about whether your solution will still be working in a year.

There are many, many times that people in other parts of the company have gone off on their own and hired contractors to do something because they wanted it quickly. Usually what they've gotten is a rush-job that is barely stable to begin with, let alone a year or three in the future. Excel macro "applications" that can only run on Office 2000. Thrown-together server apps that can't handle anything other than the JRE they were originally written for, let alone an OS upgrade like Windows 2000 -> 2003. Web apps that have strict dependencies on outdated and unsupported versions of multiple products - each from a different company.

These things go on to become critical parts of the business, with years of data stored in them. Then they break. Who do the users blame? IT. Because they don't understand why we can't make their undocumented, nonstandard, proprietary software work on a modern OS.

We do use contractors for a lot of things - when they're hired by IT, and work with IT employees, they can be great. But non-IT workers hiring contractors on their own isn't necessarily a sign of a problem in IT. Sometimes it's a sign of a lack of understanding of IT, and why sometimes it takes a little longer to do things the right way.

Re:Make sure to outsource

[toddbu]

I don't intend you any personal offence

Just as long as you don't tell me that my mother wears Army boots. :-)

As a contractor, you don't have to worry about whether your solution will still be working in a year.

As an ASP that charges a monthly fee, if we screw up then our business goes away. That's the "service" part of the term ASP. We tell our customers that they should hold us accountable not only for getting them up and running but for making sure stuff works on a continuous basis. We listen to our customers and implement new features based on their requests because it helps us grow our business and gives them less reason to go to another solution.

These things go on to become critical parts of the business, with years of data stored in them. Then they break.

We have software in place that allows our clients to import and export their data. Any good ASP should have a system like that in place. We'll manage your data for you, but at the end of the day it's their data and they should be able to take it elsewhere. Anyone who hires an ASP who doesn't have this policy should find a new vendor.

Having worked in a large IT department in the past, I can tell you that it's not uncommon for them to horde data as a means of protecting their jobs. I once worked on a project where we offloaded the data from our IBM mainframe onto a PC and manipulated it there, and we got way better results. The only one who stood to lose was the guy in the IT department who was trying to rewrite Excel on the IBM system.

Sadly, many IT departments have earned themselves a reputation of caring more about their own skin than that of their fellow employees. If you don't see them as your customer then you're doomed to failure because few IT departments are revenue generators. For what it's worth, I don't disagree with your point about managing data or wanting to keep the system alive. As a responsible engineer you should definitely point out the pitfalls. But these are your customers, and you ulitmately have to do things the way that they want. I guarantee that if you follow this path that everyone will be happier.

One way to measure...

[Chabil Ha']

Just pull the plug on IT and see how much the business looses. Maybe then they'll appreciate you more.

By Comparison

[Planesdragon]

So one question that arises is: how do we actually value our work?

The way you value your work is by expressing how much cost you have kept the company from otherwise spending. If you had unlimited time to evaluate, you could go out and see how many dollars the company would have to spend to have someone come in and support every project you support, with the same level of response that you have.

Demostrating it is less easy, especially if there aren't any renovations you can do to immediately save your company money (that is, the job has been done right already.)

How do we value the contributions of each IT staff member (say, for a bonus or raise) in an objective, quantifiable manner?

Decide on a perfomance metric that you can objectively measure for each task. A web developer should be rated differently than a tech support person, for example. Metrics should preferrably be such that a tech trying to game the numbers will be working more like your ideal employee (i.e., instead of looking at "time to close ticket", look at "time to first response" and "satisfaction of non-IT")

Get to know, then educate the dominant coalition

[1369IC]

I see parallels between the IT field and my own (communications, as in PR, not telcom) because, while you can come up with certain measurements, they mean very little in the end. This is because most of the people who matter don't understand what you do anyway. What they know is leadership, management and teamwork, and IT and communication departments many times really are black holes in these areas. I've been a couple of places where the only reasons people put up with IT and PR is that they don't understand computers and they're afraid of the media.

On the PR side we have a concept called the dominant coalition. I just means the group of people who dominate the organization (it's usually more than just the boss, and rarely mirrors the organization chart exactly). Do a little strategic communications research on them: figure out what they like, how they talk, what they value and how they do things. Then get on board as much as your field and personal ethics will allow. You don't have to go whole hog sycophant, either. Just find out what's important to them and do the things you feel comfortable doing -- learn their staff work, wear "management" clothes, take a couple business courses to learn the lingo. Just remember that sometimes the leader has to take one (or several) for the team. You might have to buy a couple sport coats to wear to meetings so you can better represent your guys while you're there. And it all counts in the end -- how you work, your personal habits, etc. Yeah, it's not fair, but it's fact.

You're not trying to build up brownie points, either. You're trying to do two things, really: show them you're part of the team, and build up enough credibility so they're open to your education and ideas. If you really, really want to resist becoming a "suit," or whatever you want to call it, you might consider telling them you'd rather go back to being a code monkey. You've taken on a responsibility, and most of earning your money now comes down to two things: taking care of the boss and taking care of your guys. There's nothing wrong with wanting to stay on the technical side. There's only something wrong with taking the management money, sitting in the management seat and keeping your head on the technical side. You're screwing your boss and your guys.

Once you build up some credibility, you can educate them slowly and carefully about what you do and what you can do. For most people, getting down to the nitty gritty with IT guys is like listening to Geordi LaForge. It's all tachyons and sub-space transmissions. And most IT guys, I must say, seem to cultivate that. As I made my personal journey from a Mac user who was happy to fiddle with themes to a Slackware user who built his own boxes, I discovered what bullshitters many IT guys were. Not that they were evil or lazy, just that they had a perfect cover for continuing to do what they preferred to do, or not taking on something they didn't want to, or for getting money they didn't really need. And while they got away with it quite a bit, people knew. They could rarely pin the IT guys down enough to enforce their will, but in the backs of their minds they knew. That also explains a lot of seemingly capricious decisions: the management just doesn't trust you, and can't base denial on a deep understanding of what you do, so it denies things on whim or intuition, which is sometimes quite wrong. But bullshitters bring it on themselves.

Anyway, as you educate, get your guys on board with the rest of the team. Go where they're going, and don't just be the slave standing in the back with the emergency oar repair kit. Row. Figure out a way. If you take the previous poster's advice and start to get noticed for the fact things run so well you're not terribly busy, manage by walking around. Stop by some drone's desk and ask how you could help him more.

And lastly, remember you're in a service industry. Don't let the hardware fool you. You're the guys who keep the computers and the Internet working. T