Ask Slashdot: Good Metrics For a Small IT Team?

First time accepted submitter shibbyj writes "I'm a member of a small 3 person IT team for a medium sized business (approximately 300-350 employees) that has multiple locations internationally. I have been tasked with logging our performance using the statistics from our ticket management system. I've also been tasked with comparing these stats and determining if we are performing above or below what is considered optimal. I'm wondering what people opinions are on what good metrics should be in regards to mttr mtbf etc. I have had trouble finding information on this."

Comments

Hahaha

[mewsenews]

One of you is getting fired

Re:Hahaha

[ITConsultant]

As a professional IT consultant, I would concur with the parent statement.

Also, the title is a bit redundant: all IT teams are small.

Re:Hahaha

[Anonymous Coward]

Yes, this.

Management wants to eliminate someone and wants to do so in an "objective" way to hide the fact that they're firing someone while probably giving the CEO a fat Christmas bonus. You're tasked with figuring out which of the three of you gets fired and how you can cloak this in enough "objectivity" that no one can object to it. Your best bet is to make this shit up. Figure out who the weakest link besides yourself is, or who you like the least, and generate a system of metrics that's biased towards eliminating that person. Use lots of acronyms and jargon. Also, make sure no one at work reads Slashdot.

totally irrelevant CAPTCHA: forgive

Re:Hahaha

[Jeremiah Cornelius]

The very core of your writing while sounding reasonable initially, did not sit perfectly with me after some time. Somewhere throughout the paragraphs you actually were able to make me a believer but just for a very short while. I however have got a problem with your leaps in logic and one would do nicely to fill in all those gaps. If you can accomplish that, I will surely end up being fascinated.

Re:Hahaha

[JoeMerchant]

One of you is getting fired

On a three person team, metrics are irrelevant - personality and politics are 100% more important than technical skill.

If you've got 15-20 people and it's time for a 10% downsizing, or (less realistically), performance based bonus or advancement, then metrics start to be something worth looking at.

When I had to choose between 2, my boss asked me who was my choice, I told him, he agreed, I started to tell him why, he stopped me and said: "it doesn't matter why, I'm just glad that we came up with the same answer."

Re:Hahaha

[Anonymous Coward]

Yet another AC here.

If there were any justice in the world firings would start at the top. Fuck management and their big salaries. What the fuck are they doing that they have to be paid so much? What value do they bring to a company? They're too happy to fire you to make more money for themselves (the fact that most people are one paycheck away from financial ruin doesn't seem to bother them) but do they ever get fired even if they run the company into the ground? Fuck no. I'd love nothing more than to see some of the bosses I've had in the past be out there doing actual work.

Is this better?

Re:Hahaha

[wmelnick]

An obvious troll. In order to become management you have had to prove yourself over the course of many years. While you may think those above you are useless, they would not be there had they not done what you did first and for probably longer than you have.

Re:Hahaha

[slippyblade]

Do you actually believe this? Seriously? Most upper management is there because they either knew someone in the right position or had enough money/clout to force their way into it. Often time through family. Rarely these days do you see any upper corporate management that actually worked their way up.

Re:Hahaha

[ScuzzMonkey]

As I was idly paging through the comments thinking about how many of them were jumping to outlandish, unsubstantiated conclusions about why the poor submitter was being asked to come up with metrics, I also realized that pretty much nobody (in the finest tradition of Slashdot) has bothered to answer the actual question: "I'm wondering what people opinions are on what good metrics should be in regards to mttr mtbf etc." I think I misunderstood this at first as well... in light of what he does say about the goals ("...determining if we are performing above or below what is considered optimal") I don't think he's asking what metrics to measure or whether or not we think doing so is any good, but instead what reliable industry benchmarks are for those metrics, so he can tell his boss what is "optimal" or not about their operation.

Well, shibby, sorry, but I don't think there are such things, or at least none relevant enough to take back to your management team. The only way to get anything meaningful out of metrics (obviously a lot of other posters are arguing you'll get nothing meaningful out of them; I disagree, but it may not matter either way if those are your orders) is to establish baselines for your organization and track future performance against those. It will take a while and it will have to be viewed in context to be worthwhile... important points to make when you are presenting your findings to management.

I'm being optimistic and assuming genuine business goals and a desire to understand IT operations on the part of non-technical managers are the point of this request, not some haphazard effort to chop down a three-person department, but it is also worth passing along some of the critiques that are being posted here. On the other hand, if you don't already, you should understand that not all managers are buggers, and that many of the better ones have legitimate reasons for trying to understand what is going on in their IT department. We often forget how mysterious what we do looks to the un-initiated, and I have seen enough poorly run IT departments to sympathize with non-technical managers who are grasping for the tools to understand theirs. The point being, getting defensive and obstructive in the face of these requests isn't always the brightest idea; instead, you can look at it as an opportunity to present some of your perspectives and difficulties to managers who are finally prepared to hear about them (after all, they did ask!). They may not have the time or horsepower to learn everything you do in depth, but it is possible to analyze operations based on the right sort of shorthand.

The cynics may be right; only you will know. But I've seen companies run down by their IT departments as often as I've seen IT departments run down by the management team, so performance concerns on both sides are well-founded. Anyone who thinks their manager shouldn't ask for a suitably abstracted toolset for judging performance is asking for a stupid manager.

Re:Hahaha

[Anonymous Coward]

As an IT professional who was just fired from a job shortly after management started to ask for all sorts of new metrics from our ticketing system, I also fully agree with the parent statement. Get your resume together and start getting it out to recruiters pronto.

In other words, this is a sham. There ARE no standard metrics because every single company and department are different. The only real metric that matters is if your customers are happy and satisfied, and that they could find out better than you could. You're being set up.

Re:Hahaha

[BenJCarter]

You're fired...

Metrics suck

[SJ2000]

Didn't we just have a story about how metrics suck?

Re:Metrics suck

[Ethanol-fueled]

Top-post whore responder advice giver here. As somebody who works in a company with very similar numbers described in the summary, the few IT personnel should take over some of the infrastructure programming duties like databases and internal support software and use their existing knowledge of the corporation to prepare to either take on more work or transfer to a different position or department within the company.

Don't know how to program? Hit the books - your job may depend on it.

Re:Metrics suck

[ScuzzMonkey]

Bad metrics suck, good metrics are useful data.

As folks have already mentioned, time to answer and time to resolve are both important, and I think you have to watch for re-opened as well to curb "how fast can I shove this under the bed?" resolution games.

My favorite is average tickets per user, though. Particularly on a small team, what you really want to gear your measurements toward is preventing incidents in the first place. It is helpful to know what your overall ticket volume looks like, then, and to aim to decrease it over time in the same way you might try to decrease time to answer and time to resolve. That's important, because as the previous article suggests, if you will get what you measure... and your overall goal should not simply be to answer tickets faster and resolve them more quickly, but to not have as many in the first place*. Every issue represents a waste of somebody's time and therefore corporate resources that could be put to more productive uses. Steadily decreasing mtta and mttr are nothing to cheer about if your ticket count is increasing.

But you can keep it simple. You can drown yourself in metrics and lose sight of why you're tracking them and what you really want to accomplish. You may not really need any more than these few; better to start small and add what you need when you need it. I know there's always tension over getting a system in place that can capture what you need for historical purposes when you realize you need to know something new down the road, but resist the urge to over-collect. Half the time you won't need it all and are just wasting time getting it in the system.

* There is a caveat to this; in some organizations, I actually DO want to see an increasing ticket/user count, at least for a time. This is something I shoot for when relations between IT and users have broken down badly enough that users have stopped reporting problems to IT because they feel it's useless, and their issues are never resolved anyway. In those cases, a rising ticket count can represent an increasing trust level, which is good. You generally won't fix issues you don't know about.

Re:Metrics suck

[CAIMLAS]

Metrics is just a management word for "bad statistics".

With a distribution of 3, it's not really possible to have statistics of meaningful nature. You've got shared responsibilities and bounce things off of each other. One person may open and close more tickets, have a shorter duration for tickets, etc. - but he's only doing the actual "work". The others may be giving him all the input necessary to complete the task(s).

Ideally, your ticketing system will reflect, very vaguely, who's doing work and who is not, but even then it's not going to be well representative of what's actually going on.

People do different types of work, of different levels of difficulty. For instance, I may do one ticket on Monday, three on Tuesday, and one for Wednesday through Friday. Why? Aside from the fact that I'm bad about actually doing tickets for my work (my god, I'd not have the time for work, and then there'd be more things that aren't getting done, making us -all- look bad), there's the reality that my tickets aren't terribly easy, often requiring hours of log perusal and research to try to fix problems. Meanwhile, the guy who knocks out 40 tickets a week - malware disinfection, workstation reinstalls, etc. - has fairly wrote work of a repetitive nature, comparably. Also, he's following instructions or asking for advice on a regular basis, even if I'm not his boss.

You said so yourself: you're a member of a 3-person IT team. The only use 'metrics' have here aside from what should be plainly obvious in a group of 3 (who's fucking up, who's not getting back to people, who's not doing work) is to keep track of what amounts to customer requests and problems. X workstation needs to be reinstalled, Y server has a crashing whatever, and so on. If you're working on and sharing a ticket queue, you are all mutually responsible for all of the tickets: if something isn't getting done, it's everyone's fault (or nobody's fault). You may consider presenting your metrics in the light of this reality (like statistics, metrics can lie, too).

Re:Metrics suck

[MichaelKristopeit422]

i'm dealing with a bug detecting team in india right now through a client (their choice to use them)... they use "fixed" bugs as their main metric.... so they file tons of bugs... we waste tons of time explaining why there isn't a bug, then they agree and say "ok we'll mark it fixed" and we say "mark it rejected" and they say "ok we'll mark it fixed".

it's a nightmare.

metrics and quotas will only lead to less overall return... unless, of course, you're just trying to create a jobs program.

Re:Metrics suck

[Anonymous Coward]

Totally agree.

My small team was taking a beating and getting little support from management. Issues were piling up, we were getting randomized, and I felt like I needed another staff member. Management wouldn't budge though. I needed to prove to them that I could manage my department better. Nobody even cared about IT metrics at our company. Rather than see them as an evil thing, we embraced them to gain credibility in our organization.

We started by outlining some of our support boundaries, types, and set goals for response times. Then we customized our ticket system and added in some categories and priorities.

For Support (meaning something is broken and must be fixed)
High pri: 24 hour solution
Medium pri: 1 week
Low pri: 3 months or more, not a critical metric

We also had a "Request" category (not a fix, but doing standard tasks like toner replacement, adding e-mail alias, help train someone, etc.). These had different goals for solutions. We even had ticket categories for maintenance and projects. Though they junk up the ticket system, it allowed us to track time going into other tasks and paint a picture of a staff member's whole day of work, not just the support end.

Some of our favorite metrics:

1. Average time on support OR request tickets aggregated by our team or split down to individual staff members, and divided by priority level.

2. Total time spent on tickets by user (so I know if we're actually working or lounging around). ---this is a motivator for staff to actually enter tickets and get credit for their work. Some flexibility is necessary here because people eat lunch and go to the bathroom - you can't expect this to add up to 40 hours a week perfectly.

3. "Most troublesome user or department." I don't advertise the data on this report, but it lets me know who to focus on with either training or nudging their boss a little. Execs get interested in this once in a while, and the users that found out we keep such a log try to keep off of it. Many will joke about it, but still ask "Hey I'm not in the top 10, right?" It encourages users to be a little smarter and not lean so heavily on IT for silly things.

4. Most troublesome product - THIS IS BY FAR THE BEST METRIC. This has helped me gain support in dropping old junk software, getting better solutions, setting up training for people, or creating general awareness that we need to improve. For example when I was able to quantify how much time it took when people forgot their passwords for our cloud-based e-mail password, I got support for Office 365 and ADFS for single sign-on. In the interim, people stopped took a little more personal responsibility because they realized that password stupidity = preventing us from getting value-added projects done. Stuff like this is beautiful because it takes most of the heat off of IT staff, and informs other managers of our company-wide pain points that we should invest in.

5. On the same lines as #3, identifying how many high pri tickets are out there vs. medium or low (and possibly categorizing by customer or department) can sometimes sniff out IT abusers that need stuff now now now. Tread on this carefully as you don't want to constantly wage war with your metrics, but with some political prowess and some data to back you up, you can start solving some of these problems.

#3 and #4 are big for us. Don't just think of metrics in terms of how you will hang yourself. Think about the bigger picture and answer questions that will help motivate your team in a positive way, get credit for what they/you do, and enable you to tell a better story to management. You probably will want to develop two sets of metrics - a more detailed set for yourself so you can manage effectively and proactively identify problems (and solve them in advance before your boss gets to it), and a more general set for your boss so you can deliver a simple and concise picture without getting him too involved with the details you can handle yourse

Here's a few...

[HerculesMO]

Time to answer call, time to resolve ticket, abandoned tickets (unresolved).

If you google a few of those it will bring some more, but that's a simple start.

Re:Here's a few...

[suutar]

Number of calls back after initial call (measuring, in theory, how often the initial call resolved the issue) Number and duration of system outages (if you're doing sysadmin stuff as well as support stuff)

easy

[Anonymous Coward]

"what good metrics should be in regards to mttr mtbf etc"

Easy, there are no good metrics. Metrics don't lead to improved business outcomes, they rarely cover enough variables to tell the whole story, so all they lead to is people gaming the metrics, most likely leading to worse business outcomes.

Metrics are favoured by lazy management.

Re:easy

[mewsenews]

Metrics are favoured by lazy management.

Look, using metrics doesn't indicate lazy management.

Look, using metrics that you don't have available doesn't indicate lazy management.

Look, using metrics that you don't have available so you ask your staff to measure their own metrics doesn't indicate lazy management.

Look, using metrics that you don't have available, so you ask your staff to measure their own metrics, but you don't know what metrics they should measure, so they end up asking Slashdot what some good metrics are, doesn't indicate lazy management.

Re:easy

[LS]

I don't buy this. I (hate to admit that I) worked at Zynga, and their entire business model is based off of metrics, both internal and customer metrics. They are behaviorists, and brought in financial industry data modeling to design games and make business decisions. It works, and it works fucking well. In the long term this Skinner box model may or may not work for creating a sustainable business, but it has worked for the last couple years. People may not like their games, but they keep playing them.

The problem is bad or incomplete or misinterpreted metrics. Metrics in and of themselves or not bad. The problem is with the people that use them.

LS

3 People?

[stevenfuzz]

Simple... if you have a 3 person IT team at a 300 employee company and your site / it infrastructure isn't in nuclear meltdown your probably doing good. Looks like they are going out of house for IT. Welcome to the cloud-future, where your job is dissolved for magic.

optimal?

[Anonymous Coward]

talk about flow, about bottle necks. Visualize workflow. Look at Henrik Kniberg's paper on kanban as applied to IT Ops. My guess is that your ticketing systems will provide low value data on volumes on resolution time - gear up - visualize the pipeline. check http://www.infoq.com/minibooks/priming-kanban-jesper-boeg - turn the conversation around to "business value" - don't get wrapped in the ropes of volumetrics /peace.

No Metrics

[Anonymous Coward]

There is no metrics system that can't be gamed.

If you set it for "total tickets fixes" (higher=good): you just encourage people to report trivial problems you can fix easily.
If you set it for "total tickets" (higher=bad): you refuse to do things, add features etc, or you make it hard to contact IT to log a fault
If you set it for "time taken per ticket" (higher=bad): you end up pushing kludge solutions
If you set it for "user rated response" (higher=good): you end up blackmailing the end users to rate you 10/10 otherwise their emails/logs/dirt etc get published and sent to boss/wife/etc

Ask your manager how their performance is evaluated? Then start suggesting ways they could bust their KPIs, and they should get the drift.

Re:Good luck

[ScuzzMonkey]

It's not unusual for management to be clueless about what exactly it is that their IT staff does on a daily basis, nor is it unnatural that they should take an interest. Often, it's a good sign when they actually ask the guys doing the work what the metrics should be... it indicates some degree of trust, and they haven't simply read an "IT Management for Dummies" book over the weekend laying out some arbitrary system that isn't going to fit your organization.

As a more cynical commenter points out, it also provides the opportunity to create a measurement system that you can game to make you look good. But I think it isn't a terrible sign that the bosses care what their employees are up to. It may represent an opportunity to explain what you think is important that perhaps they hadn't considered previously.

how to survive in the corporate environment

[decora]

1. make your numbers.

nobody actually cares what 'the numbers' are, or if they actually mean anything. but you have to make them.

you might ask yourself - isn't this a huge waste of time? isn't it completely counter productive? doesn't it actually decrease efficiency? aren't the metrics measuring completely the wrong thing? as the slashdot story the other day said, aren't bad metrics actually worse than no metrics, because they cause people to do inane, wasteful things to make their numbers?

well, your problem is that you are asking yourself. in a corporate environment, do not ask. just do.

just make the numbers.

hopefully, if you get good enough at 'making your numbers', you will have time left over to actually do some work.

2. but what about the theory of capitalism, the free market, efficiency, etc?

its all bullshit. just like the theory of communism was bullshit. what statistics and 'numbers' were reported to the government were just flat out garbage. people somehow managed to make the system work through personal relationships and working-around the assholse in charge. but most of the theories it was built on have no resemblance to reality. think about it - if efficiency really made for the best corporation, why would you be spending 4 hours a week filling out meaningless statistical performance reports that nobody will ever read, let alone understand?

the only difference between the soviet union and 'the west' is that 'the west' still hasnt collapsed yet.

Done in one

[Weaselmancer]

Seriously I just have to say that this is the single funniest comment I've ever read on Slashdot. Laughing, pointing at the screen, drug my wife over here to have her read it funny. Brutal. Absolutely brutal.

From one cynical bastard to another, I salute you.

Re:Done in one

[turing_m]

You cynical man. For all you know, upper management have a budget flush with cash and have singled out someone in the hard working but unacknowledged IT department for a raise and a promotion.

Re:Done in one

[nahdude812]

You cynical man. For all you know, upper management have a budget flush with cash and have singled out someone in the hard working but unacknowledged IT department for a raise and a promotion.

Don't forget the free pony.

Re:call / ticket time is bad metrics

[DarthBart]

I got written up once because my ticket stats were radically different than the other people on my team. 15% lower "total time on tickets" but 20% more tickets closed. I was apparently fudging numbers and closing unresolved tickets.

Fortunately, a trip to HR with a ream of printouts from closed tickets proved otherwise.

Still left the company a few months later.

I just can't add anything to that

[bfwebster]

Might as well close the comments now. :-)

Go look up Robert Austin's book on measurements and management. Read it and recognize that you've been given a task that is at best counterproductive and at worst impossible. Dust off your resume, because it may be more than one of you that are getting fired. ..bruce..

My metrics

[slashmydots]

Well, when they say small, I'm the sole IT worker and thus the IT Manager at a 4 server, 40 workstation business so my only metrics are: I didn't make them spend a bunch of money, nothing lit on fire, I didn't quit. That's seriously about it and this quarter, I got all but the middle one but I wasn't the one who ordered that HP workstation, nor would I have, so it's sort of a gray area lol.