Slashdot Mirror
Ask Slashdot: Good Metrics For a Small IT Team?
First time accepted submitter shibbyj writes "I'm a member of a small 3 person IT team for a medium sized business (approximately 300-350 employees) that has multiple locations internationally. I have been tasked with logging our performance using the statistics from our ticket management system. I've also been tasked with comparing these stats and determining if we are performing above or below what is considered optimal. I'm wondering what people opinions are on what good metrics should be in regards to mttr mtbf etc. I have had trouble finding information on this."
One of you is getting fired
Didn't we just have a story about how metrics suck?
Time to answer call, time to resolve ticket, abandoned tickets (unresolved).
If you google a few of those it will bring some more, but that's a simple start.
The price is always right if someone else is paying.
"what good metrics should be in regards to mttr mtbf etc"
Easy, there are no good metrics. Metrics don't lead to improved business outcomes, they rarely cover enough variables to tell the whole story, so all they lead to is people gaming the metrics, most likely leading to worse business outcomes.
Metrics are favoured by lazy management.
Simple... if you have a 3 person IT team at a 300 employee company and your site / it infrastructure isn't in nuclear meltdown your probably doing good. Looks like they are going out of house for IT. Welcome to the cloud-future, where your job is dissolved for magic.
I'm afraid if some one is asking for metrics of three people supporting 350 in international terms does not know what the hell they are asking for or what it is that you 3 do. That being said, it can be expected you will be scrutinized over every little detail. Be careful. I won't ask, but I do wonder who you work for. I have been a similar situation and it is completely frustrating. Good luck...
You're always below optimal, by definition of optimal.
talk about flow, about bottle necks. Visualize workflow. Look at Henrik Kniberg's paper on kanban as applied to IT Ops. My guess is that your ticketing systems will provide low value data on volumes on resolution time - gear up - visualize the pipeline. check http://www.infoq.com/minibooks/priming-kanban-jesper-boeg - turn the conversation around to "business value" - don't get wrapped in the ropes of volumetrics /peace.
There is no metrics system that can't be gamed.
If you set it for "total tickets fixes" (higher=good): you just encourage people to report trivial problems you can fix easily.
If you set it for "total tickets" (higher=bad): you refuse to do things, add features etc, or you make it hard to contact IT to log a fault
If you set it for "time taken per ticket" (higher=bad): you end up pushing kludge solutions
If you set it for "user rated response" (higher=good): you end up blackmailing the end users to rate you 10/10 otherwise their emails/logs/dirt etc get published and sent to boss/wife/etc
Ask your manager how their performance is evaluated? Then start suggesting ways they could bust their KPIs, and they should get the drift.
Metrics. Excellent, I hate when bosses use the Imperial system.
All jokes aside: If you care about your job in this economic climate, I suggest you do what your 2 other teammates are doing - picking through the stats that make YOU look the best. The company isn't going to look out for you. IT is an expense to be cut, remember. Boosts the temporary bottom line, promotes "growth" in this fiscal quarter, gets the investors going so the CEO can shuffle another fold into his golden parachute. If non-important metrics are selected that sacrifice your job, it's a brief victory lap straight into the unemployment line.
We can't answer your question, though. In the end, I recommend you watch a clip from "Office Space" - wherein the Bobs interview the employees:
Bob: "So tell me, what is it, exactly, that you do here?"
If you can't answer that question, you probably should be job hunting already. Or should have kept a copy of the job posting from when you applied.
There's a spot in User Info for World of Warcraft account names? Really?
I have been tasked with logging our performance using the statistics from our ticket management system. I've also been tasked with comparing these stats and determining if we are performing above or below what is considered optimal.
Standard ticketing system metrics are no good. Add a post-incident survey to your customer interactions. Your metrics don't measure performance, unless you can actually measure how 'hard' the ticket is intrinsically / what skill is required, and how well the ticket is solved.
A good solution might prevent further issues, a bad solution might cause more issues to occur. The fewer IT support issues a customer has, the more each resolution was worth, because that indicates their problem was actually solved.
Define a new metric: "Ticket importance" = Importance level given to the issue by the customer relative to the other issues.
Define a new metric: "Customer Happiness" = Score given by the customer with the resolution of the issue, from 0% to 100%.
Define a new metric: Revenue per Ticket = Revenue of applicable IT support Service Delivered per period of time divided by average Number of Trouble tickets for that service for the same period of time.
Define a new metric: Ticket Weight = Rank of the ticket out of all other tickets issued by the service, as a percentage. 100% would indicate it's the only ticket that matters, 0% would indicate solving the ticket is not an accomplishment, e.g. wasted time
Define a new metric: Ticket worth = Ticket Weight * Revenue per Ticket
Define a new metric: Ticket resolution value = Ticket Worth * Customer Happiness.
Sum the resolution value of all the tickets solved by IT support personnel. When multiple IT persons are involved in the same ticket, calculate total hours spent on the ticket, and divvy out the 'Resolution points' based on the amount of time spent on that ticket by each team member.
Is shit broken?
Does shit get fixed fairly quickly?
Are your people busy, but not swamped?
Yesterday's Slashdot, to be exact: http://it.slashdot.org/story/11/12/14/225204/the-four-fallacies-of-it-metrics
-- This sig is only a test. If this were a real sig it would say something witty. --
..... is how much failure they have!
If shit is working, don't come bitching! Don't ask about shit you don't know shit about and do your job like you are doing yours!
- Percentage of staff with ITILv3 foundations certification: zero.
I know this because of your question. Watch thes videos as a start: http://pmit.pl/en/it-management/free-itil-v3-course-collection-of-itil-v3-moviesdarmowe-szkolenie-itil-v3-zbior-filmikow-o-itil-v3/ -- and sell a formal training course to your management.
The people joking about how one of you is getting fired, or you're all getting outsourced. . . probably true. Learning ITIL is all about learning what's important to your business stakeholders, how to monitor/measure these things, and how to make sure you're always making the right decisions based on the business priorities.
If you can't convince them to pony up for you three to take the certification course, then pay for it out of your own pocket, you'll need it to find a new job.
Seriously. Once management fall under the belief that they can have reports automagically generated for them by measuring your working habits, you will never hear the end of it. Why does X have less keystrokes per hour than Y? You only committed 15 lines of code yesterday. You must not be working. Why did this junior employee fix 100 bugs, when this senior only fixed 10? And take a look at yesterday's topic too.
as not all ticket take the same time.
Do want quick password resets to count the same as doing some think that needs a more time? like setup / reload a pc?
Your ticketing system needs or needs to have added an automatic followup to the customers. The system sends out an email after every ticket asking "Did the problem get resolved in a reasonable amount of time? Did the IT staff respond in a way that enabled you to get back to work?" Nothing more complex than that, though you can parse things out by ticket priority (though deciding what's a higher priority than other things is, just by itself, a major undertaking).
Your goal should be to increase the percentage of positive responses.
Why this touchy-feely stuff instead of a hard metric? Easy. There are no metrics that work in your situation. It's quite easy to argue that there are no metrics that work, period.
By adding this email feedback to your ticketing system, you have met the requirement to come up with a metric derived from the ticketing system.
Selling this to management can be simple, depending on how you handle it. Something along the lines of "Given that the IT staff is so idiotically understaffed, we must be given the agility to solve problems instead of meeting random metrics. Only our customers can know if we met their needs, considering all pertinent factors. Someday, when we actually have enough people and money to divide work more rigidly, we can add metrics like timeliness of ticket closure, etc." Then you hope they never notice that you never divvy up the work rigidly. All of this requires having an IT manager who is dedicated to the inescapable truth - that their function is to keep the MBAs off your ass and let you do your job.
I've worked where my performance was measured in this way. It can be heaven.
One more thing - if your upper management doesn't already have faith in you, they'll never go for it. They need to already appreciate your contribution to go along with this. The very fact that they're asking for metrics tends to suggest they don't sufficiently appreciate you now. If that's the case, than all I can say is that I've worked under those circumstances, too, and my heart goes out to you.
> determining if we are performing above or below what is considered optimal
Scenario 1: you are below optimal -> you are inefficient so they replace you
Scenario 2: you are above optimal -> you are overkill so they replace you
Bottom line, I would rent The Wire and learn how to "juke the stats" because that's the only way you won't get to jump on that grenade.
Been there, done that - my advice: be just under optimal so you have room to grow and show improvement, but don't be too low so they don't feel the need to consider a business case for outsourcing.
lucm, indeed.
Optimal being 'Best possible', how does one perform above what is optimal? You are either perfect (optimal) or not perfect (sub-optimal).
Comment removed based on user account deletion
I'm not with most IT management on this one, but I always thought the best metric was customer satisfaction. For instance every time I open a ticket with Cisco I get a survey at the end of like 5 questions. Was my problem resolved, was the person polite, etc.
The other metrics suggested are things to graph and look at trends. Are repair times getting worse or better? Is the average time per ticket going up or down? They are great int he aggregate. They break down quickly when divided. Only one guy on your team knows network devices, so he gets all the network devices which include the 8 hour fiber cuts, so his times always are worse than the guys fixing printer problems, as an example. You have to be very careful as you start to divide them up.
At the end of the day though you're trying to make the customer happy. Track it, and see how your staff is doing. If people are happy with their IT support, your department will be seen in a more positive light.
1. make your numbers.
nobody actually cares what 'the numbers' are, or if they actually mean anything. but you have to make them.
you might ask yourself - isn't this a huge waste of time? isn't it completely counter productive? doesn't it actually decrease efficiency? aren't the metrics measuring completely the wrong thing? as the slashdot story the other day said, aren't bad metrics actually worse than no metrics, because they cause people to do inane, wasteful things to make their numbers?
well, your problem is that you are asking yourself. in a corporate environment, do not ask. just do.
just make the numbers.
hopefully, if you get good enough at 'making your numbers', you will have time left over to actually do some work.
2. but what about the theory of capitalism, the free market, efficiency, etc?
its all bullshit. just like the theory of communism was bullshit. what statistics and 'numbers' were reported to the government were just flat out garbage. people somehow managed to make the system work through personal relationships and working-around the assholse in charge. but most of the theories it was built on have no resemblance to reality. think about it - if efficiency really made for the best corporation, why would you be spending 4 hours a week filling out meaningless statistical performance reports that nobody will ever read, let alone understand?
the only difference between the soviet union and 'the west' is that 'the west' still hasnt collapsed yet.
MTBF = Mean Time Between Failures
MTRR should be MTTR = Mean Time To Repair
Explanations would've been nice for convenience but frankly if you didn't know those acronyms anyway you probably don't have much to contribute to answering the question.
where people are constantly calling me asking to fix stuff, then my numbers will be awesome!
god, capitalism is awesome.
The only truly meaningful metric is customer satisfaction. After each ticket is closed send a survey email to the user. If your team plows through enough tickets you get a statistically significant success % per tech that you can compare to the other techs.
Without knowing a lot more about the nature of tickets it is hard to give a better response. It might be very important to gauge the difficulty (trivial-hard) and novelty (common-rare) of tickets but it could also be a waste of time. Does one tech plow through dozens of tickets a day or a few? Are the techs specialized? Anything email related goes to joe, hardware issues to tom, etc.. Are the tickets auto-generated from emails, called in or do they fill out a web form?
Given the size of the team the company thinks 1) you aren't getting the job done, 2) one or more people on the team are dead weight or 3) you are overworked and need more people. I wouldn't bet on #3.
Falling back to metrics is a lazy manager's way of proving to her superiors that her drones are operating at peak efficiency. The most lazy of all will rely on utterly meaningless metrics such as the number of help tickets closed per day, per individual per day, etc. A metric such as this is completely useless as all tickets don't require an equal amount of effort to complete. Diagnosing a problem due to an intermittent hardware issue doesn't take the same amount of effort as helping a user change their password. Unfortunately these types of issues generally comprise the vast majority of tickets generated and therefore often end up being the ones that are 'measured. ' This often leads to a drop in morale and thereby negatively impacts performance; ironically the opposite of what the whole exercise is attempting to accomplish.
Trouble ticket data is primarily useful for detecting trends, thereby helping an IT team appropriately focus their human capital on issues that will enable their users to be more efficient. Going back to the password issue above, the speed and alacrity with which the IT staff help users change their passwords isn't a useful metric at all. A more meaningful metric would be the frequency of password change requests before and after the installation of a self-service password reset solution that was put in place in response to the analysis of help ticket data that showed that this was one of the most frequent issues and one that could be easily solved with little effort and financial expenditure. Measuring a sharp drop in password reset requests would show that the solution worked and was therefore beneficial to the organization by enabling users to help themselves, resulting in their having more time to concentrate on their primary tasks, and also by allowing IT staff to allocate their resources on issues that are less amenable to resolution via automation.
Unfortunately, in my experience, ticket systems get used to determine useless metrics such as the first example mentioned above, and therefore end up being the bane of IT staff, rather than a useful analytical tool.
While not a standard metric, consider the amount of time you spend planning and implementing upgrades to your infrastructure, as opposed to the amount of time spent supporting it. In a healthy company, the weight is towards planning and implementing upgrades to your infrastructure, where not a day passes that a new piece of equipment isn't being delivered. Why, you ask? Because it shows that there are enough people to handle the daily problems adequately, freeing up others to plan for several months (or years) into the future...to deliver a feature before it is requested, to move the network to a faster speed before you hit bottlenecks, to ensure that people spend more time completing their work on their machines than waiting for their machine to finish a task or complete an upload / download. To take time consuming, menial tasks away from the workers, and free them up to pursue tasks that might bring in more money for the company.
In an unhealthy company, you spend more of your time dreaming about upgrades. You end up taking care of old, inefficient equipment, and are stuck in meetings, trying to explain why a minor upgrade can have a large impact to a group of people who are 'too busy' to learn anything more technical than messaging on their Blackberrys / iPhones, which they do while you speak. You don't have enough people to perform adequate daily support, and plan for future upgrades (which may require several hours of research, and speaking with vendors on the phone, to get a proper quote; on top of studying new technology or attending a training course). If you've hit a bottleneck in the last several months for any feature on a computer / network, and you're not Google / IBM / someone playing with a supercomputer, you're doing it wrong.
But then, you get what you pay for. If you are a company like this, leave. The company isn't going to grow, the culture is dead, and you have a chance to short the stock before the collapse. They'll keep switching CEOs / figureheads, and merging, but that kind of corporate culture is like a virus. See what happened to HP when Compaq merged with them.
I am John Hurt.
Seriously I just have to say that this is the single funniest comment I've ever read on Slashdot. Laughing, pointing at the screen, drug my wife over here to have her read it funny. Brutal. Absolutely brutal.
From one cynical bastard to another, I salute you.
Weaselmancer
rediculous.
Customer/User satisfaction. If problems are solved in a reasonable way (in terms of time considering your workload, budget and expertise) then that's the only thing management should care about. You're not running a support call center where your job is to read solutions from pre-written scripts - you're solving real problems with real people. There is no script.
Management should be surveying their employees and your reports and work out if more people are needed to do general support. Are you finding time to do project work? If no, then maybe they need to hire somebody to keep users off your back a few hours a day while you do projects.
If you have to go down that path, I'd go with TAT. Simple to measure even over old data, hard to game consistently, and closely related to customer impact. If you want to, weigh it by customer-perceived severity of each request so that people don't boost their numbers by cherry picking easy tickets.
To do list for Windows
I found this gem in the other metrics thread the other day.
-2000 lines of code.
Weaselmancer
rediculous.
There is only one metric that really matters, and that is what your users think. You can collect all the stats you want, and put whatever spin you like on the figures, but if your users think you suck, then in the view of 99% of the company, you suck.
So my suggestion is; do what Cisco and other customer focused companies do, and for every ticket closed, send a satisfaction survey to the user. Don't make it long, it only has to be 4-7 questions where they rate the key things you did on a scale of 1-5, and should take the user less than 2 minutes to complete.
Then, you will have incredibly meaningful stats to show the good work you are doing. Or, you will have the precise information that shows what you need to do to get recognized for doing good work.
Just think about it., what is better:
a. For the last month there were 82.5% if tickets were resolved within the department KPI target,
or
b. For the last month 82.5% of staff indicated they were happy or very happy with IT support
The best answer is to not use tickets as your metric. I worked at a smaller company and our metric was a simple yearly user survey. We would ask what they thought of their equipment, the how the support team did (response time, niceness, knowledge, etc) and even took it so far as to ask for suggestions on what could be improved. In the end, how many tickets you close really doesn't matter. What matters is how happy your users are. If you are doing the job correctly, the majority of your users will at least be satisfied, if not happy. If the majority are unhappy, this will give you specific ideas on areas you can improve on.
"Information wants to be expensive" - Stewart Brand, the same guy who said "Information wants to be free"
1: You didn't give enough information on what you support to get an accurate answer.
2: You shouldn't be the one evaluating your own own job (see very first post)
3: Here are several common metrics which may or may not apply to your situation as they were pulled from a couple of very large corporations metrics system.
CSAT: customer satisfaction based on automatically emailed surveys
AHT: average handle time, or the time you spent on the phone resolving the issue
ACW: average call waiting or how long did they have to be on hold to talk to you
Call Resolution: self explanatory
Logging: how much did you actually log in tickets to the number of calls received.
It gets pretty granular after this with things like "overall satisfaction" "agent expertise" (you're the agent) "agent listening"
One last thing, I actually snorted reading the first post because I had an "Office Space" flash back, time to hide your red Swingline.
"If any question why we died, Tell them because our fathers lied."
Some metrics have been shown to be clear indicators of good performance in IT teams. Hopefully the following equations are clear enough. 1./ Number of metrics minus number of metrics known by employees 2./ Time in day minus time spent on developing metrics 3./ Number of metrics forgotten divided by Number of managers who care about metrics
Time to answer call, time to resolve ticket, abandoned tickets (unresolved).
In business school it is a common theme in various classes that you get what you reward, not what you ask for, not what is necessarily best for the organization. Here is a highly relevant Dilbert cartoon illustrating this point, http://dilbert.com/strips/comic/1995-11-13/.
The underlying problem is that metrics applied to humans leads to people working towards the metrics, not necessarily doing good work. It is a classic environment for unintended consequences. Its not even that the people are necessarily being opportunistic, there is also a certain amount of practicality. If you are being measured by some metric and keeping your job or getting a raise is dependent upon that metric you may quite rationally decide to act to that metric rather than what is necessarily in the best interest of customers.
Are you measured by resolved tickets? Then tickets will get resolved quickly. Not necessarily thoroughly, completely, or robustly resolved. Which leads to related followup tickets because of a minimal effort put into resolving the original ticket. I saw this in a programming environment where the tickets consisted of new features or bug fixes.
Are you measured by abandoned tickets? Then tickets will get resolved, even if they don't reasonably deserve to be considered resolved. You will get things unnecessarily classified as "unable to duplicate", "insufficient information", etc.
In these two examples, where is the difficulty of the task factored in? Not all task, tickets, are equivalent. Furthermore sometimes there are external dependencies, a part is being shipped, where is this factored in?
The metrics you offer are reminiscent of stats from call centers. There such metrics are a little more reasonable, not perfect but perhaps OK, given that the calls are somewhat equivalent in the amount of effort required, a small number of minutes not hours, and that they are randomly assigned. Over the period of say a month the large number of calls handled by any operator will resemble a normal curve with respect to effort required. For an IT organization the evaluation period may need to be some number of years to get to a normal curve with respect to effort required.
I can only imagine that most people replying with derision about metrics have never been in the position of having to justify what they are doing, and when they have been they have acted dishonestly.
People that actually work in the real world, with real companies with real budgets, and that have some self respect, honesty and pride in what they do will have to justify their salaries or rates somehow, and one of the tools used is some kind of metrics.
A professional will find metrics that are meaningful to both their team and their bosses or paymasters, and contrary to what most people are implying here, they can be quite useful to identify reasons for which a team is overworked and maybe bring somebody else on board.
It should also be pointed out that proper metrics may point to a team that is overstaffed, but an honest professional should not fear this since drawing a salary for doing nothing is frankly not my idea of a honest day's work (those people asking to make up or game the metrics simply are unethical, if you think metrics are a sham then do please suggest how you intend to evaluate obejectively if you are becoming better or worst at doing your job).
IANAL but write like a drunk one.
Might as well close the comments now. :-)
Go look up Robert Austin's book on measurements and management. Read it and recognize that you've been given a task that is at best counterproductive and at worst impossible. Dust off your resume, because it may be more than one of you that are getting fired. ..bruce..
Bruce F. Webster (brucefwebster.com)
Nagios isn't too difficult to set up to monitor lots of things, and lots of useful uptime metrics for every service, planned and unplanned maintenance, etc. fall out quite naturally from it. And you can kinda just keep adding modules to it into it and grow it until it's full of awesome.
I haven't personally used Redmine yet, but have been using Trac and everyone seems to agree that Redmine is the clear successor in terms of lightweight but capable trouble-ticket / project / task management systems.
Well, when they say small, I'm the sole IT worker and thus the IT Manager at a 4 server, 40 workstation business so my only metrics are: I didn't make them spend a bunch of money, nothing lit on fire, I didn't quit. That's seriously about it and this quarter, I got all but the middle one but I wasn't the one who ordered that HP workstation, nor would I have, so it's sort of a gray area lol.
Touch up your Resume', go tell your boss to get bent, pound sand, etc. and look for a new place to work. Anyone who needs metrics on a three person team deserves anything they get, up to and including a swift kick in the ass. If the manager can't figure it out on his or her own, they should be the one being sent out the door with boxes.
"My immediate reaction is "WTF? What kind of moron doesn't make things 64-bit safe to begin with?" Linus
Bingo. This and the first post are the ones to pay attention to. Either nip this shit in the bud or get out of there. You're not going to be able to come up with meaningful "metrics" for a 3 person team. If I help ten people attach documents to emails before lunch and you spend all day helping a team optimize their new production process, who did more meaningful work? With crappy metrics, I did. I've got ten closed tickets and you've got one.
You shouldn't need some complicated process to track and evaluate the work of three people. If there are three people and one of them isn't pulling their weight, all three know it.
A tracking/ticketing system is one thing (and a good thing) because it lets everyone see what need to be done and make sure tasks don't fall through the cracks. But, if that data is going to be used to evaluate performance, someone's going to have to develop a "difficulty rating" system and sit there assigning difficulty ratings to every item that goes through the system. Otherwise, I'm going to look like the king of turd mountain because I'm closing dozens of simple tickets every week while you two do all the work.
Before you do anything else, read Robert Austin's book, "Measuring and Managing Performance in Organizations".
The points I got from the book:
1) Measuring the wrong thing or in the wrong way makes things much, much worse.
2) Good measurements are possible but take a lot of hard work.
3) Measuring things that are easy to measure is almost certainly wrong.
I also endorse BenEnglishAtHome's comment timestamped 8:55pm.
I manage a department roughly the same size for a company about the same size.
I use two main metrics to see how we're doing:
1. The IT budget should be about 3% of the total company revenue. That's pretty average for companies of this size. If you're significantly under or significantly over, you need to do some soul searching to figure out why.
2. Every year we put together a survey and ask the exact same questions. Its going on 5 years now so we can compare our performance year over year. We ask about 20 questions and score them on a scale of 1-5. Things like 'hows training?' to 'how well does your cell phone work?'
Counting trouble tickets is mostly a worthless exercise. Although, you can manipulate it to your advantage. Start closing 120 tickets this month, 140 next month, etc. When you get to 240 tickets a month you can take that graph to upper management and say, "we're working twice as hard as we were 7 months ago and need to hire someone."
In the end, you have two ways to view this: as a bullshit exercise (and possibly an excuse to fire someone as others have said) or as a way to attempt to objectively evaluate your department.
----- obSig
"...determining if we are performing above or below what is considered optimal."
optimal, adj.: most favorable or desirable; best; optimum [Webster's New World Dictionary]
How can you possibly perform above optimal?
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
There are a number of ways you can do this:
1) For the next few weeks, only deal with issues in the ticketing system that can be resolved quickly. This shows how responsive you are on the "count of problems solved" and "time to resolution".
2) Always upgrade easy problems to "Extremely Urgent", so that they get picked up first (as per above).
3) Do NOT under any circumstances touch a complicated problem that requires consideration or actual work. Find someone to outsource it to. Then blame the outsourcing costs and lack of efficiency (obviously they do not have the same fast response time as you) for the problem.
Seriously: In a 3 man team, you and your manager should KNOW who is working and who is on facebook all day. If you are all working hard, then it is not time to add more pressure by introducing metrics, it is time to hire more help. If on the other hand you are all on facebook all day - well - then good luck to you in your new job at Walmart....
Meus subcriptio est nocens Latin quoniam bardus populus reputo is sanus callidus
I'd suggest "MTTD" and "AQ"
Mean TIme to Douchebaggery : Have your boss recruit some internal people to call your department with simple but real problems, and act like users who don't know anything. Record how long it takes the customer support rep to start acting like a complete douchebag. Longer times are better.
A-hole Ratio: As the manager, you should really know who the biggest a-holes on your team are. Determine if that's having a negative impact on other members of your team. Calculate the a-hole benefit (Tickets Closed) vs. perceived a-hole cost (lost productivity in tickets).
Both of these have a direct impact on your department's productivity. If people are individually productive but bad for business as a whole, get rid of them. But if they are "that good", maybe you need people who can better work with these a-holes.
Can they be gamed? Sure, by being less of a douchebag, or less of an a-hole. Sounds like a win-win.
Yes, I'm being flip. And yet, not really.
...means that your team may not have enough work to go around. I'm of the mindset that having a regularly light workload is good for an IT team, so that they're available when the ^#@$ hits the fan as it eventually will, murphey's law and all. But your managers probably are not.
Some users just want to get paid while other people do their job for them. If you are doing your job properly (and have a very different job to them) you are going to inadvertantly annoy them when you eventually have to tell them to do their job themselves. There's plenty of other situations because ultimately the user is not really when the loyalty has to lie - the organisation that pays you is supposed to be where the proirities are set. Even in retail the customer is not always right, that glib bit of heart warming bullshit isn't going to save your job if the customer tries to screw the company over (eg. shoplifter).
You can make the users more satisfied by employing attractive people to break the bad news that something will take a long time to fix instead of silently and rapidly fixing it. The user may be happy but they are not getting a lot done which conflicts with where the loyalties should really lie.
Performing above optimum is per definition not possible. So asking for this proves the Management is acting below optimum, measured by their ability to apply basic logic.
For such a small group I think it might be a better solution to just look at the tickets, filter for those with especially high response time, for those which where closed and reopened frequently, and for those which took longest for the first reply. Based on your gut feeling what management really wants (Do they look for excuses to fire people? Do they really want to improve the system?) you would either come up with a metric that strengthens your position most or with a metric which shows the bottleneck of your service or with a list of the worst cases [not by metric, but by common sense combining the different aspects] and try to find a common pattern / root cause to improve on.
Trolling is a art!
If the question is whether you are performing above or below what is considered optimal, the answer is easy. Below. Nobody performs above optimal, and pretty much nobody performs at an optimal level. If your IT department requires three full time employees to handle 300-350 clients you aren't that rare exception unless there're a lot of responsibilities beyond supporting those 350 clients (like say if you also manage 1000 web servers).
To measure the metrics you specified meaningfully, without wasting time over analyzing you should look up Process Behavior Analysis. Its a simple yet complex way of measuring any metrics over a long period of time, allowing you to apply concepts like `Continual Improvement` to those metrics to get them to where you want. Other metrics could be per productive hour, Backlog,if you have Sev1,2,3,... and you have targets, measure % Sev1,2,3 resolved out of criteria, if you do changes, measure %failed changes, or measure the types (e.g HW failure) of work coming in to explain certain deviations in your other metrics,etc... hope this puts you on the right track
I really hope you have good categories for your tickets. One of the things the business never understands (or refuses to understand) is that a significant number of the tickets should NOT be used as a metric to indicate there is a problem with the IT systems. If someone calls you because they need access to a shared folder, that's not an IT "issue" - it's a user request. Yes, they all take your time, and they need to be counted to show the total workload - but those types of tickets should not be used by the business to say "there's too many IT issues - you suck". Of course, there are many other types of tickets that should be categorized separately from real "issues" - including user training issues, password issues, client SW requests, etc. Just because you track it as a ticket doesn't mean it's a "problem".
Also: If there are lots of "easy" things you do that are not tracked as tickets - then you're hosed. Without metrics to back-up your claim that you spend XX% of your time talking to users, you have nothing.
We are having a year long experience in ticketing and performance stats/metrics systems (i.e. Atlassian Jira in combination with Tempo, nut that is just my two cents) and I see it like this:
For a system to deliver good metrics it needs a certain time to be setup and configured according to the requirements.
No one and no system (do not believe the high-gloss printouts!) can be setup in 5 minutes.
So it all boils down to this:
The setup will take probably a couple of days until you have tweaked everything and the color scheme pleases your boss.
By that time it is clear *who* gets laid off: *the one who configured that system* !
"Why?", you ask.
Simple answer: the other two were able two handle the IT workload without the third *"who was playing arodung with new software"*.
Questions anyone ?
The question is what management want from IT. Let them define what they want and need, then you can create metrics to show them to what degree you are providing it.
There is a pretty good chance that they will be clueless.
As a sidenote: measuring the performance of a 3 person team using the ticketing system sounds pretty stupid to me. There are just not enough numbers to provide it. Do you REALLY create tickets for everything you do?
And if it is of any comfort: In our 100K employee company with a pretty well structured IT support organization, most of the the metrics are still useless. This is proven by the fact that we are asked to do/not to do certain things "because they hurt our numbers", not because it makes sense from a productivity perspective.
And while we are at it: The simplest and most informative metrics I have see on a helpdesk (this was 1st line):
- Time to pick up phone (80% to be in less than 30 seconds)
- % of tickets resolved at 1st line (minimum 75 %)
- User satisfaction survey at certain numbers
If you can't spot the sucker at the table, it's you.
--
$tar -xvf
Twice you used the phrase "been tasked with". This is not the English you learned in school. Is your wife 'tasked with' washing the dishes?
There is a certain kind of lizard who embraces corporate speak, perhaps to kiss the ass of management or impress co-workers. This is a soulless sort of individual who is doomed to a lifetime of servitude.
If you reclaim your dignity, speak correctly and stand up straight you will either be respected or fired. Either way you will have found a better path in life.
...omphaloskepsis often...
Place I currently work, each team/department have a set of Service Level Agreements that are set for a year. Each department has a different set of SLAs (Note this is not done at a individual level). Each monthly team brief, your SLAs are discussed. If you keep hitting/exceeding your SLAs then a review at a higher level in the organisation is undertaken to question if the SLAs that the team/department set are too easy or not relevant any more.
It seams to work.
Nice piece, i remember looking somewhere else to do with this. Made me want to do another search in yahoo for more info and found your blog post. Great piece
As other have stated, you company and your department are unique. The only criteria is: does anyone complain? Does IT hold up new products/expansion? If the answer to both of those is "No", then there is no problem. Carry on.
Any metrics you collect will be totally pointless. Work for a company of morons? All tickets will be simple "How do I turn the monitor off?" types and solved in seconds. This makes you look good. The users will like you for helping them.l
Work for a company of technical literates? Any tickets they raise are bound to be utter bastards that take weeks of investigation/implementation because the user will fix most issues themselves. This makes you look bad but the users will still like you for not getting in their way.
So perhaps there is one metric: how many complaints has the team received and what action was taken to resolve those complaints?
Also, I'd look at updating the CV and begin to look for a new job. Better to leave than be pushed (unless voluntary redundancy is in the offing)
At the end of the day stats are used to compare people.
Why not take a 'free-market' approach:
Have people 'bid' points for tickets and the lowest bidder gets to do the work.
When they complete the ticket they get the 'points'.
This way how 'hard' a ticket is 'priced' in and people tend to do the tickets that they are best suited to.
Sig (appended to the end of comments you post, 120 chars)
Did you miss on this one?
http://www.google.es/url?sa=t&rct=j&q=bad%20it%20metrics&source=web&cd=2&ved=0CDwQFjAB&url=http%3A%2F%2Fit.slashdot.org%2Fstory%2F11%2F12%2F14%2F225204%2Fthe-four-fallacies-of-it-metrics&ei=QDjrTtaEKYufOqKw8asI&usg=AFQjCNHSyBkJ3WIN3Ii2ypE9Ga4b0Er0nA
If the overall work gets done, it's ok. If the work doesn't get done and there is an acceptable reason, it ok too. Otherwise it's not ok. There are always people that are better at one thing than others, but that doesn't mean the other people don't contribute other things, which you forget to measure. When the team is too big you should look at essential knowledge/experience, contributions to the team, etc. to figure out who to drop.
Metrics, like statistics, can be manipulated to appear to show many different things.
Surely there's some way you can dream up a metric that says you're all overworked and need to hire more people?
Just a thought
... and today's pet project has
What a tragic waste of time and resource. Use the time you're supposed to be gathering metrics to look for a new job.
I noticed that a lot of people seem to submit recommendations about your interaction with customers, but you could be also servicing the servers and network. If that's the case then add in there the number of outages that had no impact to the users. A power supply in a server went down, but the server had dual power supplies and no one noticed. Someone had a good idea at convening management to spring for that extra money. You connect your access switches with dual links and one of them went down?
If your going to collect metrics, let them know how much downtime the company avoided because you guys did your job. I've seen too many bad metrics around how much a person caused the company, only to find that 80/20 rule ended up biting them. 80% of the work was done by 20% of the workforce so all the murphy law scenarios only hit a limited number of people because they are the ones that did all the work. While all the people that sit back and did nothing reap'd all the metric glory.
-- Ed Bugg --You have freedom of choice, but not of consequences.--
This is the kind of crap that happens when companies with a crappy business model start realizing that they are spending more than they are making and begin making cut backs under the pretense of "efficiency".
Try to leave on your own terms if you can, rather than have the rug swept out from underneath you.
"Don't be so humble - you are not that great." - Golda Meir
Although you can get metrics from your call tracking system, Customer Surveys are a better judge of performance. Don't get me wrong, measuring the ability of the IT department to document stuff has its place but in the end you want to measure how well they are doing their job. Their job includes more than just the ability to write up a ticket.
"I myself am made entirely of flaws, stitched together with good intentions."
I think it is because IT doesn't have teeth, has no political leverage, and they have real skills which make people jealous. They are going to fire someone and expect everyone else to work harder.
Democracy Now! - your daily, uncensored, corporate-free
I was recently involved in putting together a small team to provide MIS reporting in an area where there's never been an MIS team, or any decent kind of metric other than volume, and identified mistake, before. Important how to define mistake, as they are often lagged, by days, months, or years, yet somewhere down the line a financial impact is identified. How is your team structured, how did you tailor the structure to the task?
The team consists:
In the above we tried to keep redundancy in key skill set for the task, mainly utilising our required Office packages plus SQL which I didn't mention but some have, while mixing it up with some strong skill flashes in all that can let them have an individual impact. Hopefully we'll tread new water and everyone will do well. Key is, did you and/or OP think how to structure your team, or were you just 'selected' as an existing team to fulfill a new task?
In case you missed it, here is the link to the story.
If you want the article itself, there ya go.
Having read that article, and the ones the author links to, it is quite clear why IT, and business in general, is in such shambles. People want to continually measure things, but they don't know what they want to measure.
I hadn't realized it, but I have been saying the same thing the author of the article said for some time: what is your goal? I use this quote from Seneca:
If a man does not know to what port he is sailing, no wind is favorable
This simple phrase should apply to every project, every metric, every everything. What is your ultimate goal? What do you want to accomplish? Only then can you answer the question: how do I get to that point.
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
I also work in a small company with two staff in IT. Myself and a web developer. We do use metrics to report to upper management how well systems are running. What I've done is create an excel spreadsheet with all of the systems we have running, the licenses, version and comments for each system. We then assign that system a 1 to 5 score and average the total at the end for a total composite score. One is if there are no problems, five is for serious issues. I generate this spreadsheet every month and send it to my boss. The other Major part to a small operation is customer service. In a small company, how happy your end users are imho more important than if the systems are running perfectly. We have to market ourselves and make sure we are available if anyone needs us. Being personable and friendly is very important. A final point of advice. Please don't make the metrics too complex. Management may be looking to let somebody go. However, they also may be looking to see what needs to be fixed and/or improved for capital expenditures. Metrics help track trends and you can use them to get the equipment and/or programs you need if you design them correctly.
... "number of * between anger management episodes"?
*Days, hours, minutes as appropriate.
Bukowski said it. I believe it. That settles it.
Find out which ones are important to the business - start counting them. Create a pipeline of system improvement projects - work on them. Document everything you do. The gods will smile upon you.
In the abstract sense where 'above' means 'better' I'd say no; if you were to say a certain speed is 'above' optimal, or a certain temperature is 'above optimal' then I'd imagine that's OK. But "better than the best" is nonsensical, so to 'perform' above optimal..? For me that doesn't make sense.
I do not want your cheap brainburning drugs. They are useless for work. And I am a working man today.
My point was that a "satisfaction metric" is almost useless unless applied as how satisfied their direct management is with their performance after considering several factors. In such a case it's not a single metric anymore.
Outside of very well defined situations it is not easy to come out with measurements that can effectively describe a situation to people with no grounding in the field they are trying to manage. "Metrics" are no replacement for IMHO the more effective management style of asking somebody with a clue how things are going, even if that means passing messages up and down a tree.
We had a new IT director show up a few years ago that came around to talk to everyone about their hopes and dreams and all the rest of it. Because he cared about us as people. Shortly after that the IT department shrunk by a third.
It's Friday. I took the night off. I will be VPNing in tomorrow to do a bunch of stuff. I have to go in on Sunday to do a bunch of other stuff I can't do remotely.
Fuck this shit.
Having had the delightful good fortune of being the 'it manager' of a 300-ish person company I will tell you that when you have 3 people, there are no meaningful metrics. Work load is too elastic and with a 10k user company, there won't be enough context in which to evaluate any metrics anyhow.
The questions are instead answered with common sense :
- Do my people have good work ethic?
- Do our solutions work, and are we encouraging best practice?
- Do we do what the business needs, in time to meet the businesses goals?
- Are the people paying the bills happy?
And that's all you can do. Trying to evaluate SLAs, ticket closures, and other non-statistics won't produce anything meaningful.
For what it's worth, after 20 years of this business most of the stats I've seen from big businesses (100K + employees) were worthless at best and downright misleading at worst, where notoriously bad employees were actually able to game the system by cherry picking easy-close tickets, refusing to give their names when customers with hard issues called, hanging up on users, and not creating tickets when issues required followup to avoid having long ticket times. Those people end up getting rewarded, which demoralizes everyone and encourages defection.
Unfortunately, in an absence of common sense and connection to the work, management refuses to admit their lack of competence of judgement and would rather tout metrics as a serious tool.
"No good deed goes unpunished"
1 IT person for each 100 people they are supporting. Make sure when entering the amount of time worked on tickets it equals at least 80% of the workday. Average production of a worker is actually about 70 to 75% of the day. The remaining time is typically spent on research, admin, etc. Include in your time on your tickets all the time spent researching a problem, waiting on hold for a SME/Vendor support, etc. Also ensure they know that if there are only 2 of you the probability of lack of support at any given time will go up. One person is sick and the other has a doctor appointment or goes to lunch, etc. there is zero support during that hour or couple hours. Hope this helps. Oh and let us know what your new job is. Merry Christmas!
The Thing is.