Slashdot Mirror
Ideal, and Actual, IT Performance Metrics?
An anonymous reader writes "Recently it was revealed that our company measures IT performance by the time it takes to close trouble tickets. I consider IT's primary goal to be as transparent to the user as possible, thus this metric was rather troubling to me. Shouldn't we be focused on reducing calls, rather than simply closing them quickly?
My question is: How is your IT performance measured, and how do you think it should be measured?"
I think that when the metric is to reduce the number of calls, the natural human tendency is to ignore calls, shift calls to other people, etc. to make it look like you're doing better when you're not.
So that's why most people look at your find versus fix ratio, the number of bugs you find versus the number you fix / the length of time it takes to fix them. It's not great to have zillions of issues, but you should always try to fix the issues as quickly as possible.
There's one metric that can capture everything:
Bits of Shannon entropy processed per hour.
My only political goal is to see to it that no political party achieves its goals.
Shouldn't we be focused on reducing calls, rather than simply closing them quickly?
We should be focussed on both.
My question is: How is your IT performance measured, and how do you think it should be measured?
ITIL principles are a great starting point.
Examples are using Key Performance Indicators (KPIs) such as at the bottom of this page and this page.
At my former employer, customers would call the national helpdesk, who were rated by their time on a call. Let me tell you, the type of customer service you get from that environment is crap. They would have the customer reboot their machine, and if that didn't work, they would escalate the call to a state level operations center that could dispatch technicians (where I worked). They were, for the most part, useless. They made the customers angry, and really served no purpose other than a filter.
The whoosh belongs to you. GP is a troll. Cut and paste troll slightly modified to appear on topic. You bit.
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
I remember it as being 10% of the people (whiners) take up 90% of the support time.
Bouncing customers is a good way to keep them from calling back -- grandma is much more likely to phone up 'lil Tim for computer advice if she knows the hotline tech is going to bounce her to ten different places; where I work, we get a good bit of troubleshooting work because the customers hate calling the hotlines provided by the manufacturer. Sadly, annoying your customers is a good way to keep them from calling back, and as long as your product is good enough people will still pay-up. E.g. I'm screwed into Suddenlink where I live. After being promised $85.01 TV/Net, I got a $100.00 bill because of hidden fees. Guess what -- I'm screwed into paying, because the only alternative (Cox) was bought out by Suddenlink.
"Sorrow is better than laughter, for by sadness of face the heart is made glad." [Ecclesiastes 7:3]
This, too. It's a sign of a bad management structure. Useless statistics resulting in managers getting bonuses is also a great flag for "cost saving" measures that can be taken. Eliminate the managers that fit this category with no other redeeming qualities, and you'll save a boat-load of cash.
I don't know about you, but my servers run on the power of cotton candy and happy thoughts. -Anonymous Coward
42.42
Just need to upgrade your network and disk I/O. I get 14, easy. :-)
Seriously, though...I think the submitter is right. You should be trying to reduce the total number of tickets, but then you've got to be wary of trying to improve your performance score by saying "That's too small an issue to be opening a ticket for. I'll just ignore it/fix it on my lunch break/tell the user to bugger off."
I don't think any single metric is useful. Probably something like:
average # of tickets open X 2 +
average hours from open to close X 5 +
# of security breaches in past year X 100 +
# of times with no open tickets in past week X 1 =
your IT performance score. Obviously, lower is better. Change the weightings to your preference, and if you'd rather a higher number be better, divide 10 by your result.
Surely somebody's got some formula like this already. I wouldn't be surprised if there's some obscure standard somewhere that nobody uses because it'll make management look bad......
"City hall" in German is "Rathaus" Kinda explains a few things......
Sad to say but the more invisible the IT department is, the harder it is for them to get funding for new projects.
Pain is merely failure leaving the body
that your organization has made your job measurable. It does not matter what they measure your performance by, as long as it is something tangible.
So, you get payed by how many tickets you managed to close in a month. Fine. So, you close as many as you can in a month, resulting in lower quality of each problem fix, resulting in more tickets posted and assigned to you, resulting in you having ensured that next month you have enough tickets as well.
This can go on indefinitely, or your wise superiors might decide to measure your work somehow else.
As the island of our knowledge grows, so does the shore of our ignorance.
It's been my experience that most "no's" come from bad or lazy employees. Most good ones will at least explain themselves and TRY to help (even if they're underfunded).
SJW: Someone who has run out of real oppression, and has to fake it.
Basically what the parent poster is pointing out is that absolutely any metric that management can come up with can be fooled.
For instance, now that you know that "time to close the ticket" is the metric, you can always close the ticket and then go work on the problem rather than the other way around. Or if it's an average time to close, you can get a pal who works in another department to open up a bunch of phony tickets for you to resolve quickly.
To quote extensively from Joel Spolsky:
Robert Austin, in his book Measuring and Managing Performance in Organizations, says there are two phases when you introduce new performance metrics. At first, you actually get what you wanted, because nobody has figured out how to cheat. In the second phase, you actually get something worse, as everyone figures out the trick to maximizing the thing that you're measuring, even at the cost of ruining the company.
Worse, Econ 101 managers think that they can somehow avoid this situation just by tweaking the metrics. Dr. Austin's conclusion is that you just can't. It never works. No matter how much you try to adjust the metrics to reflect what you think you want, it always backfires.
I am officially gone from
The metric is valid when looking at the model where you have INCIDENT MANAGEMENT versus PROBLEM MANAGEMENT.
That first line of call-in is about making sure the human caller gets to a human as quickly as possible. Within 15 minutes flipping that call should be done OR escalated to PROBLEM MANAGEMENT. The reasoning is while you are talking with somone there is another caller trying to get a hold of someone.
Turn Around time is relevant to INCIDENT MANAGEMENT versus PROBLEM MANAGEMENT. The problem is when there is not a clear difference between incident and problem management groups.
Three metrics that are needed:
Caller Hold Time
Call Turn Over Time
Ticket Resolve Time
Hold time is the customer's experience in getting thier problem addressed. Not neccessarily resolved, but addressed.
Call Turn Over Time is key on hinting at the type of problems. If 90% of your calls are resolved in under 5 minutes, you more then likely have training issues. If 50% are resolved in the first 5 minutes and 25% are escalated to PROBLEM MANAGEMENT then you may have a process failure or technical issue.
Ticket resolve time is over all the volume of touble you have in regards to the severity of the problem. Logging 1200 hours a week of SEV1 tickets tells of serious problems verus 1200 hours a week of SEV3 or 4 problems.
Mostly management uses those metric for determining what areas need to be addressed. They are not performance metrics on their own, in fact useless for measuring performance. You would need at least the % of tickets escalated to even start determining performance.
This of couse is under the assumption of a split between INCIDENT and PROBLEM management.
-=[ Who Is John Galt? ]=-
Here's the short version: "Fast, cheap, accurate. Pick two."
I only post comments when someone on the internet is wrong.
I thought IT got paid for the number of times they said 'No' to us during the day.
go figure.
Most of my no's come from protecting the user from themselves.
The last company I worked at, I was the manager in a department of 130 people. When I left, the company had deployed server #100,000 to its datacenter.
I understand CM well, but (and I hope you're exaggerating), if your IT team requires backout plans for "Install OS" as a deployment step, then someone needs to step in and reign in the bureaucrats.
Your COLO probably bought RACKS of identical servers, and for every rack has one hot standby. THAT is what I would do if I were running a COLO.
When running a large operation it is easy to bury an extra standby server in with 24 or 48 other servers (add 1/24 or 1/48 to the cost of each server). However when you order the server as a one off, you can't double the cost just to have a hot spare.
Here is an example.
Let us assume you're building a server, and it cost $5000. Are you gonna cry over the $210 needed for the spare ? Will you cry over $5000 for the spare?
The difference is that when you have MANY it is much easier to cover the spare.
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.