Ideal, and Actual, IT Performance Metrics?

An anonymous reader writes "Recently it was revealed that our company measures IT performance by the time it takes to close trouble tickets. I consider IT's primary goal to be as transparent to the user as possible, thus this metric was rather troubling to me. Shouldn't we be focused on reducing calls, rather than simply closing them quickly? My question is: How is your IT performance measured, and how do you think it should be measured?"

Comments

Sounds good to me.

[khasim]

For example, for every fax successfully sent via the fax server without IT intervention, the IT department gets one point.

For every fax that needs IT intervention to be sent, the IT department loses one point.

For every person who becomes aware of a problem with the fax server, the IT department loses one point. No more "heroics". The goal is to be as invisible as possible to the end users.

And similar items for every other server/service that IT supports. If nothing else, it will show exactly where the problems really are.

Re:Sounds good to me.

[RingDev]

I have been recently working on a system to do just this. We already had an audit system that some of our apps were using, but after generating a set of usage reports showing the volume of usage we managed to get some more buy in and it is now being used by almost all of our apps.

To go even further, we are trying to get access to the help desk's database so that we can generate reports comparing usage to tickets.

Metrics like these can still pose some risk for mis-interpretation. It'll definately be more useful to view them as a trend line with notations as to deployment dates and business factors that lead to peeks in errors per usage. But the over all trend should be a downward slope as the software matures.

I don't know if I would directly compare numbers from one app to another, but after a sufficient period of data collection, you should be able to identify goals in error per use performance and determine if one application is ahead or behind the curve.

The goal in my case is to try to negotiate the replacement of old VB6 applications that have higher error per usage rates than their modern cousins. But I won't have the data to back up my theory for a while yet :(

-Rick

Re:Sounds good to me.

[sumdumass]

I was thinking of a system similar to that. My solution came from actually being replaced because i was doing too much preventative maintenance. In my case, my replacement did no preventative maintenance and downtime were much longer plus his work load was much greater.

In order for it to be effective, you need to consider lost profit or down time potential too. I created some hypothetical numbers where the user is evaluated by costing the company 100 points a day and makes the company 300 points. so on a normal day, it would be 200 point average. I then rated the different applications and services they used based around how long they were in it and weight them appropriately. Something like the accounting app might be 40 because 30 percent of their normal workload is using it and they might need information from it to accomplish another 10 percent of their workload. Not sending faxes from the PC or IE not allowing them to shop for new shoes online was a 5. The IT ticket was 100 for anything done. And all this was divided between a normal shift of 8 hours to get an hourly rate.

Anyways, when something broke, lets say email and it effected 2 users which need it for 20 percent of their work. Suppose they were off line with it for 3 hours and it turned out that IT fixed it within an hour after being able to allocate resources to it (let's say someone sent a large email to these people which kept locking up their email clients and the solution was to copy the email out of their accounts, delete it inside it and allow them to open it externally with some other viewer). Lets also assume that an update would have fixed the email client's preview feature and it wouldn't have been locked up if updates were installed. Ok, so the costs would be the 200 x 2 employees divided by eight to get the per hour cost then multiplied by 3 hours, 200*2 /8 *3 for 150. the email was 20 percent of their work so 150*.20 drops it down to 30 points. It took IT one hour so now it's 130. If preventative maintenance could have discovered the issue with the preview and certain types of large emails and applied the patch before the user was interrupted and was able to do this for less then 130, then it benefits to do the maintenance. It's hard to justify and compare if the user isn't effected, but after they are, then it goes.

BTW, I picked an email client update because something like the scenario I described wouldn't be included in an automatic update from MS as it isn't a security risk, The PM would be looking through the non security related updates availible and applying them periodically as their organization's requirements dictate.

Re:Sliding Average

[Lumpy]

Problem is most PHB's CANT understand that.

They think that solving it faster = better. and if the number of calls goes down, you are doing something wrong.

we were unable to fight it at comcast as the idiot CTO demanded it. so I instructed my guys to put in a ticket for EVERYTHING and if it was a instant fix, open the ticket and close it in 1 minute.

Before I left my department got an award as our ticket accounting was 4X higher than the rest of the division and we had the lowest time to resolution. Problem is that Remedy sucks so it actually slowed down my guys having to enter all those useless tickets.

Yes it's technically fudging it, but the executive in charge was not listening to any of us so we skewed it to our favor.

Ideal and Actual

[sexconker]

Ideal:

I know about IT, having worked there for many years. In fact, I'm still working there. Keep up the good work, I know there's a lot of bullshit to put up with.

Actual:

I heard some buzzwords. When can we implement them in order to actualize our potential? Also, I'll need you to stay late and fix my computer. It's got some sort of virus or something I don't know.

(As you're fixing his machine, you see a note on his desk right next to the post-it with his passwords)

Hire grad student from India [x]
Get what's his name to train him. [ ]
Fire what's his name. [ ]
Synergize. [ ]

Customer Satisfaction Surveys.

[wonderboss]

Our competitors measure their performance by time to close tickets. They are consistently rated worst in support. We use surveys. Simple questions like: Was your problem resolved? Was it resolved promptly? We are consistently rated best in support.

IT should be focused on "customer service"

[hellfire]

Prior to my software company being bought out, my It department was focused on "customer service." This means that everyone in the company is treated like a customer. I personally work in our software support department and this made utter sense to me.

Under the new company, our new IT works for itself, and primarily is concerned with closing calls as quickly as possible, without regard for the quality of the information or assistance. They are concerned with reducing their own call load, but they don't try very hard, and they don't offer a lot of value over that. Any good customer service department is concerned with closing calls, but they want provide good quality service where each call is resolved as quickly as possible, but also as accurately as possible and leaving a good feeling with the customer. IT should be a resource utilitized to make the company more efficient and reduce costs, not a bunch of yahoos who fix broken PCs and then disappear back under their rock when they are finished.

In customer service, quantitative metrics are used to judge the department trends as a whole, and can be important, but even more important art qualitative measures, like surveys and feedback, example cases, and periodic reviews of every rep, team leader and supervisor. Did the rep do "The Right Thing" (tm) and how many times did they do that, and are they approaching doing the right thing 100% of the time? If a rep provided the user with the right answer, but all they did was email a timid accountant a 5 page document on setting up .NET properly just so the user can properly export his reports to an email to his boss, and then the rep closed the case and offered this less than technical person any real help, how service oriented is that, really?

Sometimes that means taking fewer cases per rep and leaving them open longer, if service improves dramatically.

Re:This is why the IT department is always cut fir

[Chirs]

I think you're stretching things a bit.

"How do you quantify the guy who spent all weekend fixing the server?" You look at the number of times it's happened and you figure out how much it would cost to get that level of service agreement from an outside vendor.

The accountants are much more likely to be asking questions like "how would the business be affected if we outsourced IT at a cost of X, thereby allowing us to save Y in salaries, at a cost of Z in reduced productivity due to longer resolution times".

There are cases where it really doesn't make sense for a shop to handle their own IT. On the other hand, there are definitely cases where it does.

Re:obvious

[bigpat]

But how do you measure the success rate of a problem you solved proactively, thus ensuring it never becomes a measurable problem?

That is simple: fewer number of calls or calls about a particular problem is the success rate of solving a problem proactively. Strictly speaking the IT Service desk shouldn't be solving problems, just getting customers back to the point where they can use the service again or work around some problem. Management and the level 2 engineers should be the ones solving the problems.

We have a problematic metric with password resets, they don't take very long but are the biggest amount of calls our help desk gets.

But password resets should be self service and at some point they are going to be. At which point our Service Desk metrics are going to get screwed up because suddenly we will have far fewer calls but the average time spent per call will go up. Depending on which metrics management has been caring about, this will either be understood as an overall improvement or will be misunderstood as some negative change in the quality of service.

It takes some management understanding to make sense of metrics, otherwise if you have management which is out of touch with operations then you are going to have problems.

You need accountability

[Colin Smith]

If a business service owner signs off then what is the problem? They are the ones getting fired when it all goes to shit.

Just make sure your change management board includes them, and finance as well. If you have a change management system you can even point to the change number and the requestor and say this guy caused N million doillars worth of bad press/whatever to the share price,

It isn't ITs job to say no, it's ITs job to explain the risks.

Re:No cnt++

[MushingBits]

[sales to IT] We need (something that is a huge security risk).
[IT to sales] No.
[sales to administration] waaaahhh.
[admin to IT] Do it.
[IT] Grumble grumble-

writes up reasons for reluctance with clear worst case scenarios, makes admin sign off and keeps multiple copies.

*does it*
[sales] yaaay!
[Admin] Damn IT.

Shit hits the fan, IT is blamed.

Paperwork is whipped out, meeting held with all parties simply to ask how this can be avoided in future, let PHBs dangle in their own rope without saying more

Needless to say there will be multiple later-rinse-repeat cycles until learning occurs.... but it usually does eventually.

Re:Exactly!

[Archangel Michael]

You missed the point. THE point is that they never really had a budget in the first place. They thought it could be done for nothing, by next Tuesday.

Much of what geeks do, and I see it a lot here on /. is that we tend to be DIYers. We see a high end server and say things like "I can build it for less". And people begin to expect that we can do things for less than to do it right, with engineered pieces.

Yes, we can build a off the shelf server to hold terabytes of storage for cheap. Too bad it is in a mini tower case instead of a rackmount chassis, and using SATA drives at 7200 RPM, instead of SAS drives at 10,000 (or 15,000), using onboard or cheap RAID rather than battery backed up RAID controller, etc and etc.

I can build two of those cheap ones for the cost of an expensive one. But that doesn't mean it is better.

There are three ways to build a system.

1) Cheap.
2) Expensive.
3) Right.

Right includes things like proper design, engineering, and build out, and planning for things that non-professionals would never think of. It is often more expensive that even #2. But in the long run the actual cost is less (time, effort, maintenance, performance, upgrades etc).

Doing things right is not easy, or else people would do things right the first time.