How Would You Benchmark an IT/IS Department?

ferretworks asks: "Our IS/IT department has been asked by our CEO to find a way to benchmark ourselves against IS/IT departments from other companies with similar technologies (none specific). This sounds like an innocent enough request, but diving into it has made me realize that this is, not necessarily undiscovered country, but a desolate one and rarely visited. So, my poll to the community is: In your Opinion, what is best way to benchmark an IS/IT department and what categories/sub categories would you base your judgment and ratings on?"

Comments

Talk about internal benefits first

[evileyetmc]

I think the first facet you should look for is uptime / accurate data rates (eg. 1% lost data etc). Beyond that, while being nearly a crapshoot, I think the satisfaction that the rest of your company is getting from your department is paramount - perhaps having a anonymous survey given company-wide to see how you're doing. Also, your upper managers may want to hear numbers such as ROI as well as IT costs as a percentage of revenue brought in...maybe even what revenue would be lost without the department.

Warning - opinionated opinion below.

[NeutronCowboy]

I do this for a living, so you'll probably hate me for some of my suggestions....

Number one thing I'd like to get out of the way is that the people who qualify this request as a waste of time are idiots who shouldn't be in an IT department. I've been to too many places where no one can tell me what they're running, where the servers are, what patch levels they are at, which machine is up or down and what they need to keep pace with growing demand (your business IS growing, right?). IT departments like these are the root cause for bloated IT initiatives, downed web servers and crappy customer service.

Number two is the realization that the IT department is in the business of the keeping the rest of the business up and running. This means two things: your responsibility is to keep the rest of the company happy and productive, and you get to charge them (system doesn't matter, as long as you keep track of what it means to service 200 requests a day for lost passwords) when you perform work to keep someones servers up, the mail flowing and the network lit.

Once you understand the place of IT in a business, metrics are easy to come by:
- how well you're doing is measured by how available your servers and your apps are.
- how efficient you are is measured by how much money you save the rest of the business in avoided downtimes and increased productivity.
- Bonus if you keep track of how long it takes to service requests and what their resolution was.
- Extra Bonus if you can do performance forecasting and figure out when you need to expand your infrastructure.

The performance of an IT department is NOT measured by any of the following:
- budgets of IT departments in other companies
- how many new products you've rolled out
- how many people you fired (or hired)

Anybody who suggets those needs to be smacked around and told not to speak in public anymore.

With that in mind, there are a million products to track your statistics. My company will be happy to quote you anything from a 4-figure to a 10-figure deal for this. There are open-source tools that will do similar things for free (though in my opinion, you get what you pay for). There are in-house and hosted answers to any of these questions. But the one thing you need to remember is that you need to know the answers to the CEOs questions about what the status of your machines and your people is. If you can tell him that your hardware uptime runs at 99.99% for mission-critical servers (the Oracle RAC that holds your financial information, for example) and your app availability is 99.7% during business hours, that it takes you 4 hours to respond to a priority 1 request and 3 days to a priority 3 request, and that based on current response time trends, you need to double the processing power and database space in 6 months, you're golden.

If you can't tell your CEO these things, you will be replaced by someone who can. Or even worse, your job will be outsourced to my company, and I get to work out these things while you're flipping burgers. Yeah, I'm being a jerk. That's because I'm flabbergasted at some of the comments (and their moderation) and how their authors still have a job. IT metrics are simple, and don't have to start with complex SLA measurements and other crap. You can start with a basic ping monitor of your critical servers, and go from there. But for heaven's sake, do something. You'll be a hero if you play it right.

Oh, and just to repeat - do not benchmark yourself against other companies. You don't have access to valid data, and you won't find an identical business. Instead, find out what your other departments need from you, and benchmark from there.