Slashdot Mirror
System Admin's Unit of Production?
RailGunSally writes "I am a (strictly technical) member of a large *nix systems admin team at a Fortune 150. Our new IT Management Overlord is a hardcore bean-counter from hell. We in the trenches have been tasked with providing 'metrics' on absolutely everything from system utilization to paper clip recycling. Of course, measuring productivity is right up there at the top of the list. We're stumped as to a definition of the basic unit of productivity for a *nix admin. There is a school of thought in our group that holds that if the PHBs are simple enough to want to operate purely from pie charts and spreadsheets, then we should just graph some output from /dev/random and have done with it. I personally love the idea, but I feel the need for due diligence, so I put the question to the Slashdot community: How does one reasonably quantify admin productivity?"
of Jolt Cola consumed.
They say the first thing to go is your penis. Well, it's either that or your brain. I forget which...
How many tickets answered per day? Completed per day? /dev/random is probably the most elegant though
to know
Write a kernel module to log the number of keystrokes.
Run away, as fast as you can...
The best sys admins are the ones you never notice. If the productive workers in a company never see or need to talk to a sys admin it's been a productive day for the admins.
Why worry? If you've got enough time to post stories to Slashdot you're clearly very efficient.
Photos.
Percentage of tickets completed. Remember, 0/0 = 100% - just reverse the math to check that one.
"How does one reasonably quantify admin productivity?""
If no one in the building but HR and your line report need to know your name, you're doing your job...
Other than that, it would be like a trash collector counting how many cans he emptied during the day or a wildfire firefighter how many burning bushes he chopped. If there weren't any fires or trash these people wouldn't be needed, would they?
You can't quantify SA productivity.
It's easy to quantify /my/ productivity as a support tech (at the U of CA) in number of tickets resolved per shift. But sysadmins have a number of duties which they are performing /continuously/, so how can you quantify that?
.:Semper Absurda:.
Since the real proof of actual productivity for network admins is negative: nothing goes wrong (no trouble tickets). Also, the PHB will get their wish: No one to pay is infinite productivity (measured as output per $ spent).
Unit of Productivity = 1 / (hours of down time)
They are paying you to keep bad things from happening.
Do uptime. Unless your team has serious problems, those numbers should always look good. If you do any sort of work in response to in-house or outside tech support requests, you can measure how long it takes to resolve issues.
A) Nagging emails
B) Logging every OS update
C) Supervising Patch Thursdays
D) recording the percentage increase in spam email intercepted (This is your business metrics friend, since that number will never go down)
E) Number of meetintgs with employees about improper email use.
F) number of Company-wide software-license-compliance surveys, and number of improper installations detected.
G) total number of top executive emails logged, with copies sent to several geographically distinct locations.
If they want metrics, give 'em metrics. And let them know that metrics will only encourage you to be more of an a$$hole.
Does he want to reduce the departments employee count? Or does he want to sustain it employee count?
Mayeb he just needs to hire more and task to them, the metrics goals.
Uptime.
The sooner you find a job that doesn't suck so much, the more productive you probably were.
Brackets contain world's first nanosig, highly magnified:[.]
I suspect the closest model with math behind it is a futures contract.
:P
The things that a good sysadmin is supposed to do is make sure that all of the things that would threaten the relevant simple metrics -- capacity, uptime, etc -- are taken care of ahead of time.
Clearly, it is more desirable to add servers a month or two before they are needed instead of after the server farm becomes unusable.
So what you want, as your metric, is to track the future value of capacity and the future value of uptime.
Got a time machine? The best we've managed is Black-Scholes and that's not going to work for this situation.
Gentoo Sucks
Those are patch TUESDAYS.
(Now, will I get flamebait for insulting myself?)
uptime, for various properties of the system.
The basic unit of measure for any good admin is, of course, slack. You never notice when an admin is doing a good job. You only notice when they're not.
A steaming cup of soykaf would be real wiz right now.
Base your (metric)work on how long things are NOT broken.
i.e. Well I was 92% effective yesterday because I had to replace a switch.
"The price good men pay for indifference to public affairs is to be ruled by evil men." ~Plato (427-347 BC)
You aren't building automobiles or painting teapots. You are a support function and not a line function.
You should have business plan objectives. These things are usually annual; there can be longer strategic objectives. If the person who set these things did it right, they should be measurable.
What I'm trying to say is, if you're banging your head against the wall trying to figure out how your performance should be measured, your higher up didn't set your objectives correctly.
This doesn't apply anywhere and everywhere. When the organization is in the business of IT itself, you might be measured differently since you'd then be contributing directly to the organization's core business. But from the description provided, it sounds like you're not.
The Banjo Players Must Die!
The standard unit for quantifying productivity in IT is generally bottles/cans of Mountain Dew. This varies from "cups of coffee" the standard metric used in many other fields, but you can easily convert between the two, so it shouldn't be any trouble for your manager to share your productivity figures to his bosses in a manner they are more familiar with. To convert cans to cups of coffee, multiply by two fifths. To convert bottles to cups of coffee, multiply by two thirds.
To do system performance, just take pages/data/e-Mail/whatever served from a server, and divide it by the operational cost of the server.
But how do you measure the value of the administrator?
I guess I would take the value of what the system would do by itself, if it was just "left running", and divide it by that cost. (You would end up with less cost, but also less productivity, of course).
And then, take it with say, a bare bones set up, with only a few, poorly trained administrators, and show what the cost per page/e-Mail delivery would be.
And then show your cost as it is now, with you and your sterling team of administrators.
Hopefully I didn't put any [] around my words.
... try not to think of your Overlord's salary being ten times yours. Plus bonuses.
I suggest you take the task very seriously and try to find indexes on the internet from Gartner, Accenture, Cap Gemini, KMPG, or other major analyst/business consulting companies. Just using their names and logos in your reports/suggestions will get you a long way!
:-)
After you have collected some reports, try to think of ways in which you can demonstrate your effectiveness. If you are in fact not very efficient, the boss is right in wanting that information - right? If you are in fact very efficient, you should have no problem with him discovering that - right?
You could make performance-indexes for the amount of users you serve (for each type of service like mail, security, infrastructure, etc.) and for the amount of servers you maintain. With any luck you should be able to prove that you serve a high numbers of users, a large amount of data, and maintain a high number of servers... compared to whatever general indexes you dug out of the reports we started out with.
- Jesper
My security clearance is so high I have to kill myself if I remember I have it...
(Out of mod points.)
You are absolutely correct imho.
Now, to explain this to a PHB... give them some downtime first ?
Putting beancounters in IT mgmt can be a prelude to axing of IT jobs. Tread lightly!
As a technical member of a highly specialized profession, you alone may be able to quantify your output, particularly if you work independently on projects. Measuring the productivity of knowledge workers for the purposes of assessing cost and improving efficiency has been a management problem since Peter Drucker began writing about knowledge workers and the knowledge economy in the 1950's. Your counter of beans should know that if he or she has had any formal management training or has picked up a book. This is also one of the problems that has plagued the information technology field due the difficulty in determining return-on-investment and setting budgets accordingly. Your manager has a challenge ahead if he or she is determined to attach a prod-o-meter to your seat. One of the methods commonly used is through trouble-call tracking systems, and measuring statistics such as calls closed and average time to closure. It is unreasonable at best and destructive at worst to assess performance of project-oriented workers using these metrics. YMMV.
It's a bit of a nuisance, but maintaining activity logs is probably the best defence. If you're really good, there won't be much trouble ticket activity which could lead a PHB to think you're not doing anything and he can score points by getting rid of staff.
There are lots of automated tools for tracking activity, but the more important part will be explaining the need for time to think about how to solve nascent problems before they manifest themselves as user calls.
Good luck!
rhb
You can't. So prepare to teach Wipro how to do your job.
I work for a Fortune 15 pharmaceuticals company. You don't keep application management and support around for the good times.
You keep them around for the bad times and thats a growing pain all IT dept's are facing as we are all pushed under the magnifying glass. Gone are the days of "what? IT? throw money at it!!!" out of fear. Here are the days of "global IT services". If they can find a way to do it cheaper, they will and they will do it at the cost of quality.
Still, the company I work for is slowly beginning to realize the importance of having someone who speaks passable English available during a crisis situation and with any luck your company will realize that before shipping your job off to Bangalore.
We seldom regret saying too little but often regret saying too much.
Unit of Productivity = 1 / (hours of down time)
Bad choice. In a well run shop, you'd get a "division by zero" error.
Microsoft is to software what Budweiser is to beer.
# of furious users outside the manager's door after you leave.
Always be helpful and make co-operative proposals. Suggest that users who lose passwords should be docked pay according to the time to fix, thus ensuring that user incompetence is not perceived as an IT cost (I know this isn't *nix admin, but it's part of the general pattern).
Produce impressive graphs showing the tradeoff between increasing the numbers of servers per admin, and the expected downtime, and suggest that increased downtime is a small price to pay for increased server/admin ratio.
I'm sure you can see the picture developing. Every proposal you make to measure productivity needs to look superficially good but have a huge cost and a huge risk. And the best thing is, you don't need to lie. Because that is the nature of reality.
True story. Many years ago I was asked to investigate a shop floor wide downtime monitoring system. As an experiment, we tried paper logging of downtime events to see what the expected traffic would be, what events needed to be handled, probable traffic etc. The paper logging showed that an entire floor of machinery had three quarters of its downtime attributed to a single design fault in the machines. The maintenance people knew this, but were keeping quiet because of all the overtime they were earning. Getting the manufacturer to fix the fault was far cheaper and quicker than putting in the monitoring system. Moral: in parallel find a few really good cost reduction proposals. You are bound to have some if you are any good. Then wave them under the nose of the bean counter in parallel with the other stuff.
Pining for the fjords
The standard unit of measure is # servers supported per administrator. Now that doesn't help identify whether a specific admin is any good, but it does provide a comparison cost.
firefighters spend a lot of their time these days preventing fires, doing stuff like controlled burns.
maybe you cant measure 'productivity', but at some point you have to make a budget of how many people you need to hire for the season, and to do that, you have to know how many people it takes to do certain activities in a given amount of time.
whicih means you need to measure those things.
---
Systems Administration falls into several categories.
:)
Projects, Service requests, Patching, and user satisfaction are a few.
Once you have an idea of what you do, define some SLAs with your customers and the metrics are easy from there.
Now compare your defined SLAs to the following.
Metrics:
Time to ticket close?
Were the requesters satisfied?
Projects completed in the expected time?
Resource allocation is at what percentage?
Don't forget to measure your ongoing education and professional development. How much should you get, are you getting it?
Patch schedule being met?
Availability metrics.
Resource loads on the systems are easy and provide management nice graphs, plus they can be automated.
My systems roll all this information up and e-mail it for me.
While none of this is really important to us, the management teams operate almost entirely on this data. Take this as an opportunity. In some shops I've worked, management defines the metrics and they mostly are irrelevant. In your case it seems you have the rope to hang yourself so take care to present the data that is important and will help you meet your goals. As always, a good admin will automate the task but not tell anyone.
--russ
What is important with CxO's is that they need to justify their spendings. So if they have a unit of sys-admins, which do something, but they do not know what, they will see it as a potential to cut your budget, and save money.
So you have to show him some figures about the size of your systems. Are you running important infrastructure for the company? If so you should give him estimates of:
Remember that if you give him sensible info about these things, you might get a better budget and more arm-space.
Just check the statistics on Spider Solitaire. Let's see you work your Six Sigma black belt on that, bitch.
the best admin can (theoretically) make the servers run by themselves forever. such an admin will have no daily work to measure productivity with.
This is my sig. There are thousands more, but this one is mine.
Just make a Pie Chart and have 66% of the pie say "Time Wasted figuring out how productive we are."
...that you didn't have to fix this week.
You should view this as an opportunity to be able to rationalize extra help in the future! Let me give you an example: I'm an industrial engineer for a large 3PL. I put together labor models for warehousing opps on a day to day basis, and I do it by breaking down individual activities that the team will perform (stageing pallets, filling out purchase orders, etc.). I then assign hourly productivity rates to them (how many units I handle per hour), and assume that my employees will really only work about 6.75 hours per day, after breaks and interaction with others, using the restroom, etcetera. Now, I used to be a *nix admin for a much smaller company back in the day, and most of my requests would come through email. I imagine you guys probably get your requests to perform different tasks through some sort of fancy IT system if you work for a larger company. So, ask yourself the major categories that these tasks break down into. For example, if there are two main categories of work you do, say, 1.) fixing stupid errors and 2.) setting up things for people to break, you estimate how long it takes to handle a request from each of these major categories, and translate that into how many you could handle a week (or other time period), keeping in mind you only have about 6.75 productive hours a day to work. Say a single employee could handle 10 stupid errors or 5 set ups per week, assuming they were doing nothing else (be conservative when setting these up so you look good when you surpass the productivity figures). Now, when you are getting in 40 requests for stupid errors per week, and you only have 3 team members, you have something to bitch about! Your boss is basically telling you that he doesn't know what you really do, and so you can pretty much define how your group should be operating - HAVE FUN WITH IT!!
"How does one reasonably quantify admin productivity?"
Go see "Doctor Summeroff"
High blood pressure from dealing with stupid people should do it. A couple of months might be enough.
--
BMO
I sense in your post that you are taken aback by this request to account for your time. Your response is quite common, though I would guess a little reactive. Many it departments are implementing project management programs as prescribed by ITIL standards. The foundation of these project management programs is having a clear understanding of how much time your resources are spending on tasks and projects. The purpose of collecting this information is to create a baseline utilization rate for the teams you manage. Once you have collected this baseline, you can observe for example that sysadmin A spends 60% of his time on tickets, 20% of his time on strategic projects, and 20% of his time on general office administration tasks. Management can also observe (if you code your hours) when Sysadmin B is spending 60% of time on tickets, 60% on projects and 20% on general administration. If you notice, this equals 140% utilization (or a 60 hour work week). This information is normally used (by project managers) to then do a resource leveling. This is when you take say 1000 man hours necessary to complete a new project (say migrating from Solaris to Redhat) and distribute it to engineers which have available cycles, or if there are no available engineers, to tell upper management that either A - the project needs to be slid out into the future when people have more time, or B - Management needs to hire more engineers to complete the project. If you read between the lines in this, there is a gotcha. The sysadmin that improperly codes his time (fills out 40 hours of tickets.. at the end of the week, instead of accurately tracking his time) can short change himself. Even worse, if you refuse to code your time, management has not metrics to request hiring of new engineers, and you get stuck working nights and weekends because management lacks proper information about your time.
I suggest that you hop on the wagon of tracking your time. If you read between the lines you will find that this is your chance to prove to management how hard you actually are working. You can also use this combined with other data to argue for pay increases (point out that your ratio of tickets closed to time spent is much better then your associates etc).
Colin McNamara
CCIE #18233
Don't forget to make (honest) suggestions on how to improve efficiency. Even though the new boss might not go for your suggestions, it will prove to him that you are a valuable worker who is always seeking the most cost-efficient and optimum solution.
:-)
Could you perhaps benefit from server virtualization which could potentially consolidate a number of servers - thereby reducing power and cooling costs? Or would it be possible to eliminate some niche systems which have high maintenance costs and which are really not used? Could key systems benefit greatly from upgrades even though the upgrade is somewhat costly? (new products are presumably better/smarter). What was (in your opinion) the five biggest neglects of his predecessor?
Think of this guy as a career opportunity - instead of a problem you have to work around.
- Jesper
My security clearance is so high I have to kill myself if I remember I have it...
What you need to do is contact some other F150 companies and ask their senior IT admins/CTOs how they measure productivity. I work for a major investment firm and we have metrics for everything we do (even though we're private) because of two primary reasons:
1. its how you improve, and
2. its what our competitors do too.
Its that simple.
pi=sigma{n:0-infinity}[(1/16)^n][(4/(8n+1))-(2/(8n +4))-(1/ (8n+5))-(1/(8n+6))]
A bean counter wants to know if the admin staff is "worth it." You can quantify some of the value of the admin staff. First, downtime. How much does that admin staff save the company by fixing problems? plot downtime & lost revenue or wasted time vs. admin staff hours spent preventing and repairing such events. The idea here is that an ounce of prevention is worth a pound of cure. Backups, patches, log files... everything. These activities prevent or reduce company losses due to downtime or similar failures.
Second, calculate lost productivity in the company while waiting for admin staff to carry out a requested task vs. the number of admin hours it takes to accomplish the task. The idea here is that reducing the admin staff head count will increase the time others have to wait for admin tasks to be completed.
The answer is easy enough: The number of pirated DragonBallZ/Prison Break/Lost/Hero episodes that a sys admin has downloaded and watched during working hours. This should be measured with a sliding windows of three weeks. Major migrations and upgrades should be taken as bonus points.
Assume for a second you had a perfect server farm. Its always up, backups are made, users are added and removed, etc. While we are at it, assume you have a staff of say two admins per shift, 24x7. That's at least 8 admins, probably more to cover holidays, vacation, etc. In this case, their productivity is zero, they have nothing to do. In reality, they are working their tails off, and deserve a nice bonus. So tell the PHB that productivity is not important, its problems. Its uptime, transactions delivered, average delay on transactions, etc. Get the Users to define what the 'requirements' are, and have the sysadmins deliver it. That is the measure of what is important.
RailGunSally wrote:
>We in the trenches have been tasked with providing 'metrics' on absolutely everything from system utilization to paper clip recycling.
This pretty much says it all; your manager wants you to do HIS job. Shouldn't he develop his own metrics? He can ask you for ideas but he should do the work himself. As for metrics, I'd suggest downtime percentages for each machine. If the services are up and running and the machines are online providing service then that should be metrics enough.
Codifex Maximus ~ In search of... a shorter sig.
You are support. You measure work in tickets. Everything you do should be in a ticket. If you do something on your own, open a ticket and assign it to yourself. The objective measurement that tickets provide is man-hours per task, but otherwise they're a relative measure, as in "how many tickets opened this week vs last". You don't have to educate them as to what it means, you just give them a report on tickets, and let them figure it out from there.
Seriously, this should be a no-brainer to anyone in IT. Anyone giving you "creative" suggestions on slashdot has probably never really even done the job, or is just showing off their geeky cleverness.
If you find yourself doing non-ticketable items because they're larger scope projects, then work your change management system into the metrics too. You do have one, don't you? If you don't, then they'll be pleased as punch with one you roll together yourself.
They consider you only as good as your last mistake. The bosses don't want to know what goes on "under the hood". It just has to work. Anything less than 100% uptime is considered a failure in their eyes.
What?
Keep track of uptime. Are the systems only down for scheduled maintenance? If they are down outside of scheduled maintenance windows, what is the percentage? Was it hardware or software or a mix (old firmware with updated driver requiring newer firmware), etc...
Was the outage extended due to vendor timing? if so, maybe stock of typical spare components should be maintained to shorten the window.
Typical maintenance like adding/deleting/unlocking user accounts, resetting passwords, printer maintenance, disk admin should be a small part of an admins day. The rest should be keeping an eye out on the real world looking for potential problems like security vulnerabilities, patches, planning the next updates / upgrades.
Tell the bean counters that their demands to quantify everything will only reduce uptime and complicate matters to where you spend more time doing paperwork than you do managing systems. If they can't understand that, it's time to go elsewhere. Be sure to tell the bean counter that they'll be lucky to find anyone talented to work under their regime.
I've seen systems that went from 20% loaded to always overloaded because of the number of *accounting* applications, programs and monitoring solutions that were *demanded* by the bean counters. After a user and business unit rebellion, the *fluff* was removed, as was the bean counter. This left the systems running in a state where the end users could do their work, and the business units had satisfied customers.
Who is general failure, and why is he reading my hard drive?
In my office we are judged by system and application uptimes and average trouble ticket resolution times.
"All those moments, will be lost in time...like tears in rain..."
and thats it.
a sysadmin may rear a big belly and sit all day long for years, yet s/he can avert compromise of the system just one day and save even a fortune 50 company from total chop busting - like 100.000 entries of customer records being stolen.
therefore forward your phbs here, and make them read these and learn that one cant direct an i.t. unit without being an i.t. person. its a totally different, magical world.
Read radical news here
Your first core metric should be uptime. Follow that with turn around time for fixing failures. Add in turn around time for setting up new servers/services and you should have enough to keep the PHB happy.
PS: If that's not enough - and if you think it's to your advantage - add in 'customer satisfaction'. Send out user surveys to evaluate their opinions of the services you are responsible for. Naturally, this is a good idea only if you think you'll get back decent results.
There are no other metrics that matter.
Metrics are nothing without insight. If he's totally in the dark about what might be good metrics to collect in your particular setup, then where will he get the insight into how to interpret them?
Hell, he probably just wants to take your list of metrics with him on the occasion of his next job hop, and claim to be God's guru of IT metrics and that his salary be paid in platinum and slave girls. Don't give him the satisfaction.
Its a simple equation.
e * employees_affected)) / number_of_hours_per_period.
/400 = 396/400 = 99% productive
productive_hours_per_period = ((number_of_hours_per_period * number of employees) - (number_of_hours_computers_are_down_or_unavailabl
10 employees, a full week, only one person looses computer access for 4 hours
productive hours = (40*10) - (4*1) / 40 = 400-4
perfectly reasonable
I reject your reality
So you work for Countrywide?
There are three aspects to system administration: Making sure everything keeps working, expansion and support.
For maintenance you'd want to look at simple downtime measurements. Then you can talk about schedule/unscheduled, etc. If you already have monitoring systems in place then this is relatively simple.
For expansion, I suggest you include any new expansion as a small "project", with a due date. Then you report %projects completed on time. The danger is having your boss then require a 10 page proposal, approval from 5 different people and a budget estimate, etc. But on most expansions that require nontrivial hardware you probably do at least the budget anyway.
For support (creating new users, fixing things, help desk stuff) you'd want to look at tickets/day and to use satisfaction surveys (automatically emailed?)
In fact, you might want to consider surveys as a regular tool. If you can show that most people consider the servers to be stable, then that's something. Of course, from the self-selected nature of these surveys, most people who fill them will have something bad to say about at least one aspect, so you have to keep that in mind
Most of these things are a good idea to record anyway, even if your boss isn't a bean-counter. Looking at trends and setting reasonable goals for your team is a reasonable way to detect problems and encourage improvement.
Here are some things you can track:
- obsolete accounts archived/deleted
- phone passwords changed
- exit interviews logged
You and your follow IT folks are in an uncomfortable situation, and you should admit that to yourself.
And the typical reaction of someone placed in a situation like this is to be as passive-agressive as possible.
However, I would reccomend a different approach.
There are two reasons why this beancounter wants you to quantify everything with numbers.
1) They don't know jack about how IT works,
2) They want to be able to divide those numbers into dollars, to get ratios and to figure out where adding dollars improves performance.
Arm yourself wisely. Read a book of TQM (Total Quality Management) that deals with software development or technical management.
You are going to have to understand a little bit of where they are coming from.
Second, tell the manager that you would be HAPPY to provide statistics (even if this isn't exactly true), but this doesn't come for free. If you're going to do this right, you need to be able to dedicate time and $$$ for your team to meet and tackle this problem, come up with a plan, and do the number crunching needed. This is important for them to understand, so you'll probably have to say it several times in several meetings.
Then get yourself to work on the stats. There's no single bullet, but understand you'll have both quantitative stats (uptime, number of tickets, number of tickets resolved/unresolved, gigs of network storage, bandwidth used, access time) and qualitative stats that you may need to assess with a survey of general or helpdesk users (happiness, responsiveness, satisfaction with IT tasks, network and application functions, etc.)
Provide the requested stats on a regular basis, and understand yourself how and why they fluctuate. And, this is the IMPORTANT part, use the stats yourself when debating with your boss over where to allocate resources and dollars. If they are going to make you collect them and are going to judge you on them, they need to involve you in decisions that will have a direct relation to the stats.
Good Luck.
As a former futures & options trader and current programmer & occasional admin:
That's the most nonsensical thing I've read this week & I've spent a bunch of time on reading comments on reddit.
It's not even wrong. Black-Scholes is used for option valuation, not futures. WTF either has to do with server metrics is beyond me.
--Pete
Figure out how much downtime costs, and how much downtime you prevent due to increased numbers of people, or special skill sets. If your webserver goes down, will it potentially hurt your business or not? Are your internal servers more important? Do printers really need 99% uptime? Everything gets weighted.
Overall, it seems like a pain in the ass to figure out, but might be quite useful in the future to justify spending for new equipment, software, or support, or even your own employment.
According to the BOFH, wouldn't that be LARTs per Subnet?
-- All That's Evil in the Geek Space
Simple.
:-)
Use a ticket system. I help customers setup RT.
(http://bestpractical.com/rt/)
How does that help?
Well, it allows coordination within a group, or shifted workgroups to identify and track changes in systems, and the status on certain things.
For example, did you change the iptables entries on the router for the primary hub in the network?
You then write a RT ticket entry on why, and what you changed.
Then everyone in your group gets a copy.
That way if something stoped working, or you made a typo people know "Ok, things stopped working, what has changed in the past 8 hours?"
You can use RT to track stuff like that.
If your a "Bean Counter from Hell", you can use it to track labor, and what was done over a given period of time and why.
At the end of the year, you can do some pretty interesting reviews of the what types of jobs were most in demand, what stuff likes to break all the time. (i.e. MMmmm...lots of windows problems on the desktops/servers. I think we should probably put more Linux boxes in this year to reduce our workload.)
Give it a try!
-Hack
Got Geometrodynamics? Awe, too hard to figure out? Too bad.
produce a metric that proves that when you're doing your best job everything's running smoothly and you're sitting on your hands doing nothing ... perhaps an inverse correlation to 'hours spent on /. or playing warcraft'
Productivity for a sysadmin is almost impossible to measure. Whatever metrics you come up with are likely to only capture very specific types of workflows or problems: when the set of issues changes (as it most certainly will), the metrics become useless. It's easy to measure the productivity of a knowledge worker, a little harder that of a sales person, but still doable. But for a knowledge worker like a sysadmin, it is impossible.
You should convince your boss that you will not measure productivity; if that fails, start looking for a new job.
Here's what's gonna happen if you do come up with some productivity metrics. Your boss will probably give incentives (bonus) or disincentives (lose job) based on the productivity. Things will be dandy for a bit until the problems you deal with start changing in nature, and your productivity metrics will start going down. At this point, you will have a choice between actually doing what's good for the company or trying to maintain your productivity metric at the expense of the company. With a boss like yours, the former spells bonus loss or job loss. Do you want to be in this situation? If not, don't measure productivity.
If you leave, find another job (that likely pays more anyway), and they have to replace you, they'll figure out pretty quickly what metrics they should have used to measure your productivity. And you'll be in a job that doesn't try and manage knowledge workers the same way as unskilled labor.
Here are some of the more basic formulas:
;'
Hours spent solving user problems per week:
% lookupd -q user | egrep '^name:' | awk '{print $2}' | wc -l | perl -e 'print 1.5 *
Hours spent putting up with useless management requests:
% find ~ -name '*.doc' -o -name '*.ppt' -o -name '*.xls' | wc -l
Hours spent reading news while at work:
% grep -i "href=" ~/.mozilla/firefox/default.isu/bookmarks.html | wc -l
"Microsoft killed my company, I hold a personal grudge. I don't use Microsoft products and neither should you."-JWZ
Hours Worked Fixing Problems divided by Hours Worked Doing Routine Work
The lower the number, the more efficient the sys admin is. A good sys admin doesn't have to do anything, because everything is already set up and working. If the admin is constantly fixing servers, bringing them up, restoring data from backups, etc., etc., etc., then he isn't doing his job. If the majority of his day is spent sleeping in his chair and responding to the occasional email and things are running smoothly, then you can't ask for anything more.
Love sees no species.
You can't measure 'productivity'. Managers with this style of management are not worth working for. Find another job.
Clearly this person is an idiot, and whatever scheme they use to measure productivity will only result in reduced productivity.
It is your manager's job to measure productivity empirically. If he can't do that, he's not fit for his job. Your manager should be sufficiently involved to determine if you are productive. If he's not, he's not productive, and should be fired.
The whole culture of targets and 'management science' is ideologically bankrupt, and practiced only by morons.
A number of things:
/. to ask, then might I humbly suggest that you have reached the holy grail of systems admin - you have succeeded in making yourself redundant.
1. Problems solved. That's easy - but unless you're constantly firefighting (which you shouldn't be anyway) you'll have to be disciplined and log everything, including things that you've noticed but haven't yet become an issue. Which leads nicely onto the next one:
2. Problems avoided through preventative maintenance.
3. System uptime.
Of course, problems don't all take the same legnth of time to solve. So you might want to include an estimate of "number of hours problem is likely to require" in there as well, to demonstrate that you aren't sitting around doing nothing all day.
But if you're so lost as to how to do this that you have to post to
Download a freeware web-poll product that will allow you to ask all users in the company (or at least your geographical location) which systems are "business critical" and to what extent they would be able to service their customers if they didn't have them. Example: "How many sales orders would you get, Mr. Sales Guy, if you didn't have e-mail?" or perhaps "Mr. Customer Tech, how many support incidents could you service without the main web-based knowledge database?"
:-)
Divide the numbers they give you with the total sales/incidents, and convert the result into a percentage of the company's total annual revenue or turnover. Compare the result with the IT spendings.
That should get his attention.
- Jesper
My security clearance is so high I have to kill myself if I remember I have it...
but for a boss like that the only metric s/he deserves is "copies of resume circulated per week". Of course this does your boss no good until you turn in notice. That's the idea.
Simple answer is that you don't. Productivity in terms of IT and related fields has become a dirty little word but more than that it is a business term, not technical. If you aren't a director or higher in title, and your duties don't include justifying expenses and planning resources for solutions, then it isn't really your realm to measure something like productivity. If this guy has an MBA or similar qualifications, it is he who should know how to measure productivity. But alas the word productivity has become corrupted by half-assed business journalists trying to write articles about over all productivity and how your employees waste too much time on facebook. If this guy just wants a number and gives you no guidelines as to how to come up with the number, then my guess is that he just wants to kiss up to the CEO that "productivity" is up 40% or he wants a number to justify laying off people. Either way, if he cant tell you how he reached his number, I would suggest getting your resume ready.
Also ideally, a CTO wouldn't be asking those in the trenches how to measure productivity, but rather how to improve it. As someone in the trenches, you probably know where the snags are in efficiency, or what software you would need to purchase to help smooth things along or even where people are over worked or over looked. This is the positive way to improve productivity. Basically he should be asking you what you need in order to get your job done, and he should get it for you (within reason of course)
meep
Productivity is measured by the number of services running and transactions performed.
Take an estimate of amount of traffic going through the computers you administer, weigh them by value - e.g. a financial transaction is of higher value than an email. Then you have a number.
Make killing spam messages weighted high and then every month set up some "honey pot" email addresses that you use to evaluate the quality of the spam filters. Every month then has a higher productivity than the past month and you get a nice bonus to boot.
This is a thoughtful and intelligent response. I would add to it though in the following way: What is the VALUE of each of these things? That is, instead of trying to measure efficiency, or "productivity", it would be more useful to the CIO to measure the actual value of what IT does. For example, if security is increased by X amount as a result of investing Y dollars, then the ROI of that investment is easily calculated and it can be compared with other investments. Note that to measure security, one must estimate (somehow) the expected future loss per unit of time. Thus, an increase in security means a lower level of expected loss per future time. The same can be done for other aspects of IT operations, including those you mention above (reliability, response time, etc.). In the end it must be measured against the value to the organization. Productivity is the wrong approach. Value is the right approach. - Cliff
We (in another life) had a term or verb for that. VURV (Verve)
--AD
Its when they ask for metrics on the number of metrics you can supply them with that you should really start worrying.
Stuff like MTBF (mean time between failures) or %uptime are possible things to keep the bean counters happy. You can probably get the data straight from monitoring systems like 'mon' or nagios. On a longer timescale, do a regular report (annual, six monthly, whatever) on number of new desktop systems installed, number of OS upgrades done, that kind of thing. Bean counters like reports.
If you run a web server, google analytics can provide you with a huge amount of numbers to pass on to management...
I have no idea what they do with all this data anyway.
The basic metric is: ( (failures * employees) / systems managed ) versus the same metric found in other teams or at other companies. The team with the lowest score is the best performer.
You could also factor in:
salaries (folks who make more money should be more effective)
system complexity (a server is more complex than a user PC)
time to repair (a 24-hour outage is worse than a 5-minute outage)
severity of failure (A total failure is worse than an irritating bug. An irritating bug is worse than a failure with no user-level impact.)
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
http://en.wikipedia.org/wiki/MTBF
The admins are just as much part of the 'system' as the hardware and software is.
All rites reversed 2010
1) Ask him what he wants to hear
2) Tell him what he wants to hear.
If you can't reasonably tell him what he wants to hear, tell him how much it will cost to produce what he wants to hear.
This is not a technical consideration. This is a political consideration. He already has an idea of how to cover his ass. Give him the asbestos he wants.
You would think uptime of servers/routers/desktops, etc, normal industry how many 9s you can brag on. Most likely your servers will have more 9s than your desktops, but the closer you can get both to the magical 5-9s the better. Those are valud benchamarks by the clock and calendar, can't get any more precise than that really. And really, that is all your job. When everything is working, you've done your job well, any downtime is a demerit and you must work harder for your normal check. Point out to the PHBs there this is one of the criteria when people go shopping for hosting, the admins there brag on their historical uptime and usually it is part of the package they offer.
Not so. If it does then he's not such a great SA. Part of an SA position is to improve the environment. Which may mean re-designing the network architecture, it may mean installing a new app, it may mean writing middleware to simplify inter-application communication. Whatever. You can't quantify SA productivity. Of course you can. With a semi-decent ticket management system it isn't even that difficult.
Deleted
Be worried. There's a very good chance your CIO is setting your company up for a transition to Offshore Hell.
Go find a new place to work, ASAP. When I see a CIO asking for metrics on every aspect of the IT department, that says to me "here's a guy who doesn't understand IT. It's prima facie evidence that he's out of ideas."
I've experienced this at a Fortune 100 company that shall go nameless. The results were UGLY. In a nutshell, two years of collecting metrics on everything led to huge technical staff layoffs and those jobs being offshored to India. The metrics were there simply to make the business case that offshoring could provide huge cost savings. From the business' viewpoint the results were devistating: service went to shit and took response time with it, no-one is accountable, innovation is dead and buried, yet senior management is convinced everything's better than ever and has declared the entire exercise a "success."
How could this happen? Here's the fundamental problem -- managers rarely understand technology. VPs know only what they read in the magazines (thus the current fad for Six Sigma and offshoring) and IMO most CIOs are absolutely clueless. Why? Simple, they're not ex-techies -- they're simply MBAs or accountants, so they run the IT department like an accounting practice: They get metrics on everything they can measure, then yell at their subordinates to improve the numbers. Shit flows downhill, so VPs talk to Division Chiefs, who talk to Directors, etc. and eventually your manager is in your cube telling you to measure what you do and improve on it.
Meanwhile, the Siren's song of Offshoring is being sung by the offshore companies. They tell how they can do it BetterFasterCheaper and promise buckets of metrics. Guess who the CIO would rather do business with?
You see, it's easier for a VP to plug some numbers into a spreadsheet and look for "% improvement" on the far-right-column vs. actually understanding what it is the technical staff does with their days. They look at technical staff in the same way one might view the janitorial staff or the lawn care crew or the Shipping/Receiving department. They don't understand that technical staff can make the rest of the employees many times more effective in their roles or provide innovation that launches the company head & shoulders above the competition. Nope, we're just the nerds who make the intertubes run and get the email delivered. Just another liability, and if they can save money on that role, then that gets them one step close to their personal year-end Performance Bonus/Golden Parachute.
You and I know that the best service ticket is the one that never gets opened to begin with. The problem that's prevented through innovation. But solving problems before they happen (through clever design) is difficult to understand and nearly impossible to quantify.
Sorry for the glum news, but it's going to be an uphill climb for you.
...no matter what your boss says. Just don't do it. It is management's responsibility to come up with metrics. If they can't do that, they're not qualified to hold their position, and frankly, I would tell them to their face. It might get you fired. But I've taken the "this is not my responsibility" tack before with some success. The reason this stuff happens is because workers allow it to happen, and if you don't stand your ground once in a while they will just keep shoving this type of crap at you.
Sys Admins most valuable job is analgous to that of a Fireman. How does one measure a firemans performance?
Daily tickets are of course a way to measure throughput, but the value is hard to assess.
Some drink at the fountain of knowledge. Others just gargle.
Maybe a pie chart of server uptime, scheduled maintenance, and downtime might work...
The unit is dollars, to measure it: all stop working for a quarter and see ho the business performs compared to a normal quarter.
:)
Well, it's worth a shot anyway
ccalam - acoustic versions of new songs.
How does one rate the value of insurance?
You do maintenance, prevention and handle emergencies. Maybe you should look at how health-related work is rated and convert human health into network health. The mapping may not be obvious, but at least they have already fought for recognition and won.
ID: the nose did not occur naturally, how would we wear glasses otherwise? (apologies to Voltaire)
"Windows Genuine Advantage Servers Out"
How does one reasonably quantify admin productivity?"
Uptime
If you save 2 mins a day over 200 days in a year that you drive, over some 50 years that you drive, you just saved two full weeks of your live by passing 'that guy.'
Obama likes poor people so much, he wants to make more of them.
Unfortunately, your value is mostly in terms of negatives. How *few* bad things happened on your watch?
* Number of minutes that network services are unavailable and/or flaky for employees (local and remote)
* Number of spams that you let through
* Number of false positives by spam detection system
* Number of hours of lost employee productivity due to overhead imposed by IT department (system software compliance, hardware inventory accounting, etc.)
* Number of kilowatts wasted due to failure to use power-saving technology
* Number of intrusions by unauthorized persons into network
* Amount of sensitive information lost due to failure to encrypt or otherwise protect data
* Amount of information (and employee time) lost due to failure to keep good backups and have a good restore procedure in place
* Amount of money wasted on licenses for software, when less expensive and equally effective alternatives are available
* Amount of time wasted on unnecessary upgrades/updates
* Amount of time wasted by dealing with virus infections
* Amount of down-time when dealing with equipment failures
* Amount of time employees had to waste working around ineffective IT policies
The IT department is there to reduce costs as much as possible throughout the organization. Failures by the IT department can largely be measured by how much employee time and effort is lost throughout the organization when the employees have to focus on something other than their actual jobs.
An unjust law is no law at all. - St. Augustine
Tracking / measuring productivity can be difficult or even counter-productive for many reasons. Some of them being...
1) the "bean counters" incorrectly define productivity - for organizations, productivity needs to be directly related to producing value for the organization; however, even the definition of "value" will change over time for an organization; also production of value can also refer to prevention of loss, production support, and more.
2) the act of tracking / measuring can sometimes consume large amounts of time, this can actually lower productivity
3) people sometimes track / measure secondary or even non-contibuting metrics - metrics that don't actually impact or have very little impact on productivity; this is extremely dangerous since it can lead companies to change behaviors / processes to improve the faulty metrics
There's a company called Ocean of One that actually specializes in this type of work. Even though they sell software related to this field, they're actually more of an R&D organization. I've read a number of research papers and other publications from these guys on the subject that are very helpful. You can also find good information related to this subject from the Institute for Operations Research and the Management Sciences (INFORMS).
Create a chart with:
Uptime
Serivce tickets completed in x minutes ( if you have a ticketing system )
Number of patches
Number of file restores/etc.
Then toss in some vague 'hours per week of maintenance' for your daily operatins of viewing logs, etc.
It wont matter much what it is, as long as you document what you do and give some times/counts along with it.
---- Booth was a patriot ----
"I am a (strictly technical) member of a large *nix systems admin team at a Fortune 150. Our new IT Management Overlord is a hardcore bean-counter from hell.
You're technical, you work for a large company and you have a new boss that sucks.
The first two points should make it relatively easy for you to find a new job. Have you considered a slight change of career? Why not engineering (development) rather than sys admin? In the Unix world, the line between the two is very blurry.
Bosses like this kill businesses.
I can't give you specific metrics for your situation, but you might like to read Lean Software Development: An Agile Toolkit for Software Development Managers for some ideas of business processes in general, why they work and don't.
In particular, there are some insightful and scathing examinations of the methods of "bean counters." You're an admin, not a developer, but it should provide a spark, a starting-off point for you.
Stick Men
Number of servers/admin (100 vs 10)
...
Number of servers up for more than 1 year (90% vs 1%)
Number of patches installed without reboot (1000 vs 0)
$$ of software in use ($0 vs $1million)
(Actually - value of $billions vs $0 - but the cost probably plays better)
Number of sleepless nights for senior management due to IT disasters (0 vs many)
Been there, done that, paid for the T-shirt
and didn't get it
I fundamentally disagree with everybody who has written here - this is an important task, because ultimately how you measure your productivity is also how you justify more money!
... that is easy, there's lots of OSS graphing software (PHB's love graphs) .. and if you write it up correctly it means you can actually be achieving your objectives while downloading pr0n (gotta test the network right?)
Whoever suggested letting your PHB develop his own metrics, clearly has never worked for a PHB - nothing good can ever come from a PHB assisting with real work, or using excel for that matter.
Letting the whole network go to crap will result in you losing your job, remember - PHB's only take credit, not blame.
While I agree with the "cannibal insurrections suppressed per week" concept - it will be difficult to quantify an improvement, when you've already got a perfect record - you can only go down.
I would suggest something tied to the network such as "Gigabytes transfered successfully"
And then call it something smart, like "The Q sysadmin load factor" :)
-- Sig down
OK, some things are going to be tough to quantify. But I suspect that at least some if not most of your responsibilities consist of putting out fires, right? I am not a sysadmin, but I know ours at my last job ended up fixing lots of stuff. When I was working in an electronic test support group, we used mean time to repair and mean time between failures to track our efforts. it does require logging problems and tracking time, but you should be doing that now anyway.
There's little creativity in the job especially when you're a member of a very large team. Tasks are repetitive. And if there's a logging system for these tasks, then surely there's a time stamp for updates for various tasks. So part of a metric would be turn-around time for each job assigned and closed.
And while you're trying to figure out how best to measure yourself, why don't you go out behind the wood shed and pick out a nice switch for me to beat you with.
Stewardship
In this scenario, a sysadmin is like a security guard. How do we measures a guard's productivity?
- Square feet of plant patrolled; or
- Asset value protected
But what if a theft occurs? Somehow we must lower the productivity number to reflect that. In this scenario, adding a lock to a door does not show up as a positive. The hope is that it will improve numbers over time.Work Units
In this scenario, we measure productivity by tasks completed, perhaps via a ticketing system. Let one Work Unit be a typical small task, like upgrading an app on one server. You can rate each inbound task in Work Units. Installing Oracle from scratch might be 2 or 3.
The problem with this scenario is that it ignores the stewardship aspect of the job. If you don't upgrade, install or fix anything one week, is that a zero productivity week?
Maybe the two approaches can be blended.
I remember an old joke about a furnace repairman coming to a home and after looking at the furnace for about a minute and a half, listening to the rumbles and gurgles. He takes his hammer out and at once precise place he hits the furnace. The furnace starts up and runs fine as if it was brand new.
The bill was $200.
The homeowner asks why so much when all he did was hit it once with a hammer?
The repairman takes back the bill, and itemizes the bill still totaling $200.
Cost of hammering, $1
Knowing where to Hammer $199
Any idiot can muck about on a UNIX box, I worked at one Fortune 500 company where everybody in the dept had Double E's. Still their main Solaris server crashed ever 3-5 hours daily and had been for months.
Took me a week to unscrew it and put everything back in order.
Me, I am high school dropout with no GED and some non-technical college courses. Still most of what I was doing was letting them do their work and not have to bother about broken systems. My value was on par with theirs as it was time they didn't lose on their work.
Nevertheless beancounters are stupid (also Beancounters are not accountants), they know the cost of everything and the value of nothing. If you really want to send their head swirling take the entire labor budget for each dept expressed as an hourly unit. Every time you work for a dept internally charge the company that much for each hour you work on a project or ticket for them or better still your company and tell them thats how much it costs. Without Sysadmins nobody does anything but fight technical fires and gets no work done.
Likely this joker found out that Auto Mechanics have a book to calculate how much to charge for each service and repair with details on how long each job should take. This doesn't work because Sysadmins are closer to being chefs or doctors then low end auto mechanics.
Even so, people who own Jaguars, Ferrari, and Maserati don't take them to Jiffy Lube.
If they complain tell them the story about about that hammer. (or better yet use on on them)
Sorry about the writing. Robot fingers, you know? Cliff Steele in DOOM PATROL #23
You can measure your productivity in the same way that doctors do: by measuring how much better you make life for your users/patients.
The medical unit is the QUALY, or "Quality Adjusted Life Year". So If you're a paramedic and you save the life of a 25-year-old who will live to 85, that that's 60 QUALYs. However, if their "quality of life" will be degraded, e.g. due to a disability, then you quantify that.
QUALYs are normally used to determine where to spend money, e.g. if this cancer drug costs $X and adds 3 weeks of semi-unconcious life, is that better than spending $X on drug Y.
How do you apply this? For each thing that you do, count how many people are affected. Then try to judge the "quality of (work) life" effect on them. E.g. being without email for a day = 30% impairment to quality of work life. So if an email outage affectng 100 people lasts half a day that's 100 * 30% * 0.5 = 15 QUALdays. To measure your productivity you then need to judge what would have happened if you weren't there (i.e. outage lasts all week or no outage at all).
Presumably you already have some sort of issue-tracking system. I think you just need to extend this to send out an additional message: "now that this ticket is closed, please tell us how much your QUAL was impaired during the period of the problem". And the rest is a perl script.....
You need to establish SLAs with your boss. If your systems meet or exceed these slas then you're doing your job and have the appropriate resources . If the SLAs aren't met, then its time to investigate.
Productivity levels are just plain stupid. A good admin with enough resources should be able to put up their feet once in a while.
> How does one reasonably quantify admin productivity?
Tell him you'll use the method he applies to himself.
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
Always be helpful and make co-operative proposals. Suggest that users who lose passwords should be docked pay according to the time to fix, thus ensuring that user incompetence is not perceived as an IT cost (I know this isn't *nix admin, but it's part of the general pattern).
Now that is effin hilarious. I think you are onto something:
Pass this suggestion on to your boss, run IT admin as a cost center. Figure out roughly how much time\money it costs to perform all your tasks, and suggest that it be billed out to other departments. Then, you can simply measure productivity based on how much you bill for (billed hours/paid hours = productivity %). Using the cost center/profit center model, you give your overlords lots of fun numbers to play with (and justify their overpaid asses), while causing all the other managers in the company to secretly loath your boss(es) for saddling them with all sorts of new costs. Plus it is really simple for you to keep track of and impliment.
HA! I just wasted some of your bandwidth with a frivolous sig!
The next thing that happens is you spend *all* your time performing tasks that improve these numbers. You don't work on anything else.
Admin type problems are essentially unforeseen (if we could foresee them, we'de fix 'em before they became problems - duh!).
Therefore no unforeseeable events will be in your metrics, so none of them will ever get fixed.
What happens next is that the people who really matter have their own, tacit, metrics. These are things like increased stock value, not going to jail, getting huge personal bonuses. You get the picture.
No matter how well yuou perform against your metrics the real work is no longer getting done. This impacts on the top-peoples' metrics, some people get fired and your budget gets cut. This cycle will continue each year the metrics are in place.
You can't win in this situation. The IT dept. goes from being invisible to the top-brass to being either a problem or a "necessary evil". Either way your position witin the company goes from someone who could & would fix stuff to being a bunch of people who are the cause of most of the problems.
I'd suggest your personal metric should be the number of times each year you update your resume.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Never, EVER, give a meaningful metric to someone who will pervert it.
The only thing they want to do here is show on a chart somewhere how "effective" the people under them are.
(distilling previous post)
I'm sure you've got a far more knowledgeable list than I do, but the suits don't care about real effectiveness. If they did, they wouldn't need to measure it! The fact is, as the poster noted, they can't measure it, so they're demanding some way to do so.
The rules are: If they take information used for one purpose (like performance) and tie it to something only remotely related (like funding), in the best case, they'll only pervert their metrics [=No (marginally poor) Child (who can't be bullied into performing just a little better) Left Behind (if by 'Behind', you mean 'Untrained at taking proficiency tests')].
If they're stupid enough to give you the choice of metrics, you either pick guaranteed inflationary metrics [=SPAM], or you pick the ones that inherently annoy them. Inflationary ones guarantee that you'll do better every quarter; annoying ones guarantee their resistance to asking you will increase. If they're really smart, they won't ask you again.
Telling them how many USERIDs you managed only works if you create more test accounts every quarter.
This pretty much says it all; your manager wants you to do HIS job. Shouldn't he develop his own metrics? He can ask you for ideas but he should do the work himself.
You are right, but you are also skating on thin ice here. Asking someone who has no clue what is happening to set metrics is just asking for trouble....
HA! I just wasted some of your bandwidth with a frivolous sig!
Try looking up MTBF, MTTR and other related metrics. Its sucks to track this stuff, but thats how most computer related fields track their metrics.
I suppose you could try something like:
service x users per admin
and remember this not an absolute but a relative value like compared to last quarter or compared with software supplier guidelines (if you can obtain them)
try to break the services down (NOT JUST 'ERP') so they can really see the value being provided.
crm
email
calendar
destops
mobile devices
phone
core financials
Use several pie charts and a few graphs. Get some of your information from /dev/random if you need to. Create a script/program that creates the charts for you from legitimate system info, such as uptime, and number of users. The more complicated you make it, the less likely he will know that you are bullshitting him. And let's face it, bullshit is your only option, aside from saying "no."
Write your own Choose Your Own Adventure. http://www.freegameengines.org/gamebook-engine/
Why is he asking for these metrics?
What is he going to do with this data?
I'll lay odds that someone upstairs of your bean-counter has looked at cutting costs in your team, and therefore appointed said bean-counter to work out how many of your team to sack, who and when.
The only way out is to *prove* that every member of your team possesses vital skills that are unique in the team, and impossible to be trained in to the level required by your company. Oh, and make sure to paint nightmare visions of what happens if any of those skills were to suddenly vanish from the team.
Ultimately, you're screwed. If he really thinks managing by spreadsheet is a good idea for knowledge workers, you have no hope. However, you might get lucky and this is a temporary infection of the stupids. Everyone gets such infections occasionally, even us geeks. The best cure is rational discussion, and he deserves a sincere effort to explain why this is a bad idea before you chalk him up as incurable and start working around him.
Point out that you're knowledge workers and it's impossible to come up with useful metrics. You can come up with non-useful metrics, but that will just encourage your presumably intelligent employees to game the system. Measure uptime? Sysadmins are encouarged to leave machines technically up but deeply disfunctional instead of actually fixing them when the fix requires a reboot or hardware change. It also encourages sinking lots of money into virtual machine systems with transparent migration on systems that really don't need 24/7 uptimes. Measure tickets resolved? Every stupid little thing will be submitted as a ticket. Don't let the admins submit tickets? Users will be turned away until they submit tickets. (That's particularly amusing when the problem is that the user can't submit tickets.) Tickets will be closed with only part of the task resolved with a note to submit a new ticket for the rest. Measure problems fixed? Encourages sloppy maintenence so you have more things to fix. Money made? Admins are a cost, not a source of profits. The value of your admins is that the parts of your company that make money continue to do so.
Try asking him what metrics he's providing on himself to his superiors. Presumably if knowledge workers can have their complex jobs with a wild mix of problems reduced to a series of numbers, a manager can. That may give him the insight to clear the fog of management by spreadsheet.
Search 2010 Gen Con events
It's best if you can find a number that will nearly always go up, rarely go down, does not approach a limit and justifies ignoring requests like that in the future. Try: number of hours since last asked to develop a metric.
I would say a mixed calculation of :
1. changes (as in ITIL changes, to any configuration item)
2. Uptime
3. Downtime
4. Tickets and their resolution times (customer opened tickets)
5. Incidents and their resolution times (any system generated ticket about a service)
Now on the exact calculation:
A. most MORONS want the admins to work 8hours straight on something and be proactive, so you get as many systems as you CAN NOT manage if an incident occurs
B. Most sysadmins know, that you are most productive when there is absolutely nothing to do, and you can polish that script, or experiment with "cool" technology.
I think the latter. I think you are the most productive, when you do not have to fix anything. Then you did your work right. But group A (Morons) do not agree with that. Probably that is why I am back in freelancing as a programmer after spending a year at a Fortune 50 company, with an hosting department ran by corrupt lying MORONS.
Just my 2c's
he will use these KPI's to try get approval for more funds. what you need to do is help him do this, since this will benifit you directly. I would suggest you talk in terms of system avalibility and utilisation. how avalible the various system services have been per day and how much they have been utilised. uptime,down time and load esstentially.
If you mod me down, I will become more powerful than you can imagine....
To all those people who say don't do it, the results are always worse without real input from the people doing the work.
If you don't do it you will end up getting something stupid like tickets responded to with no degree of difficulty. The people will fight to get the easiest tickets and the really good people who take the most difficult task will end up getting laid off or leaving first.
Been doing Sys Admin work for decades in a variety of different environments so it really depends on your shop. Uptime is a goal, not a unit of work. Risk management is also a goal and not a unit of work.
If no one knows what you are doing then keep a log and find out.
In development shops there could be support in scripting new jobs.
In production shops one would expect mostly support issues.
Automation might also be a target.
Back up and recovery always big on the list.
Upgrading of hardware and software nice to add.
The most important thing is to reinforce with this PHB that there is probably not going to be a single unit of work other then 1 staff hour.
Presumably the more senior people will get more done in the hour the the junior people.
Add up every productive activity anyone in the company accomplished that either used a computer or whose result is stored in a computer. Divide by the number of people keeping the PHB from deleting the entire fileserver and the backup tapes to make room for his MP3 collection. Bonus points if the accounting systems don't prevent timely SEC filing.
Part of the difficulty with any other metric is that an idle sysadmin is an admin that set everything up right and did his preventive maintenance well.
of his system.
Quack, quack.
There have been some very good suggestions put forth.
What came first to my mind is the question of friction. Every measurement you do is friction on the task at hand. Friction can really add up. At the poker table in a casino its called the rake. When you trade on the stock market its called the commission.
When you do your job its the additional time required to account for doing your job. The more grandularity you are required to produce the greater the amount of friction, and the less time you have available to do what you are hired to do as opposed to account for doing what you are hired to do. The accounting is pure overhead and like most forms of friction - it is non productive.
So whatever you do make sure you add a column for "friction" and make sure you properly fill in this column. Note that if you are in a meeting and end up discussing the "friction" issue then this is also friction.
That being said... one observation of systems admin is that a large part can be dealing with the learning curve. Once one determines how to solve a problem - implementing the solution may be quite easy and take little time. The thing is that one systems admin can jump into the fray and implement a poor solution and will as a result look really busy and possibly productive. A simple metric might say "Spent 2 weeks (80 hours) implementing a virus scanner".
Thing is... does the scanner really work? Next - even if it does - if an extra few days had been spent in research could the task have been accomplished in overall less time? How about if the guy had hired a consultant he might happen to know to just get it done? Most sysadmins know a number of quite proficient consultants
In the world of programming this is an order of magnitude more important.
I'll give an example. Many years ago I was faced with writing and using sort packages. In one company there were 5 different "library" sort packages. Many programmers did not use them. Even now - to this very day - a good friend of mine who is a programmer rolls out yet another copy of his very own favorite bubble sort every time he needs to sort any data. He has many ways to justify this... including that processors are so fast no-a-days that it never is an issue.
Back in the past when I looked in detail at the "library" sort routines of that company I was working for I found all were worse than bad. In fact their "SuperSort()" was a bubble sort coded in assembler and the assembler code wasn't even good.
Of the programmers who were rolling their own - virtually all were terribly redundant. No-one was doing a good job and even though Donald Knuth had researched the issues to death and published a very good book on the subject - none of those programmers had bothered to borrow the book from the library and read up on searching and sorting methods.
What of today?
We have a very good Glibc open source sort package. It can be used by anyone. It can be used in commercial code because its under the Library license.
Any programmer today who wastes his or her time rolling out yet another sort routine is wasting their employer's money. In fact they are creating a liability for their employer because someone else at some time will have to maintain the application. Of course this is a general statement. I do know of a few instances where one might be faced with a special variant of a problem and need to address it in a unique way. Nevertheless I still believe that over 80% of the problem in most software is searching and sorting.
How do you measure things like this? What sort of metrics are appropriate? Perhaps the first question to ask is "Does the task in question really need to be done?".
In my experiance people have a really big difficulty with this issue. In mathematical logic its the issue of existence. If whatever you are trying to determine does not in fact exist - then you can make no conclusions about it.
Consider the question of a court action regarding d
The Sys Admin is NOT a "Work Unit" type of job. Let me explain...Most jobs in the US are either Production or Service industry, in either case, you can clearly measure how many widgets are assembled, how many burgers are cooked, how many drinks are mixed, or how many feet of ditch were dug.
Many posters have pointed this out, but none of them suggested what type of job a Sys Admin really is. The same type of question has been asked for years regarding programmers, as people quickly realized that programmers couldn't just be measured on lines of code (which is basically similar to how most posters have suggested solving this so far). The reality is that Sys Admins are much more similar to executives in that there is no legitimate way to measure their production. Any quantifiable number (Uptime, tickets resolved or remaining, response time, etc.) can be easily explained by a counter example showing that those numbers can be fake, or even represent the opposite of the desired target.
So, here's an answer...a performance review by peers. Yeah, I know they are annoying and certainly imperfect, but it certainly provides the most measurable outcome, based entirely on what REAL people are seeing with the system. What will happen is that one admin or a team that are getting the job done and are successful will have little or no complaints and get very high scores. Even if there are several technical issues that arise (which happens even with a great team of highly knowledgeable people that don't make mistakes), the users and other admins will recognize that it's outside of control, and still give good responses. If people (including other admins) recognize that one or many admins are slacking, that too will become obvious.
- Nobody would know what RTFA meant if it didn't need to be said all the time
When they ask you to measure your work on their metrics, usually that means they need to sack someone and need numbers to make a choice.
The correct answer for those beancounters would be: you can't measure the productivity of a system where one variable (sysadmin productivity) is a function of another variable (emergencies and trouble tickets) which is completely unknown.
The most productive sysadmin in the world could spend the first month tuning things, arranging backups, teaching users etc. in order to foresee any possible failure, then spend the following 11 months playing tetris or surfing pr0n while waiting for something to happen. Pr0n jokes aside, would they fire this guy for being not productive for 11 months?
That would be like sacking your doctor because you're in good health or building a restaurant over the local police station because the crime rate is too low (or zero). Don't expect the average suit to understand this though; they often completely ignore the creative part of our job: one hammer hit = one second => one buck, 100 hammer hits = 100 seconds => 100 bucks.
The measure of a sysadmin productivity is his/her presence, and it can be one or zero. The only analog numbers here could apply to the sysadmin level of preparation. There are tests to do the measurement, but that's a diffeent story.
If they are in desperate need of numbers about productivity, probably it's time to start looking for a safer workplace, but if you feel brave enough you could always fool them at their own game before leaving by compiling the most complicated analysis a bunch of geeks can produce, then look at them wasting an entire month figuring out what those numbers mean.
Well, actual performance metrics (keeping transaction times fast, keeping track of GBs transferred or hits per hour or the like) will be able to demonstrate if your systems are being increasingly loaded over time or just steady.
As for actually measuring metrics -- I have no idea. If things are running smoothly you should be able to demonstrate this, but if the beancounter takes this to mean he can cut some people, I don't know how to use metrics to disprove this.
Someone WILL have to have a talk with the beancounter, and make sure he doesn't just decide he can cut down to the bare minimum -- people WILL go on vacation, emergencies WILL pop up from time to time, and new installations will be requested. With some skeleton crew, emergencies will take longer to solve and installs will take longer (for anything that's not fully automated, throwing people at the problem will get it done faster... for instance, physically installing the new machines.) If they cut to bare minimum, then when someone is on vacation the department is BELOW minimum.
If it looks like your beancounter's goal is to cut down to bare minimum, look for a new job now. He won't pay you more but will expect you to be on call 24/7 for free, never go on vacation and generally not have a life. It's not worth it.
There's a book out there called "The Visible Ops Handbook: Implementing ITIL in 4 Practical and Auditable Steps" that might be exactly what you need. It is written by a few heavy hitters in IT and information security, including George Spafford and Gene Kim. It is based around a subset of the Information Technology Infrastructure Library (ITIL), which is the major international standard for IT management. It revolves around sane configuration management of data centers, in particular making all changes predictable and reliable, and lowering the likelihood of unexpected downtime and failure. Metrics are a big part of this system, and it might be exactly what your boss is looking for, as well as a good way to make your own lives easier. It is really short, at about a hundred pages or so. You can pick it up in any number of places, including here. Good luck!
Call yourselves BOsFH? I find it hard to believe that anyone has yet to mention LARTs/millifortnight as the standard unit of sysadmin productivity. There's also [notches|dings]/clue-by-four and deletions/phonecall (what was your username again?). I shouldn't have to mention that the blackmails_ongoing/boss metric should be kept strictly between you and your PFY as it's only good for working out the junkets/week figure and the correct work:pub_break ratio to adopt.
Now, I'm not saying that you should use these figures. They're intended as self-improvement figures for internal ops use. A real BOFH would find something else to occupy said boss such as, oh, I don't know, say not drinking his own urine when stuck in the lift (with the broken telephone) over the weekend?
Resistance is futile. Reactance buggers it up.
If you rate, and measure by a specific goal, that's what your subordinates will achieve, to the detriement of others.
Examples:
Video card benchmark software used specific bitmaps; so some video card makers hard-coded those bitmaps into their drivers. benchmark score went WAY up, everything else went slightly down.
No Child Left Behind; except the ones you expell from zero tolerance rules (photo of a gun, asprin, etc.), saving more budget for those left.
Mortgage lenders being rated on Volume, instead of Quality of loans; see the current fiscal crisis.
Companies cutting costs to raise shareholder value, by outsourcing manufacturing of pet food, toys, and tires to the cheap labor in China, which is rated by production quantity, not quality; a few dead puppies, brain damaged children, and people killed in auto accidents.
A small software company I worked at did not want me entering bugs into the database until they were approved to be fixed; that way it appeared that all bugs were fixed.
(this was the place, where after I pointed out that a user to a website under development could create a user name with embedded script tags, the response I got was "Why would anyone want to hack the website?", this was a supposed "Web Development" company, that created a website for Wizards of the Coast... Being a contractor, I just left within the first month, not wanting to be associated with the Impending Doom.)
A productive Sys Admin should have NOTHING to do other then watch everything run smoothly! If they are busy any amount of time, then either they don't know their jobs, or things need to be replaced. So, the metric should be System Uptime!
Number of people able to work.
If your systems are up and running, that is the headcount of the company. If a server is down, it's headcount minus number of people who rely on that server.
You could also use an even simpler one: Number of IT bosses who have a job because I'm doing mine. (one: you)
Third time's charm: If you want to be a bit sarcastic use this one:
Hours spent administrating the servers vs. hours spent on pointless paperwork that contributes nothing to our SLAs.
Assorted stuff I do sometimes: Lemuria.org
Uptime of the systems the admin is responsible for.
The Wise adapts himself to the world. The Fool adapts the world to himself. Therefore, all progress depends on the Fool.
Average time taken to react to, & resolve issues.
Wanna fight ? Bend over, stick your head up your ass, and fight for air.
I'm stunned you are having trouble with this...
- Service downtime (and variants, such as unplanned service downtime, service unavailability).
- Trouble ticket response times (IE, start to finish time)
- Average trouble tickets in queue.
- Number of re-opened tickets.
- Employee satisfaction with their desktops
- Number of complaints (gotta be careful with this one, you have some users that complain a lot).
"But sysadmins have a number of duties which they are performing /continuously/, so how can you quantify that?"
Get a daytimer, and jot down one line items when you do things. There's no such thing as "continuous" actions. Everything is atomic. You don't tail a log and stare at it for eight hours, do you? No, you check it once. Then you look at it again later.
You might not like itemizing your day, but a log of responsibilities completed IS a productivity log.
Since he is a bean counter, perhaps he's thinking along the lines of Return On Investment (ROI) and/or Total Cost of Ownership (TCO). It helps to ask him more questions--find out what he *really* wants and what his long-term objectives are (e.g., cutting staff).
What one fool can do, another can. (Ancient Simian Proverb)
How do you put metrics on janitors? Sysadmins are just that, for computer software and such. There are no strict parallels. But how do you measure productivity of a person that is responsible for upkeep, vs production. You can't so you'll have to let them bring overall system evaluators, on monthly basis to see if everything is cool.
I can't any other way to measure it. How can you measure productivity of prison guard that works on top of the guard tower.You can't really. Thats why there are managers, good ones can keep beancounters at bay and have you work on interesting things.
2c
surely it's not that hard peeps...
- severity 1 and 2 cases per month, graphs of trends, cause categories, time to close vs SLA
- no. servers supported and in build, trend graph
- servers at current / n-1 / older patch levels
- **availability** , measure against SLA
- resource usage for disk/cpu/network etc, trend graph
- other major issues, eg daylight savings changes, y2k, security vulnerabilities
-jmw
All bean counters want to measure there human assets in terms of ROI. I too am faced with the same question every FY. Bottom line is that Sysadmins offer no ROI. If you are doing your job well, nobody knows you're there. if you don't, and here is the clue, everyone knows you're there and it is at that point that you can both show your "ROI" AND highlight that you are understaffed. You see, they will cut back where there is no squeak. Kinda like buying a car but deliberately omitting the spark plugs. Anyway if you want to count on the ROI taley, quit doing such a good job.
What great fortune for rulers that men do not think.
For more fine-grained stats you could even split up those values into "user" (fixing stuff for idiots), "nice" (high priority stuff), "interrupt" (dealing with users), "system" (regular maintenance), "I/O" (meetings) and "idle" (reading slashdot).
Newer sysadmins even come with cost-savings built in. Simply lower their pay if they're mostly idle and step-by-step increase pay when uptime is reaching certain thresholds.
See? It's that easy!
:/- spoon(_).
Here's how it goes . . .
Number of voice mails recieved / number replied
Number of emails received / number replied
Number of help tickets received / number answered
---
Add em up somehow and make a score
Clickety Click
I've got an Masters in Health care Administration. That's an MBA for people who run hospitals and such. If, say, doctors were to be measured for work done, it'd be my job to do it. I'm trained in human resources and management stuff. Doctors are not. Doctors (a) maintain peoples' health, and (b) fix peoples' health problems. The latter is very unpredictable in number and severity the former somewhat like that. In any case, doctors are busy people, and shouldn't have their time wasted when there's sick people that need care.
... money for stuff.
If I were to do it, it'd be in aggregate. I'd look at how many people were seen for recurring care, and for acute care, and look at how many people were needed to care for them in each case, over the course of a year. I'd go as far as quarterly if forced to, but no less. I'd use statistical testing to compare one time window for another, and that'd require a large sample of task counting. Finally, I'd make the point early, often, and forcefully, that quality of care is more important than anything like task accomplishment in a medical setting, and I'd compare those numbers to patient reports of satisfaction or dissatisfaction in order to give an answer that was in essence how GOOD the care was that's being provided, and wait time (something I can quantify by going around asking people) would figure into it heavily. Health care is about quality of life, not quantity of life, and I'd dare anyone to try to counter that. Some would, of course, but I'd tell them they're idiots and stand my ground.
But the point is *I* would do it. I'm trained in it as an HR and admin person. Figure the odds on getting doctors to do it. They'd rightly scream about having to do bean counting work when they have patients to see.
You do quality work too. It's your job to keep your users' machines and pipes healthy and happy, both through maintenance (which varies somewhat with time and can't be easily quantified) and fixing problems that crop up (which can't be predicted and therefore quantified except in counting them after the fact).
Make him do it. You're a "computer person". Since when are you qualified to do human resources work? He's the bean counter, he should be the one to come up with it. Complain to management that he's shoving his work off on you and causing you to spend time doing that instead of your own work. In fact, he's not acting like he's a qualified bean counter if he doesn't know that. Either he's dragging his feet and making you carry him, or he's not a person-bean counter but just a money-bean counter and shouldn't be a person manager. Take that above him, repeatedly and vocally, and complain that even doing THAT is taking you away from your work.
And if management doesn't agree with you, start keeping very detailed records about tasks done minute by minute during the day, and turn it in weekly. Include statements about how much time you're spending doing that and other stuff like meetings that aren't directly related to keeping your iron clean and oiled. And complain about that. Include in the complaints the needs for more people because you're accomplishing less with having to do this administrative work. THAT is an answer they can get a measurement handle on.
And if he should come up with it, complain that he's expecting you to handle unpredictable problems on a schedule, and must therefore be in need of an anal craniotomy. Either that or you're going to need a LOT of new hardware as well as people so that you have less problems to fix and therefore more time to spend doing his job. That's another thing they can measure
I've gotten shit from doctors asked to do this (NOT my idea; I'd never ask). I've taught medical students not to put up with it. You shouldn't either, and you should report to upper management anyone who's refusing to do their job, even, no ESPECIALLY if it's your immediate supervisor and he's trained to do that while you are not.
Scream, and don't stop until they make this problem go away so
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
It is common for senior management to want to have some measure of how well staff are performing. The truth of the matter is though, that for many types of job there simply aren't any good ways to measure productivity. If there were, this question wouldn't be asked so often.
I think that the difficulty of measuring productivity increases with the complexity of the task. It is easier to measure the productivity of an assembly line worker who has one simple task to perform repeatedly, and more difficult to measure the performance of a software engineer or system administrator who can make use of imagination and creativity to do a better job.
That said, it's not necessarily impossible, just more difficult to provide accurate measurements. Replying along the lines of "I can't be measured" is unlikely to result in a sympathetic response. A wise senior manager however, would rely not only on the numbers, but would use the numbers to supplement a human observation of what was going on.
To answer the question - I would recommend focusing on the bottom line. "Number of tickets answered" holds the danger of measuring how hard you are working, now how effective you are. Ask the manager what he expects from the sys admins, and base the measurements on that. "Number of e-mail problems reported per month", "Dollars spent on system administration per month", "Number of data loss problems reported per month". These are all areas that a sys admin has an influence over. The senior management recognize them as areas they would like maintained/improved.
Monitor and graph them yourself. Describe your projects in terms of how they will improve these numbers. If you find yourself wanting to work on something unrelated, perhaps you have discovered a new measurement that could be monitored. It's really important to keep the manager/beancounter involved in this. You should discuss and agree on new or changed measurements with them. Meet up with them regularly and show them how the graphs are progressing. After a while you'll feel more confident that you are doing a good job. You'll find proving it easier, and the beancounter will almost feel like part of the team.
I happen to be UNIX manager at a fortune 150 company oddly enough, and I was formerly an administrator myself. I agree with many of the other people that suggest that is the manager's job. Saying that as a manager, I do feel that it is wise to have employees participate in the process to make sure they have a stake in the process and justifying their jobs existence. I come up with a set of baselines and run those past the folks and make adjustments to them as a team to make sure what we end up with is both sane and accepted. We do this early every year. Every environment is different as companies and organizations are structured differently. My opinion is that whatever your folks do, DO NOT make the metrics and graphs too complicated, otherwise higher management will not be able to follow the logic and won't accept it. In addition, it is critical to communicate from year to year what the old rules were, what the new ones are, and why they changed. Hopefully the communications of the alterations of rules, numbers, and metrics make it look like the team is taking on more work each year, adapting, and focused on the important initiatives to the business. In our case the numbers will be going up, and that is because the folks that I support are working with me to Adapt, Automate, and Alter processes (AAA). As we learn, we automate, and mature the process flow, the numbers and metrics will change from year to year. But really what rules and metrics your organization chooses is a judgment call based upon your environment. We have a lot of internal and application and software diversity, so at an ultra high level, this year we took into account number of servers per administrator, number of O/Ss supported by each administrator (the traditional UNIXs added a very significant weight). We are always building new graphs and considering new metrics mostly to show our own group how much we've changed, it gives the good workers a sense of accomplishment, and the others the feeling that they need to adapt. This will help us gauge what rules we need to come up with the next year and we always have numbers to justify new headcount when we need it. Based upon our predicted growth numbers, we'll have to add to our headcount soon, of course there is a freeze on hiring as we're in the later part of the fiscal quarter. I'm battling with the statistics now, but I likely won't win anytime soon, so we're dropping some less important work on the floor, sooner or later we'll get what we need. :)
I hope all goes well in your company, and for your team.
Sig -my username was entered as a joke to prevent association with my real life.
First of all you are a admin in the last 50 of Fortune 150, because you are not in the Fortune 100 :)
:) But that would go to the extreme. You can hint that to your PHB and maybe he'll realize that all those statistics are irrelevant.
Asking you for productivity is like asking you PHB for productivity...
You can always do like they do forward the request for productivity to the object that you manage(in this case you machines).
But machines cannot give you any output on that point, so you'll have to compile it yourself.
Put monthly average downtime there... The load averages for your machines... And such...
Ask users to provide their productivity, because your "productivity" is a result of their productivity...
Except for downtime and load averages...
The solution I see is to use tickets resolved as the metric than require a ticket for every thing you do no mater how trivial. Before every ticket is worked require it to be approved by the Sr. VP of their department. Open a ticket for every single patch you apply to every system and require a Sr. VP to approve it. Ideally you will end up spending 3 times as much time approving, acknowledging, updating, and closing tickets as you spend working on the systems and the backlog will grow to the point where users are waiting weeks to have their forgotten passwords reset. When anyone complains explain that your administrator's productivity as measured by tickets closed is higher than ever. Soon everyone will long for the old days.
Let's see: you all are system administrators, and bean counters want you to create metrics for your productivity.
That seems like it will generate pretty low productivity for you, as your skillset is to fix and prevent failures, while theirs is to derive metrics and measure productivity. They should be more effective at coming up with metrics. Why are they asking you to do their job, and how can you do your job at the same time?
Maybe someone can ask them to patch the north 40 server rack while you scratch your head over metrics.
You've been asked to do something that's not your job (as others have pointed out, metrics are what MBAs are paid to undestand.) I will warn you that trying to give an "honest" answer can easily create the opposite effect of what's desired. For example, if one of the metrics is "time to close tickets", then there is no incentive to solve pervasive problems that are easy to fix. For example, it takes one line of code to make software understand both upper and lowercase. Fixing a bug where the software doesn't take lowercase is trivial. However, if the metric is "time to close ticket", then fixing the problem would be counter-productive, because such tickets are fast and easy to close.
In fact, if you do your job well and take preventative measures that solve the easy problems, the tickets become progressively more difficult to close, which makes it look like productivity is decreasing, when in fact it is increasing.
This is the same problem as standardized tests in education - teachers teach to the tests, not to a well-rounded education. Similarly, whatever metrics you define, that's what employees will work towards, whether it benefits the company or not. People want to get raises and get promotions, so they'd be foolish to do anything else.
I've worked in several companies that put IT under the bean counters. Without exception, the result was that the competent people left.
Sysadmins != mechanics/technicians? News to me...
the BogoMIP!
... in my opinion, is to be as bored as possible. Everything which is done on a regular basis should be as automated as possible, and as much effort and resources thrown at avoiding potential problems as the finances and customers will allow (data backups, spare or redundant equipment, etc.).
Much of a "good" sysadmin's time should be spent doing regular, but occasional spot checks on the automation (which can also be greatly automated) to ensure everything is running as smoothly as possible.
Obviously, not all problems can be avoided, especially hardware failures, but if everything else is in place, even recovering a dead, but critical server can be fairly painless.
A Service Level Agreement is key to the Facilities Management deals. You spell out every little task in shelfware, and assign a priority. Then you get a ticket system. Each ticket must be resolved within the time allowed for its priority. This is totally bean-counter compatible, but at the cost of throwing real user support out of the window. If the task isn't listed in the SLA, you don't do it. After the first month it won't be hard any more, because relations with users will be so bad, hiding behind your SLA will become natural.
Think of it as evolution in action. Many orgs have been caught in the fashion for putting OCD-style bean-counting ahead of common sense. The stated minimum always becomes the effective maximum. Such orgs suffer reduced efficiency and poor morale, both of which impair profits. Of course, the more dedicated the team, the worse the hit. If you really want to do due diligence, advocate "Lean Service" - the jargon term for a post-ISO9001 return to sanity, especially in service contexts. Google it - there's plenty of that funny language they use to discuss this stuff, and case studies.
You can quantify how well a sysadmin is doing his job by how little down time there is on mission critical systems that he has control over.
Shadus
That's how many commands you typed today.
Hey, he wants to count beans, give him his beans.
I work in a research office of a regulatory agency, and the government bean counters are always pushing us to justify our existence. Now I am writing a justification of the research we will be doing in the next couple years. It is supposed to include "milestones", and "deliverables", and "stakeholders". These, of course, are demanded by people who do not do research. If I knew when we would reach a "milestone", it wouldn't be "research". If we knew what the final end product would be, it would not be research.
Three years ago we wrote such a justification, saying we will "deliver" X. Of course, we discovered X would not work, our research took a turn and we "delivered" Y instead. I should just make stuff up.
You're too smart to understand what he's talking about. You need to bring in some less intelligent to help bridge the gap between you and the bean-counter. Do you have any small children? No, strike that. How about an eldery relative with Alzheimer's? No, that would be cruel to Gramps. Do you have any pets?
As much as it would make his life easier, there is no unit of production for system administration in general(there could be for specific IT departments though).
The evalutation of any employees activity can only be evaluated by a supervisor who is actively paying attention.
This goes 2x as much for programmers in general.
Whats more, if you develop a unit of productivity and measure the employees with it, they will gravitate towards work which involves achieving the units the easiest.
If you havent noticed, it is *really* easy for system administrators and programmers to give the appearance of 'working hard' when they really arent - it take careful management to figure out what someone is really spending their time on, and its true that some of the simplest sounding tasks become an endless time sink and some of the hardest sounding turn out easy. Its the nature of the work.
The first issue is that he is asking you to do his job. So you can add him to your workload equation. Secondly, the entire company's productivity is that productivity of an admin. So if your company makes 500 million a year, divide that number by the number of admin's running that infrastructure and keeping the pace. If the phb has an issue with that calculation, have every admin leave for 3 months and see how much the company makes for that quarter.
as in the whole IT Department for say 10 days at the same time... and let them figure out how much an hour of each of your time is worth?
One of the goals of Bcfg2 (a configuration management system developed at Argonne National Laboratory) is to help provide this kind of information for both the admin and the manager. Its reports system gives a clear look at how well managed a machine is both now and in the past, making it simple to define how up-to-date machines are staying and show how they have changed over the last week/month/quarter/whatever. This is touched on in a couple of the publications that go along with it.
(Yes, I am indeed in cahoots with the project, but I'm only gushing about it here because I actually think it works quite well.)
Posted from the wireless couch.
It is not so much that having metrics is bad, rather that if he is in IT management and doesn't know the metrics for sys admin productivity already, then he doesn't know what he is doing, and won't understand, and will mis-interpret any metrics handed him. Not to mention asking people to develop the performance metrics for their own positions is the kind silly management voodoo that should have died with the rest of the idiotic 80's fads.
my old sig is obsolete, and I haven't come up with a stupid enough new one yet
I hate to say it... because I hated using it... Remedy. This is a good start for your main tasks: hardware replacement, maintenance, projects... everything is put into Remedy. If someone asks for something, ask for the Remedy ticket number. If it's not there, it doesn't get done. This will give your PHB all the information that he will want, including the ability to categorize severity levels. Hell comes in when you start using the software. You will spend more time managing your processes and people than you will actually doing the work. You will spend multiple hour long meetings deciding on the different levels of outages. You will spend days prepping Remedy tickets just so you can can do an hour's worth of work. Remedy won't cover everything, especially your day to day activity. So we had to send our weekly achievements to our PHB. Since I could never remember to send the reports I created a CLI LAMP system (w/ an web interface) that would do it for me. Whenever I did something including tickets, I would enter a command and a task description, "compressed files", and that was it. Cron mailed the reports for me, and it was consistent. I really recommend a database as well. Because when review time comes, you can pull out a nice detailed report of everything you did for the company from the last year. And don't forget a vacation function for when you go on vacation... the PHB will say, "But you didn't do any work on Dec 23, 24, 26, or 27".
Well I'd say it would be inversely proportional to the number of slashdot posts made...
'taskisnotaverb'
(same tag was used in a different 'ask slashdot' article earlier this week)
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
When a IT dept is feeling underfunded they merely threaten to quit. The reaction is akin to a three year old being told mommy is leaving. Good IT people are worth their cost, and most are not so greedy. They stay because they want to not because of the money. Beancounter bosses tend to pay for this sort of thing later as nobody wants to work for them when word gets around. Soon they have ex-MickyD employees running the Desktop support and the smell of French Fries just won't leave the server room.
Has was stated earlier, You can not derive a metric for a negative event. You can only measure a sysadmin's stability factor in terms of (well) stability. Other metrics could be created by how long new services appear after they are requested, but there is no way to gauge the factors relating to them.
Management:
We would like to replace Oracle with MySQL how long is it going to take?
All our servers are running Solaris 8, the current version is Solaris 10, please upgrade asap.
We want to move off linux and on to HP-UX before next month.
As to the opinions of others....
Hi, welcome to IT you must be new. Per hour performance metrics is about the stupidest thing I have heard. We are not factory workers we are information workers. One problem can generate 2000 helpdesk requests in 10 minutes and take less then that to fix. I am not going to write out two thousand emails to tell everybody that we had a fiber cut from the ISP and our mail server will be down. Sounds like you've been reading the latest management fad like Sigma 7 or some other such tripe.
Actually thats not quite how it works. If the folks in accounting (non-beancounters) usually call up and ask if there is a problem and you give them a update on whats up and they go back to what ever they where doing. If the dept needs more money next time around they just explain what its for in a meeting. If it's not ludicrous then the usually get it. If the manager gets the numbers wrong then that manager has to fix it some way. A good manager will pad it and use Departmental luncheons which tend to eat up the slack.
Sorry about the writing. Robot fingers, you know? Cliff Steele in DOOM PATROL #23
Should be some sort of metric which expresses a comparison of calories burned by sysadmins, vs the uptime of services on the system. If the uptime of services is high, while calories burned are low, then that's a good thing. If sysadmins are burning lots of calories putting out fires struggling to keep a highly unstable system from collapsing, that's a bad thing.
Loose lips lose spit.
You need to flip all these metrics around. What's a good metric for an IT manager? How about the percentage of staff that is still working there after a month of 'performance metrics'. Time to quit and find a new job.
... is the number of minutes elapsed that underage gay pr0n does not appear mysteriously in the PHB's account.
Have gnu, will travel.
"How many tickets were not opened in the first place because things just work."
"Functionally stable": A euphemisim meaning your project/system is going nowhere and getting nothing.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
To keep the system running smoothly.
So, find some reasonable metric for the number of problems you have, and define
productivity = 1/problems
Quidquid Latine dictum sit, altum videtur (anything said in Latin sounds important)
BLiPS - Bash Lines Per Second
Does it hurt to hear them lying? Was this the only world you had?
I am an SA who became a bean counter. One of my primary motivations was that I saw f*ck-ups getting rewarded with less work and raises while hard-working SAs suffered with more work and dead end jobs.
I think management deserves to know what is good work and what isn't. If you leave it up to them, they are going to pick something like tickets resolved or customer satisfaction and you are going to see the a**-kissers move up while the hard-working straight-shooters get the shaft.
I think the metrics described here are good ones, but I'd change #4 to the ratio of load to capacity -- which is a measure of efficiency and good planning. Overall, a good SA should be able to maximize delivery of services. I'd also change #5 to security risk measured as ELV (expected loss value). I know a lot of security professionals who hate this and think it is meaningless, but so far none has given me any better metric to show management that security risks are actually getting better managed over time.
In short, think of what a good SA does for a company and propose metrics that reflect that. Do NOT leave it up to management like some have suggested. THey are asking for your opinion as an expert. Step up and show that you are the expert by giving them an expert answer. Show them that you know the difference between a good job and a bad job.
the best sysadmin is the one you don't know exists
until you need them -- as long as things are working well
without glitches or downtime, i'd say your sysadmins are working well.
2cents
j
You can identify companies that are being managed along these lines by their sudden interest in generating metrics. In production line situations where units of productivity are obvious and easily measured this approach has some validity. But trying to apply these theories to jobs where what we do is reactive - there's no valid way to do it - and no valid reason other than to sort out who's going to get canned. By reactive, I mean that the workload is determined by outside events; you don't know what you'll be doing or how long it'll take until something breaks and you've fixed it.
Here's a whack with the cluestick for you: if you think that the fact that you (and your team) keep the systems running smoothly means you're doing great - that's not what management is thinking. They're looking at the labor cost and what they get for their dollar. If nothing breaks then they don't need to keep you on the payroll. And they'll probably get a bonus for reducing expenses after letting a few of you non-productive people go.
If they're measuring performance by the number of tickets you handle - get your resume in order now. When a virus comes along and disrupts operations you'll be buried under a huge number of tickets; you'll take care of business and get things back in order - and the management will look at how you've handled 500 tickets in a month and consider that a good benchmark. A few months later when things are a bit boring - they'll look at the metrics again and see that you've only handled 50 tickets in a month. Good luck trying to explain how your productivity dropped 90% and it's really OK. If you convince them that you couldn't do more because there wasn't that much to do then you're arguing for a headcount reduction. If you can't convince them, then you're a slacker. Either way, somebody's likely to lose their job.
Or if they're measuring performance by the time it takes you to resolve problems - get your resume in order now. If your resolution was to reset a hung server then you got done very quickly and you're a hero. But if Exchange burped and ate its database, the hours you spend reloading the database from backup and getting the mail back online will count heavily against you.
Really, it doesn't matter what set of metrics they choose - since our work doesn't come in equal sized packets and it doesn't come at regular intervals, any production measurement is essentially a random number. But it doesn't matter to management; they've been tasked with cutting payroll expense by 10% and they'll use these "metrics" to determine who goes and who stays. Implementation of performance metrics in an IT department is ALWAYS followed by cuts in staffing.
Who will it be? They're figuring it out using the metrics they've collected - regardless of the validity of the data. What this means is that it's just as likely to be you as anyone else. And even if you're "lucky" and don't lose your job - you'll still get stuck doing not just your work but also the work of the people who were let go. It's a real lose/lose situation.
Did I mention that you need to get your resume in order?
Is 95% boredom and 5% sheer terror.
read www.deming.org, and maybe Walter Shewhart's book (at Amazon)
Then define your system, list measurable system attributes that are or should be instrumented, then weight by impact (& reorder as hierarchical list, if possible). To define operator productivity?: maybe report operators frequency of monitoring/sampling of key system attributes? [Given there are 2 phases: waiting for STH, and responding when SDH]. Real point is to discuss with your team how to sustain optimal system productivity EVEN IF/WHEN operator productivity is not worth instrumenting. (Do you have CONTROL of your system? If not, do you know how to start improving your degree of control? Is SysAdmin productivity anywhere near the top of your key steps for gaining control?)
Most productive SysAdmin? Someone honest enough to tell their team what to focus on if they want to continuously improve operational control.
... because a system administrator isn't producing anything, any more than a safety engineer is. They're there to preserve certain non-functional properties of the system. The appropriate measure is how much of the time the system meets or exceeds the service level agreed to, and what the cost is in staff hours to do that.
Trying to turn it into a "productivity" measure will have the inevitable effect of maximizing whatever is being measured, whether it's LOC of scripts, service tickets closed per hour, or kumquats per fortnight.
The issue should never be how much does it cost to have a working car (read: system administrator) but how much "pain" can be absorbed when its "unavailable" and you have to get someplace (like a wedding). In that case a you can always send a proxy (read: outsource) or suck it up and ride your chopper bicycle through the inner city (don't forget your helmet). I work in a manufacturing facility where it I sit alone but when production is stopped "they need me really bad." Because of the perception of underutilization I get a LOT of projects (which is good) but when Production is literally pounding on the door, then the situation is transformed. Calling a proxy who was "awarded" the cheapest Service Level Agreement gets in the way of getting stuff working again. A fifteen minute proxy callback can run into a lot of money that failed to get made. So. My suggestion is to calculate how much it will cost if the worst case scenario happens and then work backwards.
Whenever the answer to a question seems particularly nebulous, it's because you're asking the wrong question.
System administrators are not, in themselves, productive. They do not produce the product that the company sells. Therefore, their productivity is not measurable.
What _is_ measurable is the effect of proper system administration on the rest of the company. It's not even difficult to quantify.
What you need to measure is minutes of user downtime for every technology user in the company. Every minute that someone in the company wastes because a resource that you're responsible for is not available, >or because they don't know how to use it counts against you. Every minute they spend on the phone explaining the problem to the help desk, every minute they wait, period.
How to know? Ask the users to report their minutes of downtime in each ticket, in a machine queryable way, such as: [27m]. Major server went down? Multiply it's users and the time down as minutes in the ticket IT opens to fix it. Want to make sure people report minutes? Report on the number of tickets with no minute report as a negative, and assign that blame to the technician who closed the ticket. Worried about technicians lying down their numbers? Copy the affected users on the minutes reported in the ticket. They'll correct it.
It doesn't matter if a server is down for patching over the weekend if nobody is trying to use it. This is why measuring user uptime is stupid. It leads to not patching.
Make your IT department responsible for training. IT's job is to create efficiency for everyone else. The only way for you to get credit for the amazing technology you deploy is if users know how to consume it. Training isn't a budget buster for IT, it is the realization of IT at the point of use. If it's not part of your process, you will never reach zero user downtime.
You are what you measure. Only measure those things whose trend towards ideal has no negative impact. People will make their quotas and statistics happen, so you better damn well not have any unintended consequences in your statistics.
Of course, there's a lack of clarity around who is to blame for downtime, but it doesn't really matter because only you can get them back to work, and you're not trying for perfection, merely improvement.
The best you can do is to get up to zero. Zero downtime is SA nirvana--not a state you can attain, only a state you can aspire to.
Want to count beans? Those are your fucking beans. Go count 'em.
aka Matthew at SlashNOT/!
I like your hammer story. Here's one that I use. I'm not a system admin, I'm a programmer so it doesn't map very well to the problem at hand...but since we're discussing these kinds of things, here goes...
I've had managers who ask me how long something will take. "How long will it take you to finish item Q on this Gantt chart?"
They ask this because they think all work is alike. Programming is task/hours=rate. Like bricklayers. If the wall has 120 bricks in it, and a worker can place a brick a minute, then one guy can build the wall in two hours. Or two workers can build the wall in an hour.
Of course, programming is not like that at all. I tell them it's more like trying to find something you've lost. Their question, in another form is like this. "I've lost my car keys." "Well, how long do you think it'll take you to find them?"
Weaselmancer
rediculous.
There's really only a few numbers you need to be concerned about.
* How many people is the bean counter going to cut from your department?
* How much is the bean counter going to cut your department's budget?
* How many weeks until your job is eliminated?
The final number to be concerned about is: What's the going rate for ex-F150 sysadmins?
Form a union or form a company. Go off site with the other sysadmin colleagues and either form or join a union or form a company.
If union: Form it legally. Then negotiate for better pay, recognition and benefits. Take legal actions as necessary to achieve your collective bargaining objectives. Use the "unreasonable" demands for undefined metrics as justification. Cite the beancounters by name.
The basic problem with current IT Sysadmins and Janitors (which is what they think you are) is that no one has the balls our grandparents had to join together to stand up to PHB's. Collective bargaining is a reasonable and effective strategy, if you let them divide you, they will win.
Are you really so naive as to believe that MBA classes are about how to measure things and how to "listen" to employees. ROFL. Grow up will ya. MBA's are about how to obtain and wield power to become personally wealthy. It's that simple.
Part of their strategy is hiding behind smoke screens and FUD, using HR and other gullible people types to deploy change. Measurement of things where the units are intrinsically undefinable is an old and effective technique. Play their game, their way, and you will lose.
If "form a company": This is a much better alternative (if you are a small group of Sysadmins they need and can reach a practical agreement). Form your own services company. Then all quit on the same day and immediately offer your services (on reasonable terms) to the same company. The terms should offer no loss of continuity or support and an effective SLA, based on Your Metrics This is quite legal and reasonable, it's how many companies start, it's called free enterprise. Hopefully during the few days it takes to complete the negotiations, none of their systems fail catastrophically. But if so... why would you care... Somebody else's problem.. right. Either your worth paying for your services, or your not. The real question is do you have the testosterone to really find out?
The only question any MBA cares about, is who dies with the most toys. Ethics is a course taken so they know what to avoid doing by accident.
The only option for working practitioners is either collective bargaining (poor and outdated) or running your own services company (better imho, but takes real organization and balls)
So... have you got a pair?
What part of "Get Control", don't you guys understand?
There is no god; get over it already! Never exchange a walk on part in the war, for a lead role in a cage.
If you fail to make your boss happy - let me tell you what is coming next:
!!! Time Cards !!!
9:00 - Read Slashdot.
10:00 - Posted to Slashdot.
11:00 - Read a few other web sites.
12:00 - Left for lunch.
1:45 - Deleted old users.
2:00 - Restored deleted users from backup, oops.
3:00 - Sat in a meeting about SysAdmin productivity.
4:00 - Complained about sutpid productivity meetings in IRC.
5:00 - Posted "Ask Slashdot" about how to get out of productivity meetings.
Expect this to continue day in, day out, year after year - until you retire, die, are fired, or quit. Hopefully some day soon you'll realize the shop down the street expects none of this from you because they have competent managers. Even more hopful, you'll get hired there soon after this realization.
All I can say is, I feel for you. I really do. I was in a horrible company doing this until I came to my senses. Thank goodness I got out when I did. (Un)fortunatly for the company it sounds like you're an honest guy. Remember that a SysAdmin who's watchers are cluless appears the magician - if (s)he has quick enough hands.
Your boss is really asking you to report what you actually do. If you can't tell him that, your answer is that you don't do anything.
It's scary, but it may also be necessary.
Your boss is probably being pushed into this.
You need to find out what you can measure. Some suggestions,
1. Try keeping a journal recording your time for a couple of days. What takes up your time?
2. Try to write down what your responsibilities are. Make a list. It may be a long list.
3. Look at your to do list. (You really should have one.) Look at old lists, keep them for a while.
4. Look at your phone log. How many of those were trouble calls. How many were resolved? You do have a phone log, right?
A couple of things to remember. IT is not a profit center, it is a cost center in most organizations. Your benefit may be more in how you get out of the way, than how many things you can control. The best metrics may be benefit/cost. Make the benefits high, and the costs as low as possible. Your contribution is best done without dominating or controlling others. If you can do that and still contribute, you are doing your job. Now you just need to find a way to demonstrate that. You boss needs to do that for the department too. That's why he (she?) is asking for the information.
One last thing, what is important here is not how you view things, it is how your bosses bosses view them. If you want to find out what that difference means, talk to some of the people who are in the major profit centers of your organization. Those folks are usually in manufacturing or sales. Everything else is overhead. You may find that they don't have a very good opinion of your group. A lot of IT groups have a reputation for empire building, and a lot of self aggrandizement. Many are real big on their own systems, and not interested in others problems. If that is the case for you, then your boss may have orders to shake things up. Then you might be in for some major job changes. The best way to survive that kind of thing is to find what is really important to the organization, then make sure you are supporting that function.
Good luck. If you can weather the changes, you will be better for it.
Everybody knows 3 people with my name.
It's been said before, but the realm isn't producing anything.
Systems administrators are managers. They are purely overhead.
Your boss is the same thing. This is what people managers don't understand.
So we manage computer and systems the same as they manage people. Well not the same, because our systems aren't deciding to surf youtube instead of working.
The point is, your boss needs to change his thinking. We manage resources.
Even if there has to be something quantizable, it could be uptime. Uptime starting at 100% and working backwards.
Mostly there is really nothing to produce. We are managers. We are overhead. This is why so many people managers have a hard time with us. They can't grasp the concept of managing a manager who doesn't produce results.
Actually the results are in an inverse vacuum. The results are good if nothing happens. The results are always negative if there are events. People managers can't understand this.
Downtime is when the team is necessary to management.
Monitoring of the system doesn't sound like work to a people manager.
If nothing happens, they don't need you.
Not realizing that everything works because of you not in spite of you.
They're using their grammar skills there.
# of systems and pieces of software requiring sysadmin attention * hours in a day - downtime in those systems
If you guys do your job so that trouble tickets never happen, you get credit for it. If you handle trouble tickets quickly, you get credit for it.
What better way of tracking your sysadmin efficiency than how many systems do they administer without them going down?
Instead of leaving for a week, spend a week documenting what actually happened (or didn't) because you were there and imagining what else there is that could or could not happen due to your absence. After making your lists, categorize into kinds of services that have value and extrapolate from there. Now you have a list of what valued services you provide and a start of a more detailed way to categorize events and successes into what service they belong.
For example:
1. Server #xx load spike, killed rogue process
2. Migrate dept #xx to using server #yy for applications
3. New hire added to servers
4. Rewrote procedure #xx for clarity
5. UPS test
6. Investigated report of fan noise
7. Tracked down rogue wireless access point
Server maintenance 1, 2, 3, 6
Security 2, 4, 5, 7
User services 2, 3, 6
OK, now there are some tasks that overlap, either divide their time, or better split into two distinct jobs and assign each part of the task according to how much it belongs in each category.
When I did this, I found that with a requirement of too much detail I was averaging about 300 tasks per week, and obviously that made for a large amount of overhead keeping track of them with such fine granularity. I asked for, and eventually got, the ability to just account for tasks taking over a half hour with that much detail and assigning all the other time proportional to how many people there were in each area.
YMMV - HTH
got a lisence => got a taxi license
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
You pose an excellent question and the industry does need an exact measure to gauge one sysadmin against another. Uptime is obviously a good metric. I think you can break it down into Individual and collective uptime as well. If your server is slow, everyone is slow. It may be up but it sucks.
A good sysadmin has 99% uptime on servers and 99% uptime on individuals. A good sysadmin costs $90,000 in the USA. A bad sysadmin has 80% uptime and costs $40,000 in the USA. You can pay less for a sysadmin but that's like asking your neighbor's kid to rebuild the engine on your mercedes because he can do oil changes.
I like to think of my profession, System Administrator, like that of a car mechanic. It helps put it into perspective for non-computer people. You can think of replacing your server like replacing your engine. How much is that worth? Who would you trust to do that? Would you take your Merceds to Wal-Mart for an engine rebuild? Equivalently, would you take your server to Best-Buy for an upgrade? Hell NO!! You're going to get the best mechanic you can.
Seriously people, it's time we organize and form a computer workers union. We can set standards so script kiddies from India don't come in and screw up our voting machines.
Frequency and detail of log review is an excellent measurement of what kind of shape the system is in. It has direct bearing on the length of time before problems are discovered, and therefore the stability of the overall system. It's hard to determine, of course, whether admins actually look at the logs. But if you can find some way to verify that log review is occurring, and estimate the amount of time put into it, I think that would be a pretty good way to address the tricky problem of quantifying the effort that goes into maintaining systems.
In addition to regular maintenence, I would include some numbers relating to how many systems were set up and appropriately retired, and how often OS/application updates occurred.
Whether or not the sysadmins are doing their job, it will show.
Otherwise, measuring one point of the organization has a tendency to skew work at that point toward a favorable measure. For example, measuring # of tickets closed within n hours tends to lead to all tickets being closed within n hours, regardless of problem status. (I have witnessed this as a customer. It's not nice.)
In other words, you get what you measure. Thus, if you get what you measure, you might just as well measure the thing you actually want, causing the work to skew in that direction...
Usage: km/h for speed (kilometers per hour); kph for very slow impulses (kilopond hours).
System Uptime would surely be a good measure for day to day.
Visit TheHavenNet [ http://thehavennet.org.uk ]
mentioned at least by reference several times already:
Estimated volume of new spam being avoided (could be attached to the actual volume of spam hitting the filters).
Estimated number of malware infections prevented.
Estimated avoided hours of user downtime.
I think the negative list could continue quite a ways. The first or second would probably get an "oh, get real" kind of response, but if you line up enough of the (real) consequences of you not doing your job, the value should become apparent eventually.
Except in the worst cases. Let's hope your PHBeanCounter is not one of those.
Use /dev/random or ask the slashdot community? /dev/random=slashdot_community
Haven't you noticed the little random integers that go with each posting. Duh!
He who said 1,000,000 monkeys on 1,000,000 typewriters would eventually type the great novel, never saw an AOL chat room
If the sysadmin is supposed to make everything run smoothly, then
it follows that the fraction of idle hours (or hours spent
playing killbill) per day is a measure of your productivity.
You're clearly producing a smoothly running system environment.
Is there a metric for over micro-management to actual work getting done?
Measuring programmer output in KLOCs is broken (who want's 1000 lines of bad code??). Bugs fixed is a bad measure (who put those bugs in there anyway??). The dummies often get to fix lots of silly/obvious one-liners while the better programmers will tackle the more complex/subtle problems. The best will be generating very few bugs.
It is wise to think of how you could justify your existence if the CEO stopped you in the hallway and asked you why the company should be employing you.
Anything activity that ends up on the cost side of the balance sheet is very difficult to measure. It is a bit easier for programmers because at least there is some tie between product features and sales: "I implemented feature abc or fixed bug xyz that resulted in $2million more revenue".
For pure service sectors (like IT) it is hard to link actions to revenue. About the best you can do is to measure trends or compare them against some industry benchmarks: "We had 1 virus event last year, industry average for comparable organisations was 5 events", "Our average time to restore a backup is x vs y"...
Engineering is the art of compromise.
User satisfaction. This would probably involve periodic surveys or something. This would cover almost everything. Of course, you do have a problem when the users are 'dissatisfied' - you dont know what the problem is!
Regards, Martin IT: http://methodsupport.com Personal: http://thereisnoend.org
I seem to recall that it used to be said that the sign of a productive UNIX admin was that they sat there doing nothing most of the time, whilst all the systems were running fine.
It's flippant and probably not what a boss wants to hear, but it always amused me. Maybe you should just measure the number of /. stories read in a day? That's what I do with my idle time at work anyway :)
Yeah, I had a sig once; I got bored of it.
I'd measure downtime. As an admin, your value is in ensuring the other 10, 100, or 1000+ people that rely on your systems can work without interruption. It's pretty easy to quantify the cost in man hours of 1 hour of downtime multiplied by the company's average hourly salary. Not to mention how much lost revenue could be involved. The easy summary of this measurement, is from Ben Franklin, "A penny saved is a penny earned."
The metric should be 'number of times the sysadmin has to be consulted', and it should be driven as close to zero as possible.
I might get moded 'funny', or 'flamebait', but I'm serious.
Think about it. When is a sysadmin needed? When there is some kind of crisis. "I can't get to the internet", "I can't check m email", "My computer thinks I might have won a million dollars", "I lost that important project file". A good sysadmin will prevent these things from ever happening, and when they do happen will have them resolved quickly, without a lot of technobabble or attitude (like the SNL skit guy), and will fade into the woodwork. Ironically, the middle-of-the-road IT guys are often thought of as heroes by the staff they support. They might be thought of as the firefighters, but unfortunately, they are also often the pyromaniacs.
Other useful metrics:
If you don't already have a ticket support system, get one. It will generate useful metrics for you. Some useful things out of it would be:
- The AGE of the OLDEST OPEN SUPPORT TICKET. Proves you aren't dilly-dallying
- Number of Priority 1 Tickets opened per quarter (see above - should be as low as possible)
- Everything you do, you should open a ticket for. Upgrading that linux box? Ticket it. Updating anti-virus definitions? ticket it. From this you will get:
- Nunber of tickets open per day
- Nunber of proactive vs. reactive tickets (tickets you opened vs. someone else opened. You should get credit for fixing things before they become an issue someone notices.
And if the bean counter needs some big numbers to justify things, just count up stuff that the logs on public boxes find. Seriously - have you ever looked at the stuff from logwatch? Just yesterday I had 2163 unique failed attempts to log in as root, not to mention all of the other assorted hackery it catches. "Number of successfully defended intrusion attempts" is a metric that will scare a bean counter enough that he won't take the liability of getting rid of you.
Hint: 'due diligence' isn't posing the question to the 'slashdot community'.
Lie. =)
Depending on your exact position in the company, you may (or may not) be an irreplaceable asset of your company.
Given my own job, the best meassurement of "value to the company" is the amount of co-workers and bosses of other departments who would stand up and protest if your department would try to fire you. ALWAYS REMEMBER, the other departments are customers of the IT department, which gives them quite a power over your boss (because THEY grant him his budget). You should ask THEM to provide you with the required meassurements.
For example, make an official poll as how satisfied they are with your work, what are the strong and weak points of the service you are offering etc. That will give you (and your boss) the best feedback, how smooth your operation runs, where more work should be invested and what your departments top priorities should be in the future.
Look, this thing is totally safe! Built it myself, you know. You just press that button like this and then turn that lev
I think it is very important to measure your IT departments performance as a whole. The reason is simple - you want to know how to do better. You will also want to track what individuals do, to make sure work is divided in a good way.
Some examples of what to do:
How many problem tickets do you have? How long does it take before they are answered? Who answers them?
People does time reporting. How much time is spent on the different systems, doing support, doing maintenance?
You should have a list of long term projects. Are you solving them? Who solves them?
You should also document what you do. Who is active doing documentation?
Use your surveillance system (nagios or whatever) to get data for uptime of machines and important services.
As another measure, you could let the people you are supporting rate your work.
For the group you have some quantified measures in the statistics for the tickets and the long term projects that are solved. If you have gotten data from the people you support, that is still another measure, as well as uptime.
More interesting internally for the group is how different people spend their time. Probably you want an even distribution of support and maintenance work between the people. If someone is doing lots of maintenance work, but haven't written much in the documentation, it probably means that person needs to document his or her work more.
How long the system (subsystems) were down.
How fast was the response time to eliminate a problem.
How many problems were there.
Which business unit was responsible for the problem.
(for example, if a HD gets full it could be that a user stored his porn collection on it - user error, or simply someone bought a too small HD).
Patents Drive Free Software as Hurricanes Drive Construction Industry
For every service that runs on any of the systems you administer, you graph its availabile (and, if applicable, unavailable) time as a pie chart. If you want bar charts, measure the number of seconds of continuous availability since the last service interruption on a logarithmic scale -- for each service. That is, after all, the primary job of a *nix admin: to make sure everything continues to run smoothly.
Note that availability can be different from uptime. If the network cable is unplugged, the system is unavailable even if it continues running. Conversely, if you use redundancy and failover and so forth (which I'd think you would in a Fortune 500 environment) you can shut down a server for upgrades without having any detrimental effect on uptime.
Come to think of it, that also works for graphing the availability of system administrators. Any time at least one of you is on the clock counts as available time for the system administrators' service, but if you all leave at once for any reason, that's unavailable time.
Cut that out, or I will ship you to Norilsk in a box.
Try to get him to understand that some deliverables are 'negative' deliverables. Uptime (lack of downtime) or security (lack of intrusions) are good examples. They are partly the expression of your due diligence, good practices, savoir faire, and flair. These will never be piechart-able. If he does not understand that or does not want to understand it, pack away and get working somewhere they deserve you better. A job is not just an exchange of money for work. You have to get some consideration and self-fulfillment out of it.
Ticketing systems are great for quantizing what people do, how often, how fast, etc as has been mentioned. Having a target for availability and meeting or exceeding it being tied to performance reviews also works. Then again, when people question a need for me, I go on vacation. I turn my phone off, never check email, etc. When I get back and they're crying that they're so happy to see me, I guess they learned what my value is as someone nagged them instead of me when something broke. I've also heard mention of a well architected system (app to network, etc as I am a network focused engineer that covers interaction of apps and servers as well) need less immediate attention as things are redundant. I reply many times to such questioning (though it happens less and less the longer I stay at any job as they get used to how and what I do) that the thought put into the architecture in the beginning is why theres less pain now. Make a list of things you've done, explain them. Think of yourself as a product, the manager needs to be convinced to make the purchase, same with interviewing. Or, get a job where the management staff are more technical and don't ask stupid questions, I need a good unix admin, send me a resume. ;)
You know, some ship captains can guide their ships to dock without an on-shore pilot, but they generally don't, because they have the good sense (mostly) to avoid risking their gear in unfamiliar waters. A good business captain should heed the example.
Do not mock my vision of impractical footwear
...that if the bean-counters a requiring performance metrics, then a percentage of your team will soon be destined for a meeting with the downsize axe. What starts out as an innocent request for innocuous metrics inevitably results in people losing their jobs. It's time you make sure your resume is up to date just to be safe.
My mom always said, "Jim, you're 1 in a million." Given the current population, there are 7000 of me. God help us all!
I am a systems programmer for a mainframe system which is basically the same thing as a sysadmin for *nix or Windows systems. We have been asked before about such metrics (although the PHB seems to have been distracted by other things for a while).
I haven't read all of the replies, so this may have been covered, but when we were thinking about this, all of our best metrics were "negative" ones with zero being the ideal number for the metric:
Number of times the system failed during 'prime time'
Number of times users were unable to connect to the system during published hours of availability
Number of times backups failed to run
Number of restore requests which failed due to a bad backup
Number of intervals where response time for application 'X' exceeded the 'good' value
Etcetera. Our jobs as administrators of systems used by others is to ensure that bad things do not happen, therefore it seems logical to measure or count the bad things, and report on those. When we're doing our jobs well, no one should notice us.
As a lot of people have noted here, how a sysadmin's job is usually very transparent because they are doing their job when things are not broken. Sometimes there are servers to install, updates to apply, etc, and those are easy enough to track. However I think uptime is the #1 way to show that people are doing their job.
In my case, I prepare regular reports for the PHBs showing helpdesk requests, helpdesk response time, backup results, SMTP traffic, and most importantly, a very large month-by-month uptime graph on the first page.
In my case, at salary review time, I can plot our uptime percentage from a few months prior to me "taking over" to present, draw an arrow showing where I took over, and show that our uptime percentage has gone up significantly since then. The data can be just as valuable to you as it is for the PHB.
(and no, I'm not tweaking the numbers!)
I'm an Oracle DBA, not an SA, but I deal with SAs daily. My concept of a good SA is one who has it covered, with alarms and proactive procedures already in place, just like a good Oracle DBA should. Some years ago I was on contract to an eCommerce company. I set up alarming so well and put in place fix-it scipts, plus logged in remotely even at odd times of the night, catching developers doing bad things and sending them email. Eventually the boss told me that he didn't need to see my face anymore. I was giving him 5 9s and that was good enough for him. He said I could come to the office when I wanted, or stay at home. Recently I've seen a rash of metrics come into the SA and Oracle Administration area. Frankly, I think it's stupid. I was given my freedom to just come in for my consulting fee by the boss some years ago not because I fixed problems, but because I set up an environment in which there weren't any. The problem with metrics is that if you work of tickets, you don't have the incentive to just browse and look where there might be trouble spots brewing. In effect, you're forced to let things fall apart then get credit to fix what could have been prevented.
I agree with the list suggested by metlin (258108) and augmented by dubl-u (51156) and "Anonymous Coward on Saturday August 25, @10:41PM":
Stability
Reliability
Response time
Network load
Security
Efficiency
User satisfaction
Crisis preparedness
I would add, separate from "Response Time [to problems]", some measure of how change requests have been handled, both in responsiveness and quantity. A significant portion of our sysadmin time is spent on bringing new servers or applications online, or adding new features or upgrades.
Many of the responses I read seemed to assume a fairly static configurations, which doesn't seem realistic to me. (Or maybe such changes are handled by other staff in their organization).
-dd/
Their productivity metric is really about the real gains they achieve for others, not how much they are doing to realize said gains.
I would turn the tables on this --and quick. Essentially, they have no idea what "work" for a sysadmin is. They know they need it, but they don't know how much of it they need, or if the people they have doing it are actually delivering as well as they could be.
Where bottlenecks are identified, along with inconsistent behavior, admins apply information technology to the problem, with the goal of eliminating them. Why are they not all eliminated? Cost and time are the two biggest contributors. Nearly anything can be done, but it will either cost a ton, or take a long time, thus reducing the potential for other gains to be realized and that's the balance right there.
If they understand that dynamic, real conversations can be had about what makes sense and what does not. Having established that, IT is then in a position where they contribute to the management of the enterprise as a whole.
Either one understands these things, or one does not. It is not possible to control a dynamic that is not understood.
So, this bozo is down to either not managing or he needs to develop a trust relationship that enables the kinds of conversations necessary for the dynamic to work.
Essentially, he's asking you to package this up and present it in some neutral way, so he can avoid trust. Of course, he's a complete fool! Lack of understanding means he cannot verify what he is given and therefore cannot make reasoned, rational judgments about it.
He could being up accountability. That's a simple one. IT is accountable to those it serves. It also is accountable to those providing the approved technology. There is no free lunch.
Buy the wrong tech, end up with hard working admins who are not empowered to realize the gains promised. Not their fault, unless they were involved with selection and the trust relationship, mentioned above, is leveraged toward that goal.
Short story: You guys are happy to work with him, but there are boundaries and needs.
You all have the shared goal of a healthy enterprise that pays your way. Start from there and get this guys trust. This will avoid all the silly reporting that just ends up being a drain on innovation and process improvement / troubleshooting. It will also open the door for more business information your crew can use to make solid decisions about what gets done, how much it costs, and the ramifications. That's what they really need, not some silly metric that says you were surfing too much, by percentage, last week.
There is one other thing this guy needs to know. Sysadmin is a self correcting affair, for the most part. If too much time is not spent applying brains and tech to process and people problems, then the admins get buried under a growing pile of crap that makes their lives hell. All admins want to have time to be proactive, not reactive. (all the smart ones do anyway) Being reactive leads to a hostile user environment and that's just not cool! Cuts into the UTube time!
What he suggests is reactive, and the start of a bad cycle that will seriously impact the company over time. Your job is to prevent that crap from happening, which justifies the conversation in the first place!
Shake the geek image for a bit and win this guy over. It will be one of the very best working relationships he has ever had. Do not let him pretend he can manage a dynamic he does not understand. Do let him know rational goals, discussed and agreed to, can and will regularly be met. And there is his productivity dynamic. If your crew says what they can do and for how much time and cost, then does what they said, he's got zero worries.
Seeing that is not a spreadsheet, but an ongoing conversation that communicates the state of those things on a level both parties find trustworthy.
Having said that, I've seen plenty of these broken metrics. If he cannot be
Blogging because I can...
Step 1: Update your resume
Step 2: Ask him to prioritize HIS responsibilities as he sees them, then break them down into measurable units of work. Then he can do the same for yours.
Step 3: Point out to the beancounter that you're a professional, not a widget maker, and that you manage systems. And as he well knows (from step 2) it is difficult to quantify and measure management in an empirical fashion.
I got asked a very similar question by the head of the department I was working for one time, and I told him that truthfully there isn't an effective way. I gave him two suggestions: 1) if the users are able to do their jobs with the technology being an asset to productivity and 2) there aren't any issues he's hearing about then I must be doing something right. He was satisfied with that.
-- "Never underestimate the power of human stupidity." - R.A.H.
Graph from /dev/urandom instead. It won't make any difference to the bossman's pie charts but you won't consume as much precious entropy :-)
"'I pass the test,' she said. 'I will diminish, and go into the West, and remain Galadriel.'"
- JRR Tolkien.
How do you measure the productivity of an intercept pilot? You can't. Productivity is the wrong paradigm. You need a SysAdmin to be AVAILABLE and effective.
SysAdmins are the intercept pilots of the IT world. We often sit on the runway, engine at idle, fueled up (on caffeine) and ready to launch if something needs our attention.
Imagine if the Air Force tried to measure the productivity of intercept pilots. Your PHB is attempting the equivalent.
Fortune 150. There is Fortune 500, and Fortune 1000.
If you are looking for an analogue to KLOCS, a meaningless metric, from software engineering, consider
collecting the average number of characters typed per second per admin.
Additionally, if management were to adopt this metric, it would provide justification for getting rid of all
of the Windows (tm) administrators.
-- jbs36
Is it possible TheMCP even figured out the source of an answer above?
... rate it on a value scale. Nuisance class fixes "improve overall staff effectiveness". So - install the Java update for them, get that weird anti-virus directory error to go away, find someone's lost file. The value provided is something like the number of times the user doesn't have to deal with the nuisance (that AV error on bootup), multiplied by the time they lose *stewing about it* (politely: disrupting their workflow concentration).
Maybe the IT Admin does have a metric - Lost Costs Avoided.
Starting especially with low-level user nuisances... instead of funneling "Mgt is mean", log both the original user request, its resolution, and
If your project log contains some 100 of these a month, that's 100 cases of someone not staring at the screen (or talking at the office fridge) about the glitch.
For higher level fixes, estimate the lost productivity when the major data server goes down, and then use a cost-prevented calculation to show you got it back up in 2 hours instead of five.
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
If everything is ticking along nicely then a manager is doing his job.
A sysadmin is a manager who manages machines instead of people.
Once he understands you're "management" then all your problems are over...!
No sig today...
Part of the problem is that the sysadmin job is somewhat reactive (like the plumber who responds to problems), somewhat preventative (like the security guard keeping the bad guys out), and somewhat prescriptive (like the carpenter adding on another 20000 SF of building). Try to divide the general role into these different categories and come up with metrics for each. Coming up with a single metric will be nearly impossible because of the diversity of the responsibilities of the job.
Find other jobs that have similar, "preventing the negative" jobs. How would you measure the security guard's efficacy?
Say to the guy: Look, I've never been in this situation before, but I do think that I would be the best person to ask. Let's work out a system that we can tune over time; say we get together every quarter to assess the effectiveness of our little system. Let's start with the quality of ticket resolving and outage and work from there. I realize that you would like to have a solution ready to present to whomever could fire you yesterday, but that's just not going to happen. Let's make it our goal to have evolved relatively fine tuned metrics ready in a year's time. In the mean time, give me half a day out of every week to develop it.
Religion is what happens when nature strikes and groupthink goes wrong.
Now ZEN, that was a computer.
Sorry about the writing. Robot fingers, you know? Cliff Steele in DOOM PATROL #23
Honestly, There is only one reason a computer, a network, a developer, or a Sys Admin exists and it is to provide applications to the end-users. Look at the output and I'm sure you will be able to come up with some metrics - application uptime, application response time, application throughput, number of users, amount of storage, etc. Good luck, Netperformance
. . . it's the productivity of others that a Sys Admin enables.
You need to show how productive the rest of the workers are thanks to your efforts.
You can do this two ways:
o negative: "when the Order Entry System fails, we lose $XX,XXX per hour"
o positive: "if we improve the Order Entry System, we gain $xx,xxx"
That's a nice bean counting machine you've got there. It'd be a real shame if it were to develop a sudden system failure, and oh my.. it looks like the backups have inexplicably failed as well, but just yours. What are the odds?
https://www.eff.org/https-everywhere
The sysad chooses and uses the tools, and can claim some value in being able to choose wisely and wield skillfully.
Particularly since wise choice requires ongoing research and skill requires refinement. Barracudas only operate themselves for a little while, and Symantec, of course, is not a wise choice for many shops. If a sysad uses Symantec, he or she must keep up-to-date on whether the shop is running stuff hackers want in badly enough to take the (diminishing) extra effort to get the jump on Symantec.
A wise choice for antivirus and/or firewall requires a lot of continuing support activity in the present internet environment.
joudanzuki
Go on vacation without replacements, and measure how much money the company loses. Hey, the bottom line is the bottom line.
Admins are like firemen (or firewomen) - they prevent and recover from damage more than they contribute directly to production. They spin plates.
This is not to say that their task isn't vital; Just look at how much the rest of the company could produce without admins. It gets this way because everyone wants, needs, and uses them to the fullest possible capacity, making business without admins uncompetitive.
If they want metrics, give 'em metrics. And let them know that metrics will only encourage you to be more of an a$$hole.
Management, particularly bean counters, will not care that you have become more of an asshole unless it somehow negatively affects morale. So if you implement a bunch of stuff, then you'll be forced to do lots of pointless asshole things rather than do your job, because otherwise your metrics go down and then you're in trouble.
Best not to be judged by a system you specifically set up for failure. Especially if management doesn't see it as such.
Karma: Non-Heinous
.
One dosen't. Many do. One person cannot show how their efforts in a cooperative environment contribute to overall improvement in up time and productivity in any meaningful way. System uptime is a colbrative effort of many people from network admins, to technicians, to system admins, to the users themselves. Requiring a personal identifying moment where everything would fail without you is both easy and hard.
The best response in this case is to update your resume and put it on line for other employers. You are locked in a death march, and only delay keeps you employed. You are not being appreciated for the various contributions you make, you are being subjected to review from people that cannot possibally have any iota of understanding of your talant. It's time to kiss your current employer goodbye and find somewhere else where your skill and efforts will be, if not appreciated, at least assumed.
Any company that puts a bean counter in charge of employment decisions for management or technical fields is simply looking for a way to remove payroll debits. Leave, and leave as quickly as possible. Your talant, skill, and dedication are not at issue. Only how much you cost the company to employ is. It doesn't matter what you contribute, only your cost is seen. Your contributions to the company are at best assumed, most commonly disregarded or ignored. At this point, however unwilling the company is to see it, they need you far more than you need a paycheck from them. Find someplace that will value your contribution to the corporate effort rather than regard you as a cost center without benifit.
Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves.
Here's a thought ... Why don't you supply your boss with what he is asking rather than what you thing he is asking. If you want to feed him /dev/random, remember, garbage in garbage out. After all he is supposedly trained in MANAGEMENT and able to translate the highly technical information (which you as a technical expert would provide) to your upper management, who may not even know you exist. Sounds like a good fit to me.
A sysadmin's job is productive when you don't need to do the things that show you've not been productive. That's an awkward wayto say something that most admins already understand, so let me explain.
If you're researching new software, running performance numbers on new hardware before putting it in production, testing boxes, upgrading systems, patching systems, monitoring systems, rotating logs, etc, you're generally doing a pretty good job.
If you're fixing problems in production after they happen, coming in to the beeper because you don't have sufficient fail-over to last until morning, yelling at your hardware vendor that you need your expedited shipping even more expedited because your system utilization outstripped your expectations and your fudge factor, and basically "fighting fires" and reacting to problems rather than planning for them and preventing them, you could do better.
Measuring productivity of a sysadmin is a somewhat silly concept, really, because a sysadmin doesn't produce anything. A sysadmin is a manager (used to often be called "system manager" in fact), and "administrator" should be a sign of that. The fact that your resources are technological and not employees makes no difference. It is not your job for you to produce anything nor to be productive in any way.
Your job as a sysadmin is to make sure those you manage are productive. Hence, "production systems".
You could base your metrics on how many machine hours at what utilization you're getting out of your hardware and software. One way is the "useful utilization" metric. If you have ten production systems load balanced and one testing system that represents that configuration in tests, and you're running 90% utilization of the most limiting resource on those systems (CPU, memory, disk I/O, etc), then you're getting 90% productivity out of production servers and 81% productivity overall. Try to find another department in your company that runs at 90% average theoretical efficiency. If half of your systems are down on any given day of a week for four hours apiece for one reason or another, that's 4 * 5, or 20 hours out of your 168 * 10, or 1680 hours, times 0.9 (for the 90% utilization). That brings you down to about 87% on the production servers.
Another good metric I suppose is client requests served per dollar (yen, yuan, peso...) of hardware, software, and power investment. If you can figure out a dollar value of the average client request in some way, that can even point you towards ROI for the systems. A good sysadmin has a higher ROI from his systems than a poor one given the same hardware and software, after all.
I guess you could try to measure the admin by hours spent in proactive work vs. hours spent in reactive work. A good ticketing system could make this pretty trivial to track. However, if the standard ticket resolution is to power cycle a whole rack, this makes a very bad sysadmin look stellar. You'd also have to have a ticket queue and resolution tracking on research, upgrades, patches, server purchases, requisition writing, and other things which typically don't get tracked as closely as problem reports.
We have a VP that is somewhat of a bean-counter. I swear, he gets most of his ideas from Management 101 for Dummies and not from real experience dealing with technology professionals. My belief is that if the systems continue to run with little or no interruption to business then by all means the admins have been productive. It is better to be an enabler thus allowing your staff to discover techniques and/or technologies that will improve the environment rather than stifle those admins to where their knowledge disintegrates or where they would rather find another job.
Ideally you want to measure two things:
1. How much the system is theoretically capable of contributing to the company's bottom line, in increased revenues or reduced costs, if it always ran perfectly.
2. How much the system actually contributed to the company's bottom line.
#1 tells you how important the admin's job is and #2/#1 tells you how well he's doing it.
I've been on both sides of the fence on this one. I can appreciate your resistance--when I was a network admin, I hated this kind of management 'interference', but now, looking back, I realize what a help it would have been if we had good clear measures of what we were supposed to be working for.
/dev/random, how would you feel about the vendor behind that?
Coming up with performance metrics is a real challenge, but if you approach it right, can force you to take a fresh look at what you're doing, and, in the best case, help you (re)focus your efforts in much more efficient ways. Think of it this way: how would you optimize a server/network if you didn't have good numbers on how the various parts of the system were working? If you found out that one component wasn't really giving you good data, but was piping
Don't treat this as some sort of spy program to make sure you're not loafing on company time. Really think about "Why are they paying me?" and "What does it mean for us to be doing a good job?" and try to capture that in your metrics.
In contrast to a few of the posters here, I'd recommend being sure you come up with metrics that measure outcomes, not inputs or outputs. That is, try to measure the things that (1) matter to the organization outside of your department and (2) are controllable by you. The easiest trap is to measure outputs instead of outcomes. You don't want to report on how much time/effort/money you spend on things, you want to report on what they get for that expenditure; the Maytag repairman should have good performance metrics. One example is the famous Dilbert cartoon with the punch line "I'm gonna write myself a new minivan this afternoon." Another is the mythical Cuban bicycle factor that had a quota expressed in tonnes/month--they just decided to make 80 pound bicycles.
So, time spent on server maintenance is a bad metric. Server crashes/month is OK. User downtime is better, and dollars lost due to server crashes is probably best. Similarly, number of tickets resolved per month is bad--there's no way to tell if that's because you're efficient, because you're deploying flakey systems, or because you're just blowing off users and calling the tickets resolved. (Much of the behavior of telephone support workers can be explained by them having bad performance metrics.). A better measure is, as one poster suggested, age of oldest outstanding ticket. Even better is some measure of 'customer' satisfaction.
If your company has good IT policies, compliance with them can be very important metrics. Can you report '% of data backed up according to backup policy' or '% of systems in compliance with security policies?' Yes, those will require you to put auditing procedures in place to generate the numbers, but those are very useful numbers to have. If you couldn't produce either of those two numbers, you really can't say you're running backups or have secure systems.
Avoid strictly technical metrics. Nobody cares what your average CPU utilization or network congestion is. People do care about how long it takes their screens to come up. Even better would be to measure how happy they are with application performance.
I know this comes across as obnoxious management bean-counting, but try to understand what they want to accomplish and go with it. It'll be better both for your systems and your career. As a sysadmin, it's way too easy to get swamped by the daily firefighting and forget why the systems you're maintaining are there, and what the company as a whole is getting out of your efforts. Good performance metrics measure how what you do every day relates to the company making money--and honestly, if you can't explain that, why are they paying you?
Just say, "I've searched all over and could not find any decent metrics for sys admin. It appears to be difficult to measure, partly because a lot of it is more art than science and there are multiple ways to do the same thing, with complex pros and cons. If you can find some good metrics, I would like to evaluate them, but so far I am stumped."
Table-ized A.I.
As I am old, experience tells me you'd be better served to spend your remaining time in that job getting your resumé up to date, pick up some .not books and apply for every course you can get on.
/dev/random, it won't make any difference anyways and you'll only end up in therapy trying to generate 'meaningful' numbers for your boss, because if they don't tell you how to measure it, everything will be 'not what they wanted'.
Feed the bean counter with numbers from
Sorry to bring you the bad news an'all.
threadeds blog
Stuff that didn't go wrong.
Everything worked today? The sysadmins were 100% productive.
To boldly use to and too two times and get it right too! They're not gonna believe their eyes when they see it there!
would provide you an excellent score
Say you have 20 servers and the company generates $114/hr ($1 million/yr company) [I'm going purely on 24x7x365 not business days] then the above would generate:
(60*24*365) * 20 * 114 = 1,198,368,000 server dollar revenue you generate
Now subtract any downtime you might have multiplied by the same $$/hour rate. Say you had a total of 5 minutes downtime on each server over the past year (doubtful on *nix, but just for sake of argument), so that's 100 minutes * (114/hr/60) or 100 * 1.9 = $190 lost to downtime.
If someone asks for stupid numbers, give them stupid numbers.
I've heard that doctors in China are paid when the patient is well, rather that when they are sick.
Systems Administration is much the same thing.
The best admins do things right the first time and aren't suprefically busy all of the time.
Poor admins appear busy all of the time.
I'm not sure how to convey the essence of this to a PHB,
Competition Good, Monopoly Bad.
One could say that silence is golden. Pre-empting the pitfalls - doing your job properly - does indeed have the downside of making you mostly invisible. So while the sloppy w**** admins stay in spotlights constantly repairing things that were broken from day one, the *ix admins who actually know how to do things may seem to be doing nothing most of the time.
So what should the bored admins do when they've automated more or less everything? Simulate catastrophes. Have a red team if you can. Doublecheck how much faster your emergency reimaging scripts are compared to OS reinstall, etc.
From my own experience, the people who do the best work are the ones who have better ways to use their time than to keep track of "work done" metrics. Only the slackers, who are great at exaggerating what they do, are the ones with the unused time to make up such 'metrics' and skew the facts in their own favor.
... how to make up bullshit metrics yet, that actually have some basis in reality, you haven't been working as a sysadmin long enough.
Most long-time sysadmins can create mountains of "metrics" that actually look useful, that really aren't.
The really good admins not only have "metrics" but automated systems to create those metrics. (e.g. The bean-counter will be very happy that you've port-scanned all the external systems THREE times this month instead of the usual two, right around review time. Of course, your port-scanning script fires up nmap and nessus, does its thing, and generates a pretty PDF and stashes it somewhere you can retrieve it every time it runs from cron.)
At the end of the day, the only thing that matters is:
- Did I do work worth what the company paid me today?
And:
- Did I generate or save revenue for the company today?
Bonus points for the last one being somewhere around 6 figures or more.
Document THAT for your next review so they can offer you a piddly 3% increase while the "leadership" cashes out millions worth of stock options, and then lays you off when the stock price plummets for a single quarter.
+++OK ATH