Measuring Coder Performance?

An Anonymous Coward asks: "Our company is a small web development firm that focuses on dynamic and database-driven websites. We are thinking about creating a bonus/recognition program for our coders based on their performance. The question is, how exactly does one measure programmer performance? We don't want to use the number of lines of code per day (why encourage 10 lines when 5 will do?). We don't want to use the time it took to finish the project, since lack of bugs, speed, security, etc. is just as important as time. Has anyone had an experience with a measurement system that they thought was a fair and accurate measure of the quality and speed of coding?" While I'm sure some standard can be applied to answer this question, I don't see it being extremely accurate. Some things just can't be quantified. I'd be interested in knowing if you all think this is one of them.

Comments

Peer Review

[Anonymous Coward]

You are looking for an objective way to measure a subjective thing. Good luck.

My advice: fuggedaboudit. Use peer review. Allow developers to nominate each other for recognition.

That's the only kind of recognition that counts, anyway.

You said you're a SMALL company

[geophile]

As another writer said, peer review is definitely the way to go. If you were a large company and you had a lot to automate and justify, then maybe you would have to turn to metrics, but you don't.

Look at it this way: If your metrics contradict what your small band of hackers know is right, your hackers will be unhappy and quit. In other words, metrics would be, at best, as good as peer review, and at worst, they might do a lot of damage.

Thoughts

[dmorin]

Decide what behaviors you want to reward. Examples:

• Peer recognition. You want people who are respected by their teammates.
• Role model. Do you have certain processes or paperwork that need to get done as part of projects? Do you have some people that do all the paperwork, and others that always skip it?
• Being well rounded. Recognition often goes to coders who also have the communication skills to talk to the customer. Or who can write up a proposal. Or run a brown-bag lunch on a new tech topic for their teammates.
• Hitting deadlines is good, don't blow it off because it might encourage bad code. If your process is in place then sloppy code won't pass your QA and your developers will know this. They need to hit the deadline AND pass inspection.

Reading list; Re: Peer Review

[gbnewby]

Read Fred Brook's "The Mythical Man Month." Read Beck's "Extreme Programming." Then, turn around and read Thompson's (1917) "Scientific Management."

The lesson learned is that programming is more like art than like an industry. "Regular" measures of productivity don't apply well, nor do standard reward systems. I agree with AC: take a page from sourceforge.net and other locations, and implement peer-review. Bug tracking etc. are other approaches.

The biggest danger I see is turning the reward system into a popularity contest. Careful that the evaluation measures are consistent with what you want (e.g., if there's no customer interaction, then things like working hours and dress shouldn't play a role. But if there IS customer interaction... get the idea?).

Re:Measuring coder productivity

[tooth]

Personally, I think you'd be better of not doing this calculation, because you have to pay a person to work all these things out. I'd rather have that person looking for bugs/testing the code.

Customer Satisfaction

[yamla]

I presume you are working out outside clients. If that is the case, you should probably be creating design documents and defining acceptance criteria in there.

Acceptance criteria should include delivery date, important functionality, stability, etc.

So use this as your metric. Determine some weighting for each of these criteria and then get the customer and the manager to rate performance at the end.

Did they deliver early? In that case, they get over 100% for coding speed. Is the application running at an acceptable speed (but not particularly fast)? Then they get all the points for code speed. Does it have significant bugs (but was still accepted by the client)? If so, they lose points here.

After all, if you are pleasing your customers (and your managers), can you really ask anything more from your coders?

In addition, it provides feedback for where the coder is having problems and how they can improve.

--

Don't

[jon_adair]

This is exactly how this sort of thing would have worked with my old team:
"I'm almost finished with the new module, but I need to pad the lines-of-code so I can get my bonus."
"Oh, I would fix that bug, but it's only a 1-line fix."

Instead, focus on projects that went well. Ask peers and team leads who should get a bigger bonus -- they'll know and they'll be right.

Also, pay everyone a bonus that you don't want to leave.

Re:Issue tracking

[cooldev]

This is harder than you make it sound.

The number of bugs will be higher for proactive developers that take on more work. Plus, the time it takes to fix a bug varies greatly on the underlying code. Some bugs that sound trivial are actually quite hard to fix, while big features can be done quickly. And what happens when developer A writes tons of bad code, moves onto something else, and developer B gets to clean up?

Bug tracking software will give you a small part of the overall picture, but don't rely on it entirely.

Measuring coder productivity

[rednax]

This question has bugged (if you'll pardon the pun) developers, and development managers since Grace Hopper was pulling moths out from between the valves! You are right to say that lines of code is not a good way, as it encourages verbose coding, nor is time taken to complete the code a suitable measure as you must take quality into account.

In short you need to look at your whole development methodology. One of the more successful ones I have come across was based on Function Point Analysis - now there have been whole tomes written on that subject (not all of which I agree with but it may help you get some ideas). The basis of it is that each project / program is split into a number of function points, where each funciton of the program is awarded a value - eg drop down boxes might rate a "2" whereas a text input might only be a "1". Complex SQL might get a 5 - and so on (there are probably point charts somewhere on the net if you search for them). Total up the program and then the score for the program is what the coder is being asked to deliver. Now obviously there is a whole lot of stuff that can come out of this - if you know the average FPs delivered by a coder you can work out the time required etc, but what you are looking for is a measurement of productivity - so you need to look at how many FPs a coder can deliver in a day (or whatever timeframe you want really). The important thingt is that in your case you can not 'award' any points to the coder until your QA / Testing has assured you that each FP is working and can therefore be counted for that coder.

Sort out your overall development methodology and all the rest will fall into place.

Re:Measuring coder productivity

[Lt Wuff]

I hate to sounds like an echo, but tracking - tracking - tracking.

One of the steps in your issue management process is a review of all changes/bugs/enhancements/etc..If a issues is going to make it into your product (and not get rejected) part of the review is breaking it down into it's composite parts and measuring the difficulty of those tasks. When you assign the ER (engineering resource) to a given task, you have just assigned them a certain amount of difficulty/work load/points.

If they take 3 iterations to complete the task, and another coder can finish the same amount of difficulty/work load/points in a single pass, you have something that differentiates the two come 'bonus' time.

That said, this kind of a system needs to be really, really closely monitored or else it's just a bullshit way of knocking people around come review time. The real advantage is that it if you are closely watching the system you can see where you need to step in and help someone before they come to the review and find themselves with a laundry list of mistakes.