Slashdot Mirror
The Futility of Developer Productivity Metrics
snydeq writes "Fatal Exception's Neil McAllister discusses why code analysis and similar metrics provide little insight into what really makes an effective software development team, in the wake of a new scorecard system employed at IBM. 'Code metrics are fine if all you care about is raw code production. But what happens to all that code once it's written? Do you just ship it and move on? Hardly — in fact, many developers spend far more of their time maintaining code than adding to it. Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities?' McAllister writes. 'Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?'"
Refactor, refactor, refactor
KISS technology, nothing beats it.
If you don't have a use case for reuse, you shouldn't try to code for it. To many 'interfaces' are single use, see 'servlet' vs. 'http servlet'.
The force that blew the Big Bang continues to accelerate.
Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.
It almost seems better to measure a bunch of things and use a secret formula to determine productivity.
Man, you really need that seminar!
But what happens to all that code once it's written? Do you just ship it and move on? Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities? Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?
It bitrots. Yes. No. Maybe. Probably. Definitely. Possibly.
Unless it's your job to make up the metrics.
A lot of problems rating developer productivity. First, if a system is that good, then managers won't be able to game it to play favorites. Second, writing code for future use is always harder than writing code specific to the problem. Third, almost any metric is going to penalize a simpler solution. (Keep in mind that once you see a simple solution it seems obvious and everyone thinks they'd think of it. Fourth, evaluating developers well would require making the best coders managers, and that rarely happens for several reasons.
Democracy Now! - your daily, uncensored, corporate-free
Wouldn't it be the project leader who monitors these on an individual basis? If a coder isn't pulling their weight its up to the project leader to address it up to the point of termination. Above that you have a suit who monitors the project leader's team performance and decides how well the project leader is doing. Of all the places layered management doesn't work, coding is not one of them. It's a challenge to hold a developer accountable because there are so many different approaches to the same problem in coding and a lot have definitive pros and cons.
Can any automated tool measure these kinds of best practices?
No. The - for the sake of politeness, let's call them "people" - who invested time and effort into devising these schemes have actually built a complete chain of negative productivity by doing so. Remarkable.
OMG!!! Ponies!!!
Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.
If they know their productivity is being measured, it becomes a contest to see who can cook the books the best anyway.
metrics provide little insight
If only we had some kind of.... metric metric.
What about all the time you spend not coding: meetings, documentation, training users on how to use the system, working out the design before you start coding, answering emails, sending status reports, filling out time sheets, coordinating work with other developers, coordinating things with others in IT so your program will have a server to run from, being the go-between for the IT server team and the customer when the server goes down, creating database layouts and writing SQL?
Coder's Stone: The programming language quick ref for iPad
But that's the whole question, isn't it? What are the *relevant* "easily quantifiable measures"?
This all comes down to lazy, gutless management. Why take the time to get to know Dave and monitor the quality of Dave's work when we can just look at a spreadsheet at the end of the month? Managers prefer to tinker with automatic analysis software rather than manage.
Which is more fun, getting a better handle on what Dave is doing, or researching fancy new software tools that might get you all sorts of praise from metric-craving executives?
Dave's job, which was once about creating a quality product, now shifts to merely satisfying the metrics.
Metrics are valuable if you do the same thing repeatedly. If you build a new building that is like the previous one, you can collect metrics and compare your performance against history. If you write the same search algorithm again and again, you can collect metrics and compare to see how your performance changes over time. Of course, with software, you never repeat. Somewhere around the third time, you move it into some form of library, reuse it, and start on a fresh problem. Perhaps metrics are helpful in some situations, such if your team keeps repeating the same mistakes, you might find similarity in those mistakes (code smells.) There are plenty of people working on these problems and tools. But, from a management point of view, if you keep doing the same thing, you are doing it wrong, and code metrics are not going to help much.
I don't get the concept of everything needing to be quantified. Does the team accomplish what the goals of having the team are? Does it get developed in a fair timeframe? Is everyone on the team pulling their own weight, or are there complaints of someone slacking off? In the end if the product works then the team is doing well, if the product isn't there should be at least one hybrid manager/coder that actually works with the team members sees who is committing what and can tell off the bat if there is or isn't a weak link dragging the rest down. Actually putting a pen and paper number on a complex project is silly. Do authors get judged by the number of pages they write in a day, no they get paid by the success or failure of the book. You can't judge by the number of lines of code, bugs per line ratio or anything like that, because it is all subjective and has little to no bearing on the end product.
Not everything that matters can be measured; not everything that can be measured matters.
-- Einstein (or maybe it was Franklin)
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Measuring developer output/metrics effectively is a tough problem. Developers could solve it, but if they do, then they have to both change the way that they work and possibly work harder. Developers are smart enough to know that the metrics will be misused, even if the logic used to produce them is valid. Therefore, any solution will be ridiculed by the development community as insufficient, but the degree to which it is ridiculed will lessen as the solution improves. A solution though, is inevitable if development continues.
Writing for reuse can be excessive, but there are a number of reasons to move in that direction even if you don't intend to reuse that specific code block:
1) Unit Tests. If you abstract your functionality in a way that allows reuse, it also abstracts in for extremely easy unit testing. And unit testing will save you an incredible amount of effort in code maintenance.
2) Consistency. If you follow the same design pattern for all of your abstractions, all of your developers should be familiar with it. This makes it significantly easier for different developers to step into projects as the hopefully don't have to learn another person's style for abstraction.
3) Replacement and isolation. Need to implement a functional change? If your code is abstracted, like it would be for reuse, the functional change is limited to a single block, which is easily identifiable and if you're doing it right, unit-testable.
4) Just in case. Most of the time abstracted code doesn't get reused, and event when it does get reused it's usually a copy and paste job instead of a reference. Even so, if anyone ever does need the same functionality, it allows them to quickly rip off the exact piece they are looking for as opposed to trying to strip out your programs logic to get the tiny bit they want.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
And every developer starts as an engineer, and ends as an artist. As simple as that. The engineer can do anything, but have not done it yet. The artist knows what to NOT DO, because he has already done it, and does not wanna to repeat all the teenage mistakes again. The artist just gives you the solution. End of story.
... you are already doomed. You've gone so far down the wrong path there's no hope of recovery.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
I think the idea of "productivity" is a hold-over of the Industrial Revolution that does not pertain to many of today's jobs; jobs where the unit of work is hard to define, and ultimately irrelevant. Are you telling me you pick your doctor by how many patients he can see in a day? Probably quite the opposite!
In terms of software development, I find that the *effectiveness* of a developer is more important, where effectiveness considers the following (not an exhaustive list):
- Appropriateness of solution
- Thoroughness of implementation (logging, exception handling, graceful failure, input validation, etc.)
- Well-written, parsimonious code that is easy to read and descriptive of what it does
- Works right the first time, no kickbacks from QA or end user
Give me someone who is effective but slow over someone who craps out junk quickly any day of the week and twice on Sunday! In the end, I don't care about productivity metrics, I care that the end users get a useful piece of software that does what they need with a minimum of headaches.
Insisting on "correct" English is like saying that there is only one, definitive recipe for chili.
Yeah, that's an upper management failure in my book. They should be connected well enough to know what is going on at least two levels down, not one (enabling them to check the calibration of their direct reports). If that's not possible, the branching factor is probably too high.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
This is the guy who first said it.
Stop listening to the MBA and metrics nutjobs. Don't try to manage your people like the machines they operate.