Slashdot Mirror
It's Not About Lines of Code
Charles Connell writes: "What makes a programmer highly productive? Is it lines of code per
day? Lines of good code? In this article, I examine the concept of software
productivity. I look at some of the standard definitions for productivity
and show why they are wrong. I then propose a new definition that captures
what programming really is about." Read on for Connell's stab at a better way of evaluating the worth of programmer time.
CT Originally the contents of an article were here but there was
a communication problem resulting in us thinking we were given permission to
print the article here. Now that things have been cleared up, we've linked
the
original article which you can read instead.
Sorry about the inconvenience.
Phew, what a long winded way to say: KLOC in any form is a useless metric.
I was rather hoping for positive suggestions regarding better alternative, and especially some shiny references to back them up for when I take them to my boss.
The best metric I've found is simply "Time until feature complete". Just that. Elapsed time between a feature being requested and it going live in the field with no bug reports coming back. Anything else is largely bunk. Trouble with that is that it's very hard for twitchy bosses to deal with the interim stages. "Time to code complete" is easier for them to monitor and deal with, but as anyone who has actually supported a product will know, that's only the beginning of a piece of software's life. Push the time to code complete back by a week, and you can save yourself month of grief later. ;-)
If you were blocking sigs, you wouldn't have to read this.
This article was rather superficial and should have covered one point instead of several, giving the article a bit more depth. I got the feeling like it was written off the cuff this morning.
(How do I know Danny does more work just because he does the same amount of work in less code. Its like the author follows some reverse logical mistake that he is harping about.)
A very interesting book on this subject is "The Psychology of Computer Programming" by Weinberg. It will get you thinking! I think the "Mythical Man Month" also discusses related topics.
Thanks
Spot the bug in this thought, then:
"Danny's code probably will be easier to extend and modify, and likely will have a longer lifespan, because of its compactness."
This is potentially utter rot. If you make something "compact" then you're quite likely *using* the language to insert every idiom you can think of. That makes code *unreadable*, not productive at all.
(Yes, I've been known to argue in favour of using prgramming languages' idioms before now, and more to the point I expect folks who dare to look at my code output to be able to read it, or else have the decency not to criticise it) but there are limits.
The alternative is that he's implemented a sufficiently different top-level algorithm - and at that point you're not comparing like with like so you can't say one person is any more productive than another.
However, you could introduce "amount of desired algorithm implemented per day" into the formula for productivity calculation.
~Tim
--
Rushing on down to the circle of the turn
Yeah, that's great, but what manager works like that? No manager would ever say to me "John, how long would this take you?" for a project with a 6 month or 1 year time frame, and then leave me alone for 6 months while I write code. Managers want progress reports and statistical measures to reduce the risk that that they might get fired when the project they're managing fails. That's where the boneheaded requests for the "number of lines of code you've written this week", or "number of bugs you've fixed this week" come from. It's hard to look at an unfinished piece of software and know how it's coming along, but it's easy to look at a painting or a statue that's in progress and see work being done.
This article seems to raise more questions than it attempts to answer... This is not surprising because the "programmer productivity" has been the subject of many debates.
The proposed definition ("Ability to solve customer problems quickly") seems to ignore several of the good points mentioned earlier. For example, one can be able to solve a customer's problem quickly with an ugly hack. Some undocumented spaghetti code full of black magic may be able to solve one problem quickly, but it would be impossible for anyone to maintain this code later. Or to re-use it for solving someone else's problem.
So who is more productive? The one who solved the problem quickly with an ugly hack or the one who solved the same problem by writing clean, documented and re-usable code?
So it is a pity that what appears to be the conclusion of this article has thrown away the notion of clean and well-documented code mentioned earlier. Maybe a better (but more complex) definition would be: "Ability to solve customer problems quickly and to solve future, similar, problems in a quicker way". The drawback of this definition is that it cannot be measured on the current project, but only when the next one appears.
-Raphaël
I want to add another angle... I managed validation. I viewed our job as reducing 1-800 support calls to zero. In the end, support costs need to be rolled into productivity numbers for develpers also. A couple of support calls from a single user can easily make the gross margin for the sale to that user negative. (And for you "free beer" programmers -- same thing applies, wouldn't you rather write code than spend time supporting users?)
And a closing note: A wise manager once said: "Obviously you want smart, productive people on your project. Note that dumb, unproductive people are relatively harmless, because they are not productive enough to cause much damage. What you need to watch out for are dumb, productive people."
Beat me to the punch.
This is old news, and if a manager is looking at lines of code produced, as opposed to general quality of the overall product, your manager is 95.
I have yet to see a manager look at "lines of code" as any type of measurement.
Has anyone?
Good quote, too many chars. Seriously, the slashdot 120 char limit sucks!
A metric that takes real productivity into account for new projects (the Ingrid example above wasn't a great one since she was maintaining code, not creating it) would be one that measures requirements met versus time. It would of course be up to the manager to say whether the time taken was too long, about right, or less than expected.
A developer that consistently meets requirements with working code, in a timely manner, is a good developer.
Clearly the key to success with this metric is managing your manager's expectations. ;-)
Galileo: "The Earth revolves around the Sun!"
Score: -1 100% Flamebait
It is really all about time and money. Both have already been posted seperately but these 2 measurements are the clincher.
Sometimes really good code is just not worth it.
Sometimes code is not worth it period. (There are better ways to solve the problem than a custom process)
If you don't make enough money to pay for the time- you won't be in business long. (At least as long as your software engineers are willing to live off their own credit cards and pick up company expenses.)
.
It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
While I admire the author's brave attempt to establish a new model, I must conclude that he has failed: Applying a quantitative measurement to a qualitative phenomenon is absurd. Do we judge a painter by the number of brushstrokes and conclude that Rothko is lazy and Van Gogh is a diligent worker?
There are many qualitative components by which a programmer can potentially be judged: sheer output, ability to debug others' code, ability to have code debugged by others, ability to create understandable extra-code documentation, affinity for the lifestyle of the person doing the judging (i.e., likes to go out for a beer after work, roots for the same professional sports team, etc...), concentration skills, hygeine, and so on. Attaching metrics to this is nothing short of masturbating one's ego. There is a wonderful allegory to this in Robert Anton Wilson's Schrödinger's Cat trilogy, in which a self-important researcher attempts to rate a woman's orgasm using measurements on scales developed by the doctor himself.
"What is the sound of one belly slapping?"
You realize that by this measure Microsoft has the most productive coders in the world.
Oh yeah, all those people who made/created Linux are completely unproductive.
The big problem with any of these measures is you're measuring the wrong "product". Computer code isn't a product, it's a means to an end. Computer code is the final instantiation of an entire process from planning, to coding, to testing, to execution, to refining. First off, the version 1.0 of almost any program isn't the final product (fortunately). Second, wonderful, bug free code accessing the worst structured database is still a terrible application.
So, the idea of evaluating productivity by looking at the code is just plain silly. Evaluate the project.
--- I wish I could hear the soundtrack to my life. That way I'd know when to duck.
Sometimes I feel most productive after I have deleted thousands of lines of code. Code that has been replaced by better code. A good day is when CVSweb reports +500 lines -3000 lines.
Having a quick and easy equation isn't inherently bad, until it gets into the hands of a lazy manager. I would argue good managers don't need or use stupid equations to figure which member of his team is productive or not. Bad managers on the otherhand don't care enough to really understand the project, the engineers or the problem to set realistic expectations. Having an equation gives the bad manager a false sense of control and often results in the wrong person getting credit. I don't have any solutions. Ultimately, I don't think there is one solution or even a best solution. Software development is an organic process, so the more specific an equation is, the more likely it won't work in practice. If all programmers were equally trained, there wouldn't be a need for productivity metrics.
Unfortunately that doesn't exist. Having programmers of varying experience and skill is part of the challenge and makes programming more challenging. Sometimes generalizing API is necessary to make it easier for other programmers. Sometimes generalizing is over engineering. No one can predict the future, therefore the concept of productivity is temporary and arbitrary.
Placing value on n lines of code is always a sketchy excercise. At best, it's harmless.
I can related well to this article because of a past job experience. The job was software developement for a large corporation, in an office setting. The dept was devided into different teams and each team worked on different features of the same major project. To my manager it looked like I was coming in late, leaving early and not doing a whole lot to be productive around the office. What actually was happening was that I wasn't able to be as productive in the office setting. I would copy what I working on to a floppy and take it home to work on it. After waking up in the morning, more ideas came so I went ahead and put them in, making myself late in the process. The features I was working on was completed on time, despite having to wait for others to get done with their part. Always tried to write code efficiently and with as few lines a possible, with good comments and white spacing. Despite all my efforts, I was given a less than average performance review. I got chewed out for leaving early and put on a peformance improvement program (which happens to be very close to being fired) and means you don't get a raise or bonus. There needs to be some measure of performance for programmers but it should really be about getting the job done to the customer's satifaction and within a reasonable time frame. Hours spent in the office, lines of code and all the other bullshit management uses should be as considered secondary measurements at best.
For instance, the assumption that lines of code = complexity is false. Ultimately, these are what matters in programming:
How fast is the compiled program
How big is the compiled program
How easy is the source code to read
How stable is the compiled code
How secure is the program
How complete is the feature set
These are all about software quality, not quantity, though. Once you've measured qaulity, the only measure I can see for quantity is, "Did ya get the job done?" The key to measuring a programmers productivity, I think, is to have the programmer keep a log of what he's doing. With that log, the company can insist on improvement, a maintained level, give bonuses for productivity, etc. The only issue I could see with such a log idea is that the higher ups will become so obsessed with the log, and what the programmer is allowed to claim as a job done in the log, that the programmers won't be able to do their job. Oh, well, it was just an idea.
BlackGriffen
If you need comments in order for your code to make sense, you need to rewrite your code.
My personal measure of coder productivity is as follows:
1. Take all the produced code.
2. Remove all the comments.
3. Present each function to a third party who is unfamiliar with the code, and ask them to explain how it works. If they can't, delete the function.
4. If you have anything left, remove all formatting except linebreaks, and count the *distinct* lines of code. (Bad coders tends to cut+paste, which would lead to overcounting.)
Tarsnap: Online backups for the truly paranoid
I could go the easy route and check every array index in the first array against every array index in all the other arrays. For all I know this is the most efficient way to do this.
But, instead I'm going to research some algorithm books and see if I can't find a more efficient way to retrieve the common elements.
I may very well end up with something that takes a lot longer to code and has less lines of code than the brute-force compare-every-element-against-every-other-element method. But if the payoff is faster, tighter code, then my research and extra coding time will have paid off. However, to the untrained eye, it may look like I'm spending more time to produce less code.
If you are a manager reading this, remeber, the best solution may take longer and contain less code than a suboptimal solution! You have to think hard if you are going to try to quantify this - because nobody knows off the top of his/her head what the best algorithm for everything is. If you have a programmer who regularly researches the efficiency of the algorithms they use you will probably end up with a lot happier customers than if you just have people who bang out code without thinking hard about what they are doing. Unfortunately, doing things the right way makes the programmers job harder to quantify.
No, Thursday's out. How about never - is never good for you?
I do not disagree with you and to add to your point:
To create a tool for use with art (say a paint brush) in itself is a craft, but to make a beautiful styled handmade 'one of a kind paint brush' with your skills certianly takes you out of the relm of just a simple craftsman. I would think that photoshop is the 'one of a kind brush' and 'paint' is norm in the craft.
"Not my manner of thinking but the manner of thinking of others has been the source of my unhappiness." - M
In 5 years I've deleted more lines of other people's code than I've written myself, but most of the stuff I do is redistributing someone's functional implementations.
The result?
1. Less bugs due to less lines of code. Bugs are easier to spot.
2. More code reuse.
3. More readable.
4. smaller binary size.
5. Faster (mainly 'cos you can see what it's actually doing)
The ultimate irony is the point that the code runs faster. Normally people have written 'optimisations' which are unreadable and don't quite work as expected, so they compensate elsewhere...
I like commentParserDatabaseCursor. I also like i, j, p, and retval. The length of an identifier should be roughly proportional to the log of the size of its scope. A file scope "i" is an abomination. A loop scope "commentParserDatabaseCursor" is an idiocy.
If managers want a measure of progress then the team should know this.
"Percieved" progress and "actual" progress can vary. A prototype can give a lot of percieved progress but no (hopefully) actual progress.
80-20 rule always kicks in when trying to measure progress. I've known developers who manage to keep their project 90% complete (yeah, you know who you are) for a very long time.
I prefer having an almost constantly shippable product from the outset. It's difficult to achieve early on but it can be done very soon with basic functionality.
It also provides a good way to test design and implementation. Aim to get something working within spec. asap and everyone will be pleased.
It also provides a good measure of progress. Ticking features off a list reassures the managers and gives a good boost to the team. Admittedly it's not possible for some projects with lots of core features needed to be built from scratch.
Should the shit hit the fan (like a competitor doing a release) then features can be dropped and the product shipped much quicker. It means renaging on some promises made to customers but if you have been open and honest with them they'll understand, also, if you release early/often they don't mind waiting for X if the have Y.
You've just got to keep an eye on the team, harass them for quality, and reward excellence.
My measure of productivity is based on delivery of code/features that falls within my quality guidelines. This includes lack of bugs, code layout, comments and reuse. e.g. A feature can be 'working to spec.' but if it has no comments in the code then it's not 'finished'.
I also know developers that quickly write the code, do all the testing and then call the feature finished. They then go back and add comments, correct variable naming and touch the code in other ways (adding whitespace, reoganising if's) - they fail to realise that they now need to perform all the same tests again, which of course they never do. This is *low* productivity IMO and is generally poor style.
Coding *is* art and I am an 'artished'.
You want to measure documentation separately, because it obviously isn't used in the same way as code.
For the code, the metric you'd actually like is something like number of useful, novel expressions.
Copying a section of code doesn't add anything, because the lines aren't novel. Any changes you make to the lines do count, though. Making an existing block of code into a separate procedure adds a novel expression, and the expression is useful if you call it from multiple places (i.e., if the procedure is a useful abstraction). Calling a procedure with a new set of arguments is novel and generally useful.
Basically, you want to know how long the day's work had to be, given the pre-existing code, if it was done optimally.
If you add a penalty for making the code needlessly long, then a day spent reworking bad code to be shorter by combining common expressions, removing extraneous computation, etc, will also be considered productive.
For the documentation, it is easier to consider but harder to quantify. You are now measuring the number of correct and useful pieces of information that a person would get out of reading the commented code. The information may, of course, be obvious from the control structure, implied by the variable names, or actually in comments; since comments add to the length of the code without adding any functionality, using coding idioms that the reader will know (because they're part of the company's style guide or common throughout the code) and informative names is better than putting in comments, unless it is impossible (which is frequently the case).
Of course, it's hard to quantify "pieces of information" and hard to judge objectively what can be gotten out of a block of code, which is one reason this isn't something you can set up cvs to do each day or something.
This means that the ideal block of code, which should be counted as the most possible for a given problem, has these properties: it (or an equivalent block) has to be there in order for the project to do what it's supposed to do; it is not replicated, in whole or in part, anywhere else in the project (if it did, it would do better to call the other part); its behavior is clear from the names of the procedures, variables, and types; any common algorithms are implemented in the standard ways; and unusual algorithms are commented explicitly.
I think this metric would measure productivity effectively. Of course, it doesn't help productivity to actually try to measure it.
Number of lines of code written, highest score wins. In short, why write in 100 lines what you can write in 1,000?
Number of lines of code written, lowest score wins. You end up with your own obfuscated coding contest. You might also find people rewriting other people's work, redesigning APIs and other infrastructure components, for no reason other than to lower their own score. This can lead to total chaos, fights in the parking lot, etc.
Number of good lines of code written a month. What's the definition of "good," and how subjective is it? If it includes comments, then how is the usefulness of a comment determined? I've seen developers write more comments than code, and in the end the comments said nothing useful that would've helped a new developer maintain the code.
Number of bugs fixed in a month. The programmer who spent a month researching the sev 2 bug that was affecting system availability or data integrity for the last three months isn't recognized for his/her achievements, while the intern who fixed 100 bugs pertaining to typos on the website and in the documentation is rewarded.
Number of bugs created in a month, lowest score wins. Nice idea, punishing people for creating bugs, but people might get so paranoid about causing bugs that the turnaround time for code is obscenely high.
Code complexity metric, lowest score wins: All this proves is that the programmer's capable of writing non-complex code but says nothing about the documentation for the code, the overall design of the component or subsystem the programmer was working on, etc.
Number of tasks completed in a month. This screws the guy who's got a hefty task that cannot be divided any further or the programmer waiting on the sysadmin to install the necessary development tools so that he/she can actually get started.
Customer satisfaction. The customers are pissed because the website is unstable, but the rest of the back office system is running perfectly fine. In the end, everyone -- the back-office developers, the database guys -- is punished when only the website people should've been put on call center duty for a week.
Number of customer issues resolved. There's a great incentive here to solve issues with kludgy hacks which, in six months to a year, might leave the company with a very flaky, unmaintainable system.
360-degree input, or "Mutually Assurred Destruction". This was a system IBM used -- still uses? -- where your peers, some picked by you, some by your manager, would fill out a survey and offer opinions about you. The manager would then piece it all together and come up with a result. Like Dilbert called it, it's "mutually-assurred destruction," although I saw it work the exact opposite way many times.
There's so much more that goes into developing and delivering software than just writing lines of code. And the number of lines of code written isn't all that significant if the design sucks, if the documentation is unusable by the people who need it, if the call center people supporting the thing aren't trained properly, or if the systems supporting the website or the database are unstable. How do you put a score next to a name when many of the things contributing to that score are subjective or out of the control of the person being scored? We're not building CD players here!
Why would someone want to know "How many lines of code per day do you write?", or "How many good lines of code do you write?". The problem here isn't in the programmer being unable to be clocked, but not hiring proper management. If I am programming something, I don't want to report to someone who doesn't know what I'm doing, I want to be able to explain to him reasons for things, and him be able to understand it.
The issue arrises when an individual has to report to someone who just wants a number, and doesn't trust an individual. If upper management were to hire a manager that has proven performance, he should be trusted to say that "Coding is going along well, we have completed [fill_in_blank], and are well on our way to [fill_in_blank].".
Programmers can then show what they have done, and prove why it was done the way it was, and the closest management can understand. That management should also be educated or trained in communication between code, and his management (aka, his job).
We shouldn't have to say how many "pixels we painted", or "lines of code", or "notes played today".
It is amazing how much people favor their own talents over those of others. Here you say that programming is something more than a craft. It is not. A craft is the knowledge to apply a set of tools to create something that is useful or beautiful. The art comes in applying that knowledge to create something superb within your field. Certainly not all programs written are art, but some of them could certainly qualify as such due to their elegance and novelty.
The grandparent comment appears to have used the term artist synonymously with craftsman, and he is correct in doing so. Not everything an artist creates could be considered art. Only those works that speak some hidden truth beyond the surface of the medium, those that are truly beautiful, even if that beauty is not in an asthetic sense, can be called art.
Furthermore, anyone with a reasonable level of intelligence CAN learn to code well and quickly, however just as there are painters and sculptors, there are the Michaelangelos. Some craftsmen will more fully develop their ability than others.
The firm I'm working for hired a contractor to write the system software for their scientific instrument.
... other conditional code // to exit
...
They later hired me.
It is now my job to maintain, expand and rewrite his original code as the device gets further from the prototyping stage.
Here's some metrics for you:
4000 lines of C code
Avg. Variable name length: 3 to 4 characters
Avg. Function Name Length: 3 to 4 characters
Total number of functions (not including state-machine functions): 9
Amount of documentation: nearly 0. Comments are laughable.
Total number of functions, including state-machine functions labelled from stsws0 through stsws30 - 40.
Total number of constant values used without #defines or assigned names: Too many to count.
Amount of documentation of constant values that aren't obviously for buffer/scratch space, but actually do something important: Zero.
Amount of dead code: Current investigation indicates somewhere between 30% and 60%.
Amount of dead code interspered with live code, so it's really difficult to work out what's a dead function, and what's live: All of it.
Level of interweaving of dead code and live code: pretty damn high.
Use of pound-defines for code switching and giving alternate code paths: Zero.
Use of pound-defines to switch blocks on and off that really you want to keep on ALL THE TIME (particularly as the app crashes if you turn them off): 10
Interesting idioms:
-- Use of pound-if(0) and pound-endif to bracket (useless) comment sections, eg:
pound-if(0)
This is a comment.
pound-endif
-- Use of a while loop to do error handling. Eg;
while (TRUE) {
if (condition) { error = 5; break; }
break;
}
if (error)
Number of pound-includes that are actually totally unneeded:
5.
Number of Windows 3.1 programming idioms used: 2 found so far. In a piece of code that *requires* NT4 and as such is Win32 only.
Other interesting idioms: Massively nested if's EVERYWHERE. Very little modularisation. Grabbing an HDC at that start of the app and not letting it go until shutdown, without specifying a Class DC.
The guy REWROTE FROM SCRATCH button controls and edit controls, using WM_MOUSExxx message and WM_CHAR handling, as part of the main frame window. Each edit/button has a separate cut/paste if statement block to handle it. There are about 80 controls on the main screen. This code is cut and pasted for each single control.
And for the final piéce de resistance;
Total number of local variables used: ZERO. 0. Nada. Zilch. EVERYTHING IS A GLOBAL VARIABLE.
Coming soon - pyrogyra
The man-month is a myth. Didn't Steve McConnell debunk it in "Brooks' Law Repealed?".
i c0 8.htm
http://www.construx.com/stevemcc/ieeesoftware/e