Slashdot Mirror
Is Process Killing the Software Industry?
blackbearnh writes "We all know by now that Test Driven Development is a best practice. And so is having 100% of your code reviewed. And 70% unit test coverage. And keeping your CCN complexity numbers below 20. And doing pre-sprint grooming of stories. And a hundred other industry 'best practices' that in isolation seem like a great idea. But at the end of the day, how much time does it leave for developers to be innovative and creative? A piece on O'Reilly Radar argues that excessive process in software development is sucking the life out of passionate developers, all in the name of making sure that 'good code' gets written. 'The underlying feedback loop making this progressively worse is that passionate programmers write great code, but process kills passion. Disaffected programmers write poor code, and poor code makes management add more process in an attempt to "make" their programmers write good code. That just makes morale worse, and so on.'"
If you want to go off and do your own thing, fine. Have at it.
But don't expect to write code that keeps a 777 safely in the air. That is the type of scenario that we need discipline, not creativity.
I'm all-too-aware of this issue and how quickly it sucks the life out of you and prevents anything from getting done, but at the same time obviously having no process doesn't lead to stellar code either. My question is, do any of you work in a place where you think you've struck the right balance? What are you doing?
Dislike the Electoral College? Lobby your state to join the National Popular Vote Interstate Compact.
70% unit coverage? CCN complexity 20? Pre-sprint grooming of stories? This article is apparently not meant for a general geek audience. Can someone translate these terms into something that non-professional programmers can comprehend?
Part of the hardcore faithful who believed in Apple long before it was cool again to do so
Get rid of management!
Karma be damned, this is relevant to TFA:
This is why I left development. After 5 years working at a software company I found that only 10% of my time or so was spent writing new code, which is the only thing I really liked about the job. The rest of the time was spent in meetings, wading through red tape, reviewing others' code, doing maintenance on the (junkpile) legacy system, doing remedial training for the front-line tech support staff, and the 5 million small tasks that have nothing to do with why I went into the career field.
On one hand, I can understand the creative process that goes into coding and the 'fun' you have in making it your thing.
However if you're likely to have millions of people depending on your code, which will alwso be modified by other people, then you had better have a good process as well.
I used to work at a company which required that pretty much all the important pieces of code have JUnit tests for them. Whenever someone else touches your code - which is bound to happen (I was modifying code which was written years ago - and the author wasn't employed anymore), it'll be a good thing if you know you haven't smashed anything.
So there's a time and a place for everything. If its very important code, yes please, strictness. If its something small and silly, then go creative on it.
I've worked on projects where they had procedures as thick as a phone book, and it was still possible to write crap code. In fact, due to the incompetence of the person who wrote them , they sometimes pushed you into writing bad code.
On the other hand, some programmers produce good code, whether they're specifically ordered to or not.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
I think may be the guru-disciple system of education practiced (allegedly) in ancient India might bear better fruits. Guru lead teams with disciples learning from them might turn out better quality code, but the system would be expensive in the short run and takes a while to take root. The quarterly bottom line obsessed corporate world is as far away from the system of stoic ascetic guru living in a hut in a jungle accepting princes as students romanticized in Indian mythology.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
Only about 10% of programmers are "artistes" who create new things of beauty...process stifles those.
80% of programmers come to work more-or-less on time, sit where they are assigned, work using the tools and processes given, and produce most of an organizations output. Process is beneficial to those, in that it makes their work useful in the overall "whole."
10% of programmers should be fired before they do any more damage. A better HR process would help with those.
The main issue is that measuring whether code is good or not is impossible. I promise you that I can write code that has the prescribed level of unit testing and complexity that also doesn't work. Software reliability/dependability was a problem in the 1960s. It's still a problem today. No silver bullet, and all that.
What really cripples things is when process is deemed a substitute for understanding the specifics of individual situations - where a one-size-fits-all-problems approach is adopted and imposed - usually by people who have no practical experience with the processes they espouse.
If software development could be successfully reduced to a process, I'd have automated it. Where there's a considerable burden of process, either the process is inappropriate - or developing the software itself is inappropriate as it amounts merely to re-inventing the wheel... an exercise in task creation that benefits no-one.
We should think of software development techniques and apply them judiciously - and the more techniques a developer masters, the wider their skill-set and the better they will adapt to new challenges. The critical question that needs to be asked is this: why is a technique being used and is it providing tangible benefits? If this question can't be adequately answered, everyone involved is wasting their time.
Passion isn't important. Cost and risk are important. The processes are put in place to (attempt) to minimize cost and risk associated with software development. Experience teaches us that cost and risk are very high when building software.
When it's your money paying for the development effort, feel free to structure it so that you can chase your passion.
I sympathize with the idea that this kind of bureaucracy can suck the life out of developers, but guys, this is work. If it were that fun, they wouldn't have to pay you to do it.
-- Truth goes out the door when rumor comes innuendo. -- Groucho Marx
This is really the key insight of most Agile methodologies. Development should own the process and change it to suit their needs during regular retrospectives. The team (not the whining individual) should be able to say, "You know what, I think we're not getting bang for our buck out of this many unit tests, let's shift to 50% coverage." As long as that same team is taking ownership of the regression failures and making an informed trade-off their comfortable with, all is well.
If you get a good team together, you're going to get good code. You'll get better code if you empower them. Experienced and good teams will usually have a lot of these processes and tools in place because noticing things like high code complexity automagically alerts them to "bad smells" that can be examined and either accepted as necessary or invested in to address or test more thouroughly.
Generally, I think development is most fun when you're on a new project and don't give a damn about breaking things. Then it's pure creation. But once an app is older and there's some weird code you're staring at you have to decide, "is this probably a bug, or is this a bug fix for some weird situation or platform?" That's when you wish that the guy having fun three years ago had written some damn tests.
the problem is always the same:
how can manager get bonus lowering staff costs?
they get cheapo developers and throw the top notch methodologies at them in the hope those will make up for the devs inexperience and plain incompetence.
that gives in turn a bad name at methodologies and tools, because cheapo devs cannot understand nor benefit fully from them. what use is a pmd output to a guy who's never got the difference between passing by reference and passing by value?
and there are, lots of them. just check any java forum.
yet, the truth is that some of the methods and tools are a good helper for good and also top notch developers. if you work with the tools, and not by the tools, you can catch way more common errors, be notified of piece of code that may need some attention because grew too much in scope and features and lot of other small things that make your life easier.
but it has to be a cereful choice, based on your experience and knowledge, not something imposed because it's the last buzzword.
Process isn't the problem, environment isn't the problem, language isn't the problem. The problem is that everyone looks to all of these things as the silver bullet that's going to "fix" software. Test driven development doesn't make good software any more than I-495 makes New York City. There is no silver bullet.
In most engineering disciplines, the process of actually building something is long, hard, expensive, and persistent. If the project is build a bridge, you spend a lot of time making sure the design is right; why? because the process of actually building the bridge takes months or years, costs many millions of dollars, and once built is not easily replaced. There is no room for error, so process is taken very seriously as a central part of ensuring timely cost-effective correctness.
In software, the process of actually building something is instant, easy, free, and transient. Type "make all" and go get some coffee; find a bug? tweak a couple lines and do it again. This distorts the development process; "process" gets snubbed as a costly distracting annoyance instead of the means of assuring that what gets delivered is correct, because it's just so dang easy to fix and rebuild in seconds ... losing sight of the long-term cost of not doing it right the first time.
Can we get a "-1 Wrong" moderation option?
A wise elder at a defense contractor once told me that process was like cholesterol. You can't have none. But too much will kill you.
Except for ending slavery, the Nazis, communism, & securing American independence, war has never solved anything.
TDD is a painful waste of time that at best serves as a crutch for unskilled or insecure coders and at worst a smokescreen behind which serious bugs remain unfixed because they aren't picked up by any test cases.
No, TDD is painful, but it's a long-term investment. If you code once and never have to touch it again, you can get away with making it work right without tests. But if you want to be able to grow your system, you want something to make sure that you don't kill the old functionality in the process of building the new. And sometimes it's just helpful to spell out the expectations of new functionality ahead of time so that you know when you've achieved it.
Process for the sake of process is pointless. And if you have good reason to work around or jettison a given process, go for it. As I told the other programmers at my job, the only piece of our process we would probably consider an absolute necessity is source control. Code review is highly recommended, tests are very strongly pushed, formatting standards are there for good reasons, but there are times when they are not helpful.
The problem always comes down to whether or not your developers are adults with good decision-making capabilities. Process is too often employed as a way to allow you to get stuff done with people of average or less ability, but process also hurts them because they can't figure out when the process is unhelpful. And if process is imposed by fiat, those who could figure that out are frustrated by having to go through the motions. But if you have trustworthy people and you actually trust them, you institute the process for their benefit, and when it's not useful they know not to use it.
The real problem is that methodologies are followed, taught and practiced like religion instead of science. A new buzzword or buzzphrase and a new way of doing things will never substitute for thinking through whether what you're doing is actually working to attain the goal you've set out. Unit tests for example are brilliant when the developer gets control and is honest about using them. This almost never happens in practice. In practice unit test coverage becomes some bureaucratic check box to tick. What's worse is that there are dozens of industry expert consultancies wanting to sell you their own particular brand of excrement which may have some merit but the way it's sold - as a fix all for everything - will never be anything but a pile of poo.
These posts express my own personal views, not those of my employer
I completely agree. I've come to believe that Agile development is a fad invented by some marketing genius to get big loads of buck from gullible enterprises. While TDD might be useful for, say... a linux kernel module, there's very little use for it in your standard "make me a module which reports in detail our budget surplus and deficits" project.
It's much more efficient to hire a small team of beta-testers available on-demand ("Jim wrote this new model, can you test it please?") than wasting hundreds of man hours per-month in "agile" development.
My problem with process has always been budget. Folks higher up budget on the basis of minimal process and expect full process. If you want 70% unit-test coverage, that is twice the time the code would have taken if you did not write unit test. Add 3 times the time if you want good integration tests to go with it.
Unfortunately, this makes a project costly. The problem occurs if the PM then demands full process when the time is not accordingly budgeted for it.
Release cycles also become long.
You can use mediocre programmers if you have (and enforce) good development process and train them in best practices. Being able to build code that is easy for the programmer to read (and others to read), separation of sql/html,css,js/ etc into their respective parts of the MVC/ORM pattern, consistent inline documentation, consistent comments on code checkins, etc., following the same rules for writing code (development docs)
A good development process can alleviate many of the problems with having to do all that testing as you test because you have a bad development process wherein you are unable to easily have the developer look at his comments, his documentation and his clearly written code and go 'oh thats the issue'
Not saying this will ELIMINATE the need for testing but this reduces the need for at least 30-50% of it if everyone is properly trained in a good development process and it is enforced.
This is my sig. There are many like it but this one is mine.
I don't (usually) practice TDD. I do however write automated test cases for as much of the system that I'm developing as practical. Usually thats around 80% or more of the functionality.
The major benefit of automated testing comes with maintainence of the code. Making changes 6months or more later can be more difficult as you've forgotton all the intricate details and exceptional cases. This is where test cases can make your life eaiser and much more productive.
So, in the article he does suggest some process is probably good. That's good. But then he suggests that "better" coders might need less. In practice, that is probably true, but his implication goes farther than "let's hold the hand of the junior guy a bit more."
So... how to architects, civil engineers, electrical engineers, graphic designers, technical writers, project managers, and, I dunno... just about every white (and blue, for that matter) collar employee stay "passionate" in the face of some level of process and bureaucracy? How would you like to go on a bridge (or a plane, as others have mentioned) built by folks who are so smart they don't need any process to validate their genius?
This sounds more like "I'm so good I don't need to do these dumb things that slow me down" and less "perhaps we should keep the process just at the point it is beneficial".
I just spent two years with a team of 6 refactoring a critical legacy application built and maintained by people who were "smarter" and "better" than code reviews, TDD, planning, documenting, or hell, breaking that 8K header file into header/class and maybe even into a couple of objects. And truth be told: they were ALL really, really smart.... and really, really willing to cut corners just to "get something done", resulting in an unmaintainable mess that cost us somewhere in the neighborhood of 1.8MM to clean up.
To all you guys who are WAAAAAY too smart to have processes: that's awesome. Please just go work somewhere else, preferably someplace I'll never work.
The problem that I've found with TDD is that it encourages people to write code that tests the wrong thing. I like using it, because I gradually grow a set of regression tests and can be reasonably confident that I haven't broken anything, but I've seen a lot of people write tests that check for specific details, so you make a 5 line change to their code and then have to modify 100 lines of tests - not because you've broken anything, but because their tests were checking for implementation details rather than valid semantics.
I am TheRaven on Soylent News
I've been programming since the 80's and getting paid to do it since the mid 90's. It was the mid to late 90's when I got introduced to life cycle methodologies and at first I applauded them. I was even selected for a committee to help firm up our "best practices". I thought I would improving things by stopping bad practices by bad programmers. After a while it turned into a nightmare as those bad programmers started using the "best practice" rules as cover for the crap they produced. "But it passed unit testing and UAT, it must be awesome". Our project budgets went up and implementation time increased while our customers were visibly pissed at the final product.
The problem with SDLC process management has been sufficiently skewered by Joel Spolsky in a blog post several years ago. By putting a bureaucratic process in place you replace rock stars with compliance monkeys. Actually delivering a solid product is far less important than meeting checkpoints. And when your a programmer who's dedicated to a polished final product it's like murdering your soul to watch mid level managers (who know nothing of development) work hard at micro-managing you to ensure those check points are met on time. The actual outcome is irrelevant, the surgery was a success even though the patient died.
I have worked in both environments, the loosy goosy no rules and the highly rigid SDLC bureaucratic world. I understand the need for review and verification and the problems of wild west development. But development process methodologies are dangerous enablers of bad middle managers and "consultants" that sell you management processes that don't really help any body.
I use lib777, so my code looks like:
while(!wantToLand)
my777.StayUp();
my777.Land();
There is no crash in the code.