Slashdot Mirror
After the Gold Rush : Creating a True Profession of Software Engineering
Background
I've always been amazed that certain fields of endeavor exist in which many people prefer rank amateurs to trained professionals. I don't know many people who would choose their neighbor's kid to perform surgery, or hire a journalist to build a bridge, yet every day software development shops hire people whose only training is that they have read some book on Visual Basic or C++ or Perl, and put them to work on major projects. In fact, in the software development business, experience can sometimes be a liability (see the debate over immigrant programmers and age discrimination). Don't worry, you won't have to apply for a license to write code, and no one is going to confiscate your keyboard if you design your own web page. Accountability, however, is another story....
What's the book about? Steve McConnell, author of Code Complete , Software Project Survival Guide , and many other excellent books on software engineering, has returned. For those of you who might not be familiar with him, Steve is President and Chief Software Engineer of Construx Software, a software consulting firm based outside of Seattle. He's also the Editor in Chief of IEEE Software, and is generally regarded as one of today's best authorities on how to do software the right way. In After the Gold Rush, Steve gives his view of where the software world should go from here.Currently, people don't know how to build software. Or, to be more accurate, there are good techniques out there to build software, but most people ignore them for one or more reasons, none of which hold up under close scrutiny. In addition, software development depends on death-march-style schedules, whereby programmers wreck themselves trying to get the software out the door at the appointed time. One major problem is that the developers are not trained in writing software in the first place. They are either trained in computer science, or some totally unrelated field. To compound this problem, very few if any continue their education in such areas as professional reading or training.
There is hope, however! Now that software development is moving out of its "gold rush" period, where the firstest gets the mostest, we can begin to develop software engineering as a profession. This means training people in engineering instead of science, and defining what it means to be a software engineer. Unfortunately, we have a lot of work to do both on the level of education, and in the mindset of today's software developers.
The third and final part of ATGR dwells on the future, especially as it relates to what these developments will mean for software developers of today. I'll spare you some of the suspense, and tell you that you won't be forced to be certified. Does anyone really care if your desktop clock was written by a true engineer? On the other hand, you bet I care that my air traffic control software and my medical scanning software and my car's control software were signed off on by someone who knows what he's doing. Licensing will be in software engineering what it is in other engineering fields: required in some areas, meaningless in others. In the end, however, software will be better for it.
What's Good?Personally, I greatly enjoyed this book, mostly because it says what I've felt for some time now. The development of true software engineering will be akin to the development of true medicine. We will be able to move ourselves from the Age of Leeches to a more enlightened age where quality and process matter. McConnell makes a compelling case for why events will transpire in this way, and what the benefits are. No one thinks that these developments will be some sort of panacea or silver bullet, but when true software engineers, complete with a code of ethics and a professional organization, are responsible for developing software, there will exist an undercurrent of ownership and responsibility that currently does not exist.
What's Bad?There's not much I didn't like about this book. If you develop software for a living, read this book. It describes where your industry is going in the next twenty years.
So What's In It For Me?Maybe a lot, maybe nothing. If you develop your software purely as a hobby, these events won't directly effect you much. You will never need a license just to write code. If, however, you belong to that cadre of programmers that likes to think of themselves as "software engineers," or who would like to think of themselves as such, read this. You won't be forced to take a test tomorrow, but software development will never be the same.
Pick this up book up at ThinkGeek.
- Table of Contents
- Acknowledgments
- Introduction
- The Tar Pit
- Software Dinosaurs
- Fool's Gold
- Orphans Preferred
- Software Engineering Is Not Computer Science
- Prospecting
- After the Gold Rush
- Engineering a Profession
- Ptolemaic Reasoning
- Body of Knowledge
- Novum Organum
- Through the Pillars
- Stinking Badges
- Architects and Carpenters
- Hard Knocks
- The Professional's Code
- Alchemy
- The Tar Pit
- Epilogue
- Notes
- Index
Seems to me that the pressures of 'first to market' are getting worse, not better. The marketing message is "Who cares how buggy it is, as long as it's available now?"
The problems with software engineering are fundamentally a management problem. It's been many years since the "Mythical Man Month" showed us how adding programmers to a project can only make it later because of communications overhead. But, many many managers are still doing just that. Furthermore, many managers are afraid of the technology, proclaiming loudly that they don't know anything at all about programming computers; good managers can manage anything at all without knowing about it.
Gantt charts and org charts are something that programmers should never see. But yet in many companies I work with, you can't get anything done unless you've got a big graph of the organization to help you find someone who is high enough to have some weight, but low enough to actually have some brains.
And even worse, technologies are often driven by management needs. Software costs too much, so all the managers want to get some of that good object orientated stuff on the project. So, they hire a few C++ programmers out of school and give them Visio. The programmers are told that their highest goal is re-use. Never mind that the system they are building is meant to handle 320 contradictory/arbitrary business rules for selling widgets to a client in New Zeeland. So, the programmers dutifully build the system to be as re-usable as possible. And, those 320 business rules never get used anywhere else but New Zealand. Meanwhile, the primary goal of re-use has caused the maintainability of the system to go to hell, because management ignored the faster/cheaper/better trichotomy and gave the team a week and half to design the system.
I could go on and on and on about management horror stories. Every place I've ever worked has been the same way, and a lot of those places were Fortune 500 companies. Books outlining ideal software development worlds are great. We really need some new ideas in that area. But how much progress can we make if our managers have never READ the books?
If tits were wings it'd be flying around.
The art of programming has come a long way since its modern inception. Advances have largely come in stages: the advance from hardwiring to machine language, from machine language to assembler; from assembler to low-level compiled languages; and from low-level compiled language to high-level compiled (native and virtual-machine) languages.
The question now becomes: what next? What obstacles exist to the advancing of the art?
In my opinion, one of the main obstacles is the widespread use of C and C++. Both are low-level languages designed for close interaction with the hardware. (C++ may share features with high-level languages but is too tied to C to actually be an HLL.) This is a useful design for many tasks, but not for general application programming, at least at this time. Moving to safer, more powerful languages (Smalltalk, Dylan, Eiffel, Ada, ML, and Haskell to name a few with useful free implementations) is necessary, IMHO, to further the creation and maintainability of complex software systems. C still has its place of course -- critical inner loops will probably be implemented in many applications where the high-level language does not perform adequately -- but general development should move along to more useful tools.
I believe that the free-software community is uniquely prepared to do this, simply because language choice is a technical and not political problem.
Happy hacking.
Currently, people don't know how to build software. Or, to be more accurate, there are good techniques out there to build software, but most people ignore them for one or more reasons, none of which hold up under close scrutiny.
Yes unfortunately most people who program have learnt to program well before they come into any contact with actually being part of a team writing a large application. When it's just you sitting in your room at home coding something just for the sheer hell of it your style of coding and the method you use to design and implement the code are relatively unimportant. I'm sure the most popular method is the 1) Think of something you want to code, and 2) Sit there and start coding it as soon as possible :) Unfortunately this doesn't, in my experiance, work at all well for larger projects.
The software house I currently work for has a lot of problems with how the project is managed. The man in charge of the overall development for our application is far more interesting in trying to code things which have caught his fancy at the time than in making the product consistant, efficient or even bug-free. Designs have generally been of the "here's what we want - go do it" kind, and once someone has done a particular client, they're the only person who knows exactly how it works. When people leave someone else needs to spend a good two weeks just trying to figure out the code. We've recently started to move into a more structured approach to coding, but it remains to be seen whether this will have any impact.
Now that software development is moving out of its "gold rush" period, where the firstest gets the mostest, we can begin to develop software engineering as a profession. This means training people in engineering instead of science, and defining what it means to be a software engineer. Unfortunately, we have a lot of work to do both on the level of education, and in the mindset of today's software developers.
Very true - we've got a lot of good programmers, it's just that they don't know how to write a large piece of software. This is something I see a lot of, and it seems that it takes not just good programmers, but also a good structure for them to work in, to produce something that is consistant, efficient and can be easily maintained. For people out there involved in the same sort of work ask yourselves - if everyone involved in your project left at once, how long would it take for a new team to pick up where you left off?
The one important flaw to note about this book is that it treats software engineering as if the lone example of its practice is in the offices of commercial, shrinkwrapped software vendors. It may surprise a lot of people to realize that the total software sales of commercial software products (including all of Microsoft's products, Oracle's, and SAP's) are only a small, small percentage of the amount paid for custom software solutions for industry, government, and the military.
In these non-commercial arenas, strong software engineering disciplines have been the norm for decades. Good design, long development cycles, and carefully engineered software are an absolute requirement when human lives are involved (air traffic control, avionics, flight control systems, manned space flight applications, command and control systems) or extremely expensive hardware is affected (factory production lines, communication satellites, telephone systems, railroads, etc.)
To lump all software developers under the umbrella of the frenzied exercise that passes for software "engineering" in the commercial marketplace is a myopic view at best and is pretty much wrong. Yes, commercial software developers could do a better job. But market pressures don't allow for this, and frankly, they never will. So ranting about making the engineers do things "better" without addressing the market pressures is a waste of effort.
Until the tools, processes, and automation that surround software engineering improve beyond handwritten code in a text editor, this situation cannot improve. If the market continues to apply innovative pressure on the developers, the only way to catch up is to give developers a higher level platform to develop from. Twiddling around in the O/S, pushing bits around in a frame buffer, and spewing characters to and from a hard disk are far, far below the level where software engineering should be practiced, yet that's what most engineers have to suffer through given the relatively primative state of operating systems and development tools.
It's like trying to build a skyscraper with a hammer and a saw.
Shut up and eat your vegetables!!!
Lest anyone think that I'm saying the open-sourcers have "gotten it right" let me set the record straight. Closed source has its characteristic problems, some (but not all of which) can be addressed by looking to the open-source model. Open source also has its characteristic problems, which I'd say come down to lack of discipline.
The first symptom of this is that open source tends to be all about programming, without anywhere near enough attention to design, testing, documenting, etc. As much as we bash MS for using their users as guinea pigs, open source tends to rely on users even more heavily in lieu of real testing. Yes, I know, at least open-source users have volunteered to be guinea pigs and haven't paid for the honor, but I think that's an inadequate excuse for developers shirking an important part of their role.
The second symptom is lack of a means to control the jerks. I think the majority of open source folks favor cooperation, but a substantial number are more competitive than cooperative. They want to be top dog, or they don't want to be in the pack. When this attitude is recognized, they can be removed from a project by those that remain, but that recognition is often preceded by a lengthy period during which they're allowed to be obstinate and disruptive and generally counterproductive. When dealing with volunteers, it's harder to get people to shut up and do their assigned task, because there's no repercussions to them. They won't suffer for years from bad reviews or bad references, the fear of which is at some level a powerful motivating force in the commercial world not to be too much of a jerk.
As I said at the top, each approach has its problems. The benefits of open source may well outweigh the problems I've described, but it's not a panacea.
Slashdot - News for Herds. Stuff that Splatters.
As a software consultant I change jobs somewhat frequently so I go to lots of job interviews and must stay in touch with what skills the industry is looking for. Invariably, when I'm asked to provide information on my work experience, nobody wants to know what aspects of the Capability Maturity Model I've found to be most useful in ensuring project success, what design and documentation techniques lead to testable event-driven user interfaces, or asked to discuss performance engineering trade-offs between a databases' conceptual model, logical model, and physical model. Nope, their requirements only want to know what programming languages, operating systems, and application types I've worked on. And they want the latest versions of this stuff as well. If you stay on and contribute to the successful conclusion multi-year project from start to finish, you will be really helping your customer but potentially cutting your own throat. And what's really ironic is that the worst programmers and engineers in this profession change jobs the most frequently and are therefor constantly exposed to the latest technology. I easily spend 10-20 hours a week and thousands of dollars a year (my book bill alone is approx. $2000/yr) keeping up with new technology. And it doesn't help when software tool vendors hawk their wares by duping IS management into thinking that using their IDE or modeling tool or methodology causes good designs, good code, good documentation to pop out the other end. A fool with a tool is still a fool. It still takes smart people doing smart things to make projects succeed. And until IS management understands this its going to be business as usual. And you will know when the change has taken place when the posted job requirements call for engineering skills rather that tool skills. Lets face it, a decent programmer can learn VB or Perl pretty quickly. But building reliable, scalable, distributed object-based systems on time and under budget ... that's tough.
I want to be alone with the sandwich
I don't get your examples of the lawyers or the doctor. Are you suggesting that it's the certification that guarantees your doctor is competent? Poor people do have to use public attourneys, so I'm not at all sure what your saying there.
Your bridge example is more interesting, becuase it gets more to the core of the issue. It's a better comparison to software. Doctoring and lawyering are performance skills, and thus, the skill of the person involved is the only measure to use to predict the quality of the outcome.
A bridge, however - you can test it and see how well it works. You can go over it's design to understand it's theoretical limits. Who made the bridge is irrelevant at this point (except that the honesty of the person is involved in as far as the documented design might not have been followed).
This is even more true with software. It can be tested and studied, and there are no hidden elements. It is not important who wrote it, it is only important who signs off on a seal of approval for it.
Therefore, license software, not software developers. Companies can become certifiers and guarantors of software, as a business model. Kind of like Red Hat with Linux - they service it, and essentially lay themselves open for blame for it's failures, in exchange for money
First, make it work, then make it right, then make it fast, then, make it bloated!