Slashdot Mirror
We Really Don't Know Jack About Maintenance
davecb writes "The ACM has been kind enough to print Paul Stachour's and my 'jack' article about Software Maintenance. Paul first pointed out back in 1984 that we and our managers were being foolish — when we were still running Unix V7 — and if anything it's been getting worse. Turns out maintenance has been a 'solved problem in computer science' since at least then, and we're just beginning to rediscover it."
"Software maintenance" has absolutely nothing to do with computer science. I wish people would stop calling business programming computer science. Computer science work gets done at universities and research institutions, not at Initech.
Taking code and cutting its size by half, fixing up all the screwed-up inconsistent formatting, while adding functionality and reducing bug counts, is a pleasure.
It's all in the mindset. It's only boring if you limit yourself to the boring parts.
No, I'm serious. It's the latest Trend (tm).
Cut the IT production support budget, cut the IT staff, move the functions that used to be IT Prod support to business and let their budget handle it.
If you can't see the humour in seeing a business person make direct live updates to a database table in Production (IT doesn't have Production access.. but Business does .. [go figure]) then you probably can't see that SEV1 sailing in from over the horizon to make your day just that little bit more special.
As for business editing files directly in Production because the cost of having IT do it (process - backup file, edit file, copy file to prod with due authorisation, verify the change) can just be avoided.
After all, we don't really need to pay for Production Support and System Maintenance and Documentation. The system works without these things, doesn't it?
What could possibly go wrong?
You have a sick, twisted mind. Please subscribe me to your newsletter.
But I really love maintenance. Sure, the job comes with tremendous amounts of stress but it really entertaining. In CCNA, there's nothing better than subnetting a large network :)
"The difference between genius and stupidity is that genius has it's limits" - Albert Einstein
I take it you haven't noticed the 6 decade wait for a 64bit version of flash or the countless security bugs that resulted from somebody not thinking things through in an analytical manner before actually writing the code.
Sure there's going to be bugs and exploits, and it can take a long time to port code, but a lot of it is self inflicted because of changes to the objectives or a failure to ensure that the code was properly engineered, commented and audited.
Properly formatted and written code should be relatively easy to maintain or at least not a total nightmare.
"Software maintenance" has absolutely nothing to do with computer science.
Actually, it does.
I have a real Computer Science degree, so I know what computer science is about. And while a lot of corporate programming is more drudgery and form assembly than anything, there are a lot of applications of computer science in the real world - from scalability issues in large systems, to proper use of encryption.
Furthermore the supposedly boring area of "Software maintenance" has a ton of research potential focused around the optimal path to producing correct code. Do code review help? How to team dynamics factor in to code quality? Does Test First really improve code? What even defines code quality? There are programs within companies that experiment with different methods to improve code output, and those experiments are even more valid than ones performed at research institutions since they work on a real-world code base solving a real problem using real people. In fact I would go so far as to say and research being conducted outside of a company on aspect of code quality improvement is pretty much worthless, which is why it's important for researchers to partner with real companies for some studies.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
FreeBSD 1 and FreeBSD 2 had slightly different semantics for some system calls, but FreeBSD 2 changed the system call numbers, so it was possible to modify the FreeBSD 1.1.5.1 kernel to run the binary Netscape for FreeBSD 2.x by implementing the new API for one call in the old kernel. Alas, I can't find the patch now, which is embarrassing because I was the one hosting it... about 15 years ago.
Taking code and cutting its size by half, fixing up all the screwed-up inconsistent formatting, while adding functionality and reducing bug counts, is a pleasure.
Yes it is.
But that is not maintenance, as practiced by any rational company. That is development or (more specifically) optimization.
Maintenance is about solving a problem code is having, with the absolute smallest number of changes possible. Even new features can fall under this heading when software is in true "maintenance mode" in order to avoid a lot of excess testing.
I actually don't mind it, as it is a different sort of challenge to take a code base you know little about and introduce a working change with as little code as possible. But it's not as glamorous as fresh, raw coding.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Could a six sigma program help here? A systematic and structured approach to problem solving is needed. Whenever someone fixes a bug that creates a new bug, then it's a waste of everyone time and effort.
Coding is important, but what is also important is program design and architecture. This isn't thought of that much as a smaller utility that "scratches an itch" just needs to be coded.
However, larger projects, and projects with any security needs at all [1] need to have a lot of time put in to design security before a line of code is written. For example, a Web browser should see if it can run under a lower privilege context without preventing the user from downloading files. A Web browser plugin needs to assume any code it gets from a website is likely tainted, treat the code as suspect, and run it in a sandbox (CPU/filesystem/RAM limits, preferably enforced by the OS, and even better, able to have the code run in a limited context other than the user.)
Compartmentalization is important, so instance of a program cannot interfere with another. This way, something that mucks with a cache directory only affects that instance, and cannot modify another. Same with a thread of execution needing protection from others that might get code injected into them.
Separation of code modules that are vital to security is critical. This way, extreme code review can be focused on the security critical modules, while other modules (such as the one that renders stuff) can be reviewed, but it wouldn't have the extreme focus as the parser of incoming code.
This stuff is vitally important, and can't just be bolted on without a complete rewrite. It has to be part of the architecture from the first line of code on. However, there are two benefits from having this integrated into a core program design. The first is that the security critical code is in modules that can be thoroughly scrutinized and audited. The second is that the program will be far more resistant to attack, and if an attack is done, it can be fixed far faster than a product that has no security in its core design.
[1]: One would be surprised at what programs need security. Anything touching the Internet like a browser plugin, mail filter, or even a MUD server are just as important to have security factored in as a SUID root utility.
Decades ago, companies which developed technology were...technology companies. With real engineers, and highly technically skilled management. Today, companies with business-oriented management and zero technology background own and develop systems. They often do it poorly, with insufficiently empowered engineering teams, and insufficiently skilled engineers.
So today we've got a lot of Java and .Net shops filled with junior-level programmers and no disciplined, experienced systems engineers. Is it a surprise that when MS brought programming to the masses that the masses failed to learn engineering?
Doesn't modular programming solve this problem?
That is one of those answers that sound great in theory but in practice suffer from all the same problems from which all the other answers suffer.
Too much upfront cost is a contract-killer, so compromises in designing for scalability must be made.
Furthermore, today you have ideas about how the software will be used, and tomorrow those ideas will be changed, and you will discover that the dividing lines you have set up between your modules are not optimally efficient. This will result in simple-seeming enhancements being prohibitively expensive....unless you start crossing your module boundaries directly. The pressure to do this is too intense to resist, and you wind up violating the modularity of your design in the name of getting it out the door in time.
Sooner or later your system will get complaints that it is too slow, and you will need to open further holes through your module boundaries in order to optimize performance within acceptable delivery deadlines. You will warn that these are quick fixes which must be refactored properly after their release, but the opportunity to do this never comes.
As such changes form layers around your ever-growing onion, the core modules will be come too precious to change. Every little tweak you make to a core module will conflict with assumptions made by other modules, and will cause surprising bugs that are hard to detect in QA testing (since test case count grows exponentially with each new feature). This will result in more demand to fix problems without adjusting the core modules, or by making minimal adjustments to them, which will wind up forcing you to further compromise the modularity of your design.
As the complexity of your system increases, the cost of each new feature also increases, which will displease management and prompt them to say "we used to be able to do this sort of thing in a few hours...now it takes weeks!" They will continue to make unreasonable demands of their senior staff until they push said staff right out the door, leaving the ongoing maintenance of this (now very un-)modular system to the junior level programmers who never shared in the original vision of the system, and hence have no sense for where the module boundaries should be, and just fold to managerial pressure to hack in changes as quickly as possible.
Eventually the slow turnaround times and general bugginess of the system will drive clients to start looking for alternatives. Some brand-new ones will be available (some of which written by the disgruntled senior staff, in fact). Thus begins the end of the product, and of the company if this was their flagship.
Scope Creep
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
In the early 70's I had the pleasure of working with both Prof. Tony Brooker and I.R, MacCallum, both of whom had made major contributions to the Atlas timesharing system, Brooker had designed and MacCallum implemented most of the Compiler-Compiler, a meta system that included a parser generator and an MDL with interpreter to walk the parse tree and generate code. I bring these guys up to make a simple point, both were skilled technologists in their own areas, but as many University academics, they were narrow; an could not understand the need for maintenance, incomprehensible looks, the software will ROT ... etc etc.
They just could NOT get their minds around changing external circumstances, working in tiny groups, version control, V2... was easy as was getting their all 3 systems updated. Pre/Post/Co - requisites, SCM, package management, bug-tracking were all to come as the world started to really use this stuff and they also had no communication disconnect, because there were less than 1000 and they all knew each other,
Yes its a HUGE problem, but unfortunately does need real understanding!
Or in as in the case of many projects I have been subject to, Scope LEAP.
No modular system survives version +0.1.
You have a perfectly modeled modular system, and you really think you thought of everything. Only to notice that someone comes up with some problem you didn't take into account. So what do you do? Redesign the whole modular system, unravel all the finished modules and rework them to fit the new interfaces you have to design? Nah, we can handle a single nonstandard data handoff...
Do I have to go on?
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
In this age of 20yo CEOs and single-quarter companies it's hardly suprising that most software is no better than a rigged demo.
Just make it shiny enough for someone to buy the company and then let their support staff of MS trained monkeys deal with it
Then we have the "artists" (in both the software and hardware field) who have survived for twenty years without "all that sh1t". Course, like the CEO, they've gone on to their next challenge long before the chickens come home to roost. And it's not their fault everyone else is incompetent, is it?
I continue to be amazed, on a weekly basis, by the complete lack of experience shown by the actions and products of very large companies.
Oh, I reject the claim that
The author has obviously never maintained hardware: it has bugs, patches, upgrades just like any other part of your system.
-- Butlerian Jihad NOW!
I blame hard deadlines and crunch time, but I'm not sure if that's the real cause, or if it's just a different class of programmer writing video games. The excuse *used* to be optimization, but I don't think that applies nearly as much anymore.
I think the game dev industry is growing up. Part of it has to do with the fact that a mature codebase is pretty valuable nowadays, and modern games are so complicated, those who don't take good software development processes seriously end up with a technical disaster, and it tends to show. It's very rare for a game to be written completely from scratch these days. When I was started, it wasn't all that unusual to start over for each new successive generation of games. Some of the base libraries I'm working on are nearly a decade old - about when the company I currently work for was founded. They're well-tested and highly optimized. We'd be insane to toss these out and start over.
You also have to differentiate between engine code and game code. By it's nature, game code is essentially a one-off project - essentially single-purpose code. Keep in mind that unlike commercial projects, if the requirements for version 2 of the game change, the code can be rewritten. There's no need to preserve old features or functionality unless the new game requires it as well, as each version of the game is essentially a new fork. To some, it looks sloppy to see hard-coded gameplay features, but it makes a bit more sense when you think about it in that light. MMOs do have it a little harder in this aspect, being so long in continuous development, so we have to think about things like preserving file format compatibility and such, similar to other commercial apps.
Engine code is typically treated quite a bit differently. It has to be much more robust than typical game code. A number of games will be using it, so the code has to be both modular and highly optimized, as well as being fairly general purpose. Obviously, that's a fairly difficult combination, so developers typically put a lot more effort into engine code, and tend to be more protective of it.
Oh, and good heavens, yes, we still have to optimize the hell out of our code (oddly enough, I've been doing that all last week). It just happens more often at the architectural and algorithmic level rather than at the functional level like it used to. It's true that we can get away with a lot more, but... we're also doing much more complicated things.
Irony: Agile development has too much intertia to be abandoned now.
Tell me about it... There's so much feeping creaturism going on in my project that I have to make a SAN check to go to work every morning (if I fail, I go).
Doesn't modular programming solve this problem?
Theory, I'd like to introduce you to practice. You two are very different, you should have lots of things to talk about.
May contain traces of nut.
Made from the freshest electrons.
Oh, modularity is undoubtedly a good thing. There's a word for non-modular code: crap.
The problem is all the things you *can't* solve with modularity. Having chosen the wrong architecture because you had the wrong conception of the problem isn't fixed by modularity. You can swap out modules all day long and it won't help, because the problem is how the modules fit together. Then there are the interdependencies that exist outside your architectural conceptualizations; that pretty much includes most security issues. Your "modularity" is an abstraction you impose on the physical reality of computation. The black hats peek under the covers and find all the couplings you can "safely ignore".
Then there are the things your module depends on: library versions, frameworks, operating systems, databases etc. The answer to this is that most "modules" aren't really very modular because they're coupled to these things; I think of these couplings as "vertical" couplings as opposed to "horizontal" couplings with other modules I create. The best answer is to de-couple modules from those things too, to isolate dependencies in a very small number of interface modules. But that's a lot of work, and a lot of times you've got to do it with programmers who have something like "Struts for the Ignorant Programmer" open on their desk, giving them concrete, cut-and-paste examples of how to *couple* their code to some version of a framework. Those programmers are so far from the ivory tower they're living in the treacle well.
That's the problem of craft in a nutshell. You don't have unlimited time, but pure expediency can waste more of your time than anything else. You can start chopping down a tree sooner if you don't bother to sharpen your ax, and if the boss judges progress by ax cuts you'll probably end up doing it that way.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
You mean you don't remember Flash for ENIAC?
So what do you do? Redesign the whole modular system, unravel all the finished modules and rework them to fit the new interfaces you have to design?
If you have to do that to accommodate a typical change in requirements, then your system was never really modular at all, was it?
Modules represent, among other things, units of change.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.