How To Develop Unmaintainable Software

jones_supa writes "Greg Jorgensen specializes in debugging, fixing, maintaining, and extending legacy software systems. His typical client has a web site or internal application that works, more or less, but the original developer isn't available. Greg lists some things you can do in your own software projects to keep him in business. In summary, the list goes as follows: Customize your development environment a lot, don't make it easy for the next programmer to start working on the code. Create an elaborate build and deployment environment and remember to leave out the documentation. Don't bother with a testing/staging server but instead have secret logins and backdoor URLs to test new features, and mix test data with real data in your database. Don't bother with a well-understood framework, write everything from scratch instead. Add dependencies to specific versions of libraries and resources, but don't protect or document those dependencies. For the icing of the cake, use the coolest mix of cutting-edge programming languages."

Comments

Tied this

[Urban Nightmare]

and I was still replaced.

Re:Tied this

[Samantha Wright]

Well, maybe you should try this much more venerable, thorough, and entertaining encyclopedia of unmaintainable code tips. Should make you as immovable as the CEO's firstborn.

Re:Tied this

[crunchygranola]

Well, maybe you should try this much more venerable, thorough, and entertaining encyclopedia of unmaintainable code tips. Should make you as immovable as the CEO's firstborn.

He has an unfair advantage in the area of unmaintainable code since he is writing about C/C++.

Old news

[Pharmboy]

This is old news. These methods have been around at least as long as C has. It only works in isolated situations and doesn't make you a good programmer. Or person.

Re:Old news

[lgw]

Not every programmer was around when C was new. Everyone needs to learn these lessons, and preferably not the hard way.

I'm mean, you'd think anyone starting a major project today would use version control, that shouldn't even be a question, right? Right?! Sigh.

Some of his other stuff on his page is quite entertaining as well, like this one on Abject-Oriented programming.

Re:Old news

[dgatwood]

How many closed source apps do I have on my computer of that quality without knowing it?

"Most of them" would be my guess.

Re:Old news

[hurfy]

lol, 25 years ago we had ONE programmer able to maintain our software as he had done much of the customization of an out of the box accounting system. Bloody thing was so complex to start with noone wanted to touch it before we fixed it up, much less afterwards. Upto 10 Megabytes worth of code but no more than 32K loadable in memory. Must have been 100's of small chunks of code all calling each other in and out of memory every other keystroke.

I managed to customize some reports but that was only part i ever dared to touch.

The real kicker...written in a version of BASIC.

Actually a pretty impressive system and worked well for 15 years. Always wondered how they managed to write such a monstrosity. Kept picturing a blimp hanger with a flowchart wrapped around the walls.

Re:Old news

[girlintraining]

This is old news. These methods have been around at least as long as C has. It only works in isolated situations and doesn't make you a good programmer. Or person.

A lot of that list is pretty damn arrogant, or at least naive. Let's start with just the ones the submitter quoted;

Don't bother with a testing/staging server but instead have secret logins and backdoor URLs to test new features

If you're so well known you can dictate terms to your employer, then maybe you can get away with this... for awhile anyway. But most of us, who code for a living, have to do the best with the tools we're given. We can ask for a testing server, but whether we get one is another matter entirely.

As for 'secret logins' and 'backdoor URLs'... every login is supposed to be secret. If you can get a list of userIDs via any external method, that's a security risk. And as for 'backdoor URLs'... Not everything should be accessible to the google crawler.

and mix test data with real data in your database.

There's any number of failure modes in complex code in which shoving random inputs into it will cause it to break in unexpected ways that don't become apparent until properly-formatted data is submitted after. I think what the author was trying to say was not to put invalid data into a production environment on purpose; It might make the app shit a brick. Unless of course that's the only environment you have to test. See also: That 'routine backup' of the EBT system that caused 13 states' food stamp programs to die in place.

Don't bother with a well-understood framework, write everything from scratch instead.

Again, pure arrogance on the part of the author; Well understood frameworks tend not to be highly optimized. If you are dealing with something where performance is absolutely critical, writing it 'from scratch' will probably get you better performance than taking one of those 'well-understood frameworks' out for a drive. Because as most experienced programmers know -- well-understood frameworks also have a lot of extra features and bloat they don't need. See also: .NET

Add dependencies to specific versions of libraries and resources, but don't protect or document those dependencies.

Okay, that's just plain sloppy. Well, 1 out of 5.

For the icing of the cake, use the coolest mix of cutting-edge programming languages.

The choice of programming languages matters very little, except to those who lack the Tao of programming. For those who are not One with the code, who do not grok the larger patterns that permeate the spaces between that which only the machine understands, and that which only the human understands, the language choice is the difference between life and death. For those who have transcended languages... it is the patterns they seek to understand. The language will take care of itself. It's like saying "Love" in French, German, Russian, or English. It's the same damn thing, and it's possible to understand someone who loves you, even if you don't understand a bloody word they say.

Bottom line here is that this 'list' is just the logical conclusion of a career of mediocrity spent shifting blame around instead of working the problem. Real programmers can use anything; And frankly... they usually do. -_- I've seen some completely hair-brained solutions that are coded in Perl, cgi-bin, bash, PHP, and little chunks of C here and there that work flawlessly and at great speed because the programmer took the best of each of those languages and leveraged them in the solution. Of course, for every good programmer who does this, there's ten more who tried the same thing and wound up with a pile of half-working shit, sooo.... YMMV.

Re:Old news

[lgw]

Well understood frameworks tend not to be highly optimized. If you are dealing with something where performance is absolutely critical, writing it 'from scratch' will probably get you better performance than taking one of those 'well-understood frameworks' out for a drive.

Sure, of course, why not. It's not like your time is valuable or anything. And naturally, you will make this decision before coding starts, and thus before any measurements that the framework was actually slow, or for that matter that it was part of the perf bottleneck of the system. Nothing says "job security code" like elaborate cryptic algorithms to improve asymptotic performance of some in-memory task on a system where CPU never got above 5% in the first place.

Let's not forget how bug free a framework used widely by thousands will be - there's no job security there at all if there's no bugfixing! But if we write our own, then there will surely be some horrific bug in the field that we can work until 4AM to fix, and management will call us heroes!

The main thing, of course, when writing your own replacement for some common library that some new hire would already know is to avoid any documentation, and ideally abandon the effort halfway through, because the project ran out of time, only ever use 10% of what you wrote, and never document which 10% - heavens no!

To make doubly sure that no new hire could ever support this code, remember to follow this key element of Abject-Oriented Programming:

Documentation
  It's said that code should be written for people to read, so it follows that documentation is written for no one to read. Documentation should be written for every new module and then maintained as changes are put into production, or at least the next time thereâ(TM)s a lull in the workload.

A good time to write documentation is when someone in the department gives two-weeks notice: use that time to make sure the departing team member documents all of their code.

Cluttering up the source code with lots of comments explaining what the code is trying to do is distracting and slows down the compiler. That's why abject shops that follow "best practices" keep documentation in a document management system where programmers can't accidentally delete it.

Sage advice, I'd say.

Bottom line here is that this 'list' is just the logical conclusion of a career of mediocrity spent shifting blame around instead of working the problem. Real programmers can use anything

Perhaps you missed the part that the author of TFA makes his living answering panicked calls to bugfix production legacy code, presumably because some "real programmer" did in fact "use anything", and this advice is based on long experience of what makes code unmaintainable. My own experience/advice lines up with his rather well in this area, I'd say.

Re:Old news

[Draknor]

I think you are missing the author's points in your rebuttal.

Testing server -- maybe it's not ALWAYS possible with expensive enterprise-y server software, but just about anyone can spin up a VM on their dev machine to simulate (with varying degrees of accuracy) a production environment.

Secret logins & back doors -- you mean you've never created a "god" or "super admin" account (or "secret URL") that could access all kinds of technical / debugging info that regular admins/users shouldn't see? Having such an account means your application has additional logic & code paths to support -- code that's probably not being adequately (if at all) tested and probably has bugs, some of which may be security-related.

Test data in production - yeah, I've worked on systems (such as health care IT systems), where project team puts test data in the system. It sucks for operational users. "What, you mean Dr. Smith doesn't have a 2pm appt w/ TEST, DUMMY today?" "The compliance dept just got a call about a six-figure insurance claim to Medicare for a pregnancy-related hospitalization for a DUMBASS, JOE -- anyone know about that?" Test data belongs in a test database.

Frameworks -- in my humble opinion (and I'm not exactly alone) there are very few situations where run-time performance is actually "absolutely critical". More often (in my experience) time-to-develop "performance" is a bigger factor, and rolling-your-own (to get alleged better run-time performance) will cost you development time, bug-fix time, QA & testing-time -- which, for the vast majority of applications, will cost more than simply buying faster hardware (the happy medium way is to optimize just the critical slow parts in your application that the framework handles sub-optimally).

Choice of languages -- again, I think you missed the point. Any language is fine. His point is to keep your project consistent. If you've developed a hair-brained solution that involves Perl, cgi-bin, bash, PHP, and chunks of C -- it probably works great and flawlessly, like you said. Until that programmer (1) retires or (2) gets hit by a bus. Then the junior programming intern they hire to take his place is screwed. And that's the author's point -- write maintainable code. A mismash of languages & bindings "because they are cool" may function, but it's not maintainable. If your star programmer has this Perl/bash/PHP/C contraption and it's well-documented and logical, then maybe that junior intern will take it over and, with a bit of a learning curve, master it. But if your programmer used 4 different languages because "it's cool to make it complicated" -- well, good luck.

Re:Old news

[girlintraining]

Sure, of course, why not. It's not like your time is valuable or anything. And naturally, you will make this decision before coding starts, and thus before any measurements that the framework was actually slow, or for that matter that it was part of the perf bottleneck of the system. Nothing says "job security code" like elaborate cryptic algorithms to improve asymptotic performance of some in-memory task on a system where CPU never got above 5% in the first place.

You devoted considerable effort to coming up with a response. Yet you failed in every way to realize that the point being made was that good programmers often depart from guideline, and with good reason. If this was less a list and more an advice column, then I wouldn't have picked it apart. But it comes off as a "dos and don'ts" list, and frankly... such a list has neither educational nor informational value.

But whatever, if you want to continue down this path, okay then; Look smartass, you can't test a framework against your own hand-crafted code because you don't know how your hand-crafted code is going to perform until you build a prototype. So fucking duh, yes, the test happens after. And if you'd sat down on a whiteboard and stenciled out a block diagram of what your app, or at least the part of it you're working on, does... you'd already know which branches are going to be most used, and where special attention needs to be paid towards optimization. I already provided the caveat that where performance is critical. I'm not talking about business apps or databases when I say these things. I'm talking low level shit like how they're building switches that begin writing out a new packet while the previous packet is still incoming to gain precious milliseconds in high frequency trading. You aren't writing that with .NET or whatever framework you're upset I dismissed from consideration. PERIOD.

Also "But if we write our own, then there will surely be some horrific bug in the field that we can work until 4AM to fix, and management will call us heroes!" is about the most moronic thing you can say. Management doesn't consider your project resulting in piles of overtime "heroic". It considers you a poor performer and will be looking to replace you.

Perhaps you missed the part that the author of TFA makes his living answering panicked calls to

Yes, I got that. Wanna know something? Ask a police officer how honest the average person is. They'll tell you the average person is a slimy son of a bitch who's probably packing heat and looking to kill you and everyone you ever loved or knew. Wanna know why? Because that's their day in and day out experience. Ask a surgeon how to fix a problem, and every answer will involve a knife. It's the same everywhere.

You got suckered in by assuming one man's personal experience counted as fact. It doesn't. I recognize that in his particular line of work, he runs into these problems all the time, but that doesn't mean they were bad choices at the time they were made... it means that by the time it got to him, it had become a problem. And that's all.

Look, bottom line here is that when you code professionally, you are engaged in a massive balancing act between deadlines, error checking, other people's contributions, etc., etc. Coding professionally is an organic process, and while everyone knows what the best practices are (documentation, QA, modularity, readability, etc.) these are ideals to strive for, not things we can actually achieve on every single project. That would take a level of super-human ability that nobody yet born possesses. It's like the old adage in IT: "Cheap, Fast, Right. Pick any two."

My own experience/advice lines up with his rather well in this area, I'd say.

My own experience is that a mind intent on something refuses to admit to better things. In programming professionally, I've come to refer to this as target fixation -- they get tunnel

Great little article

[Anrego]

Every one of these points hits the nail square on the head.

The key to take out of this is: document document document! At minimum you should have a set of instructions to re-build your dev and build environment. "Insert the <your company> dev workstation image v4" is not allowed to be a step! Your elaborate continuous integration multi-tree setup and mountain of environment setup scripts and template directories are great until the guy who set it up takes off and you have to upgrade something. Ideally a set of instructions talking to the motivation of certain decisions, roadblocks encountered, etc.

One thing the article doesn't have is have lots of 3rd party tools and keep the license servers/license files on whatever box is most convenient for the dev working on it at the time.

Re:Great little article

[phantomfive]

"my code is self-documenting."

A statement that is always true as long as no one else has to read it.

Re:Great little article

[phantomfive]

Let me guess, you are the one who writes self-documenting code?

Use a language that no one ever heard of

[jfdavis668]

We inherited a system written in a language we never heard of. It wasn't just uncommon, it was way out there. Unfortunately, It was a while ago and I don't remember what it was called. We ran the system, but had to rewrite it immediately. Luckily, the reason we inherited it was because the old system was crap, and the customer was willing to pay us to redesign it. If the previous vender had put a little more effort into it, we wouldn't have been able to take the business.

Frameworks

[Tablizer]

Don't bother with a well-understood framework, write everything from scratch instead.

Is this saying only use widely-available frameworks (to do it "right"), or don't write your own frameworks from scratch? I have time-tested C.R.U.D. frameworks I often use for web projects that I've improved and tuned over the years. They are reasonably well-commented.

Programmer not the whole story

[jasnw]

Yeah, yeah - code clean, test-test-test, document-document-document, have separate test/run machines that are configured the same, yada yada. This is all well and good, and any halfway-decent developer knows all this. However, software development is not done in a vacuum and each and damn near everything mentioned is involved in cost/time benefit analyses when crunch-time comes (which it always does). With some exceptions, when I see a company that's saddled with horrible old legacy codes that nobody can understand, often a large measure of this is paybacks (for not adequate funding and poor schedule planning) being the bitch that they are. How to do things the best way are well known, it's just that the best way is more expensive (in the short term, which is the only term business understands these days) and takes more time than the average business will wait. If the bottom line is get something done that sorta-kinda works as fast/cheap as possible, you get spaghetti code that even the guy/gal who developed it can't follow.

There's more than one way to do it

[moonwatcher2001]

The nastiest project I ever took over was written in Perl by ten diferent programmers eah using diferent features of Perl

Bad management

[EmperorOfCanada]

Often it is how a project is initially managed that results in an unmaintainable system. A few simple mistakes can send a project straight to hell from the very beginning. A simple one would be to allow the senior management to firehose new features at the project far faster than the developers can build them. Another would be to allow the wrong people to pick the core components. That is how bad databases/languages/operating systems can be chosen. Then you get the next layer of wrong when people simply code and architect badly.

An example from my past was a company that I interviewed with (and was offered a job at) that was using Lotus notes to build a huge educational system. They had a PhD as the head of development and they were keeping the details secret until they made their offer. When they told me Lotus Notes I just laughed; I thought they were joking. I told them that building their system out of Lotus Notes was like building a car out of sand and white glue. At first you will quickly have the broad outline of a car but that as you start to work on the hard bits that you will never finish; Ever! A couple of years later the company imploded with no real product just a bunch of sales demos.

All Code is Terrible

[TempleOS]

You will never find good code in a project. All code is awful. Every programmer who looks at a project says it's shit.

Re:All Code is Terrible

[phantomfive]

For code, "Structure is the key to understanding." That is the missing thing from many pieces of documentation, an overview that gives the structure of the code. Specs are a good way to get that overview.

No such thing as 'unmaintainable' software

[johnlcallaway]

Only lazy developers. I've worked on plenty of legacy software over the years that other people wouldn't touch, and the common thread was they were either too lazy or just not smart enough. It takes determination, patience, and a lot of detail to work on it. But it was all far from unmaintainable. And the end result was I was the one kept around during layoffs instead of the guys that said "I can't do it" (translation ... it's too hard for me.)

OK .. no source code at all pretty much makes it difficult to maintain. Or not being sure if what is in use is the current code.

As for the 'use a framework' .. what BS. Frameworks come and go. What is well used today can be a long forgotten, unsupported mess 5 years from now. Like NetBeans and Swing?? All those GUIs I wrote 5 years ago probably now fall into the 'unmaintainable' software category, even though all the code is actually there, and anyone that understands GUI programming at a basic level can still make modifications. It's not easy (i.e. not for the lazy), but it's possible. And how many times has a framework version made upgrading difficult because of extensive changes???

It's OK to use Hibernate, ICE, and all that stuff that makes the job easier today, but learn how stuff actually works and only use it when necessary. It will make you more valuable in the long run, and the code more maintainable in the future. If I hear one more ignorant programmer tells me 'But Hibernate can write SQL that way even if it is more efficient' I'm going to shove a SQL manual up their ass and tell them to actually learn something.

... says the self-taught programmer...

[MasterOfGoingFaster]

Hey, glad I could keep you in business.

be sure to mess up SQL Server code as well....

[ClassicASP]

Use long looong looooooooong stored procedures (20,000+ lines long) with lots of spaghetti SQL code and very little documentation. And never use identities and primary keys; instead just get the max ID number on the fly and use source code to get the next integer-up for the next record insertion to a given table. Make sure the table and field names have almost no relevance to the data they hold. use views liberally, and query one view on top of another view on top of another view on top of another view. If possible try and work something out to where there simply isn't any relational design to the database at all, and instead just use redundancy everywhere. Or, if you really want to be nasty, you could go the opposite route and figure out some kind of a mindbending rubix-cube-like puzzle where there is only one table with only 3 fields in the entire database, and nearly all table data of any sort is stored in one field in that table, and just rely on a myriad of SQL joins to emulate the behavior of relational database design.

Re:How To Develop Unmaintainable Software

[Beardo the Bearded]

In all fairness, outsourcing it to Canada made sense. We're cheaper, we have health care already, and speak English with an approximate degree of usefulness.*

So, on behalf of our country, I apologize for any inconveniences you have suffered from the sheer shittiness of the ACA software. As a measure of our sincerity, you may pick up one(1) bottle of maple syrup from our strategic reserve.

*offre non valable au Québec

Re:Write your own language!

[dinfinity]

Just get a hold of lex & yaxx, antlr or your favoriate tools and write your own language. You could probabyl abuse Lisp macros anough to do this too.

It seems you are already well underway!

He Didn't Delve Into Specifics

[Greyfox]

How about "Set up a global int l and set it to 0, and a global int O and set it to 1. Or sometimes l, depending on what file you're in." This works better or worse, depending on your font. Or in C++, have every class inherit from every other class. I've seen this done.

A couple times I've run across some jackhole who is in love with dynamic shit and who wants to check java object code into an SQL database and dynamically load it into a program. Nothing quite spells fun like wondering where the hell the code is branching to, and spending two days figuring out it's to an object in a database the code to which has been lost 4 years earlier.

For a long time there in the '90's and a bit later, version control wasn't a normal thing. If you had to get management approval to set one up, that usually would start up the sinking feeling about that job sooner rather than later. Nothing quite like having to justify the addition of something they should have never written a line of code without in the first place. Nevermind trying to get overhead for writing unit tests. Assuming unit tests are even possible. For most of the projects I've run across, the coupling is so tight there's no way to get in and test discrete functionality of anything.

For the guys not using version control, "Lose the source code" is a good one. You don't even have to be malicious about it (Though malice helps.) Just build the thing on your personal dev box, deploy the executables to production and never mention it when you leave. Bonus points if they walk you out during a layoff.

"Don't set up a build system" is another good one. Gotta love those projects where you have to run a top-level build three times before it produces an executable!

Re:He Didn't Delve Into Specifics

[Greyfox]

Really? Try pointing out that document revision control is mandatory under the ISO 9000 process. That should get the ball rolling pretty quickly. Or A ball, anyway.

Then Quit in a Huff When it's Only 2/3 Finished.

[rueger]

At one organization I inherited a fund-raising database developed with much arrogance in Filemaker Pro. Except that the FMP "Guru" quit in a huff when it was only about 2/3 functional.

Despite this the thing was used - with many, many crashes - for two years because it was the best thing that they had. And because Mr. Guru was a pal of half the staff.

Did I mention that Mr. Guru also refused to hand over any of the logins that would allow someone to fix or administer this thing?

We eventually wound up moving over to MS Access (GOD YUCK EVIL!), and massaging, fixing, cleaning, importing ten years of data, each year on a different format, file type, or with different fields. We actually managed to turn it into a reasonably useful fund raising tool. I mean, aside from the downside of using MS Access.

Documentation? HA HA HA HA!
Instead we had lots of on-screen buttons though that didn't do anything because he never quite got around to it.

Documentation is overrated

[140Mandak262Jamuna]

Documentation goes out of synch with the code very quickly. The only thing worse than working on someone else's code without documentation is working on someone else's code with incorrect documentation. The problem is so old Dijkstra allegedly said, "Always debug code, not the comments".

Oh, yeah someone will tell me I am doing documentation wrong. How come "you are not doing agile right" is a valid response but "you are not doing watefall right" is not?

Nothing wrong with writing from scratch

[FuzzNugget]

As long as you document, label and organize your code coherently and logically in a solid language, coding from scratch (or near scratch using libraries of your own) can be a good thing. It avoids pitfalls like vulnerabilities that affect common code bases, massive libraries that take friggin' *ages* to load on an i7 with a fast internet connection or relying on the developer(s) of library X to add feature Y ("sorry, we can't add that feature yet Mr. Client, it's uh ... because of complicated programming reasons, basically, we're relying on a third party and we don't know how to do it ourselves")

Re:My favorite

[Jeremi]

You fool! That code is critical, it's the speedup loop. For the v2.0 release you delete some of those loops, and presto, your app is significantly faster -- everybody will think you're an optimizing genius.

Be careful to include some calls to rand() or similar in your loops, though, otherwise the optimizer in your compiler might screw you over by removing the loops in v1.0, denying you your speedup in the new version.

Re:There's more than one way to do it: FIXED!

[gmhowell]

The nastiest project I ever took over was written in Perl

Came for the anti-perl quip, left satisfied.