Slashdot Mirror
How To Deal With 200k Lines of Spaghetti Code
An anonymous reader writes "An article at Ars recaps a discussion from Stack Exchange about a software engineer who had the misfortune to inherit 200k lines of 'spaghetti code' cobbled together over the course of 10-20 years. A lengthy and detailed response walks through how best to proceed at development triage in the face of limited time and developer-power. From the article: 'Rigidity is (often) good. This is a controversial opinion, as rigidity is often seen as a force working against you. It's true for some phases of some projects. But once you see it as a structural support, a framework that takes away the guesswork, it greatly reduces the amount of wasted time and effort. Make it work for you, not against you. Rigidity = Process / Procedure. Software development needs good processes and procedures for exactly the same reasons that chemical plants or factories have manuals, procedures, drills, and emergency guidelines: preventing bad outcomes, increasing predictability, maximizing productivity... Rigidity comes in moderation, though!'"
no comment...
Wouldn't that be Linux? It seems to work fine for me.
If something has become spaghetti over 10-20 years, then no one cared that it became spaghetti over 10-20 years. And it will still be spaghetti over the next 10-20 years. Fixing something like this requires a commitment from management, which means money. If the management of the project aren't convinced that cleaning up the development process is worth the initial investment for the long term, then they choose to deal with the constantly higher costs forever.
Something like this makes me think that this is one of those problems that get pushed off for someone else to deal with later. And the next person perpetuates this, by doing the same.
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
Outsource it to India!
I advise printing it out and posting it in the hall in printed form, annotated with contributors. Paper the walls with it. You would be surprised how much more efficient that makes a coder even if it's not ever read.
Also, for debugging sometimes you have to get down on the floor with 50 pages of fanfold and work it out. Decorum be damned. A PC only has so much vertical resolution and sometimes you need more than to peek at the code through a sliding window.
Help stamp out iliturcy.
I knew I read this before:
http://programmers.stackexchange.com/questions/155488/ive-inherited-200k-lines-of-spaghetti-code-what-now
That article is linked in the first sentence of the summary.
Well step 1 would be to lose the attitude.
It's code, it may be in an obsolete language, it may be not to the best industry standards, but its code and it's got enough knowledge in it, that nobody wants to throw it away, and they hired you to maintain it.
Step 2, I don't know why you would define a process before you understand the code you are to apply the process too?
Seriously, wtf is all the process stuff about, you're the sole programmer, any rules you set are rods to break your back when you first hit a piece of code you have to break the rules.
Step 3, you serve them. If you want to port it to a more modern maintainable language, choose one that's easy for THEM to transition to. They've got the knowledge that drives the company, not you, you are the cleaner here. If the phone rings your turn off your vacuum and let them do their job, Mr Cleaner. Nobody gives a fork if the cleaner has best industry procedure for cleaning an office.
Step 4, break it down. tiny bit by tiny bit, port to a new CLEAN structure, bit by bit. They wrote it, they can identify the core stuff.
Step 5, once you've ported it, along comes an engineer with a code written to the old language and old methods. Again, that's fine, put away the process manual, these are the experts, if that's the language he can communicate to you in, it's fine, you can understand it, you can port it, you can help him speak the modern lingo. Don't quote your processes to him, you're just the cleaner.
As for this:
"Software development needs good processes and procedures for exactly the same reasons that chemical plants or factories have manuals"
That's someone who *implements* things, typically a bolt together module manager. He is not someone who creates *new* things. Because news things don't come with manuals. You don't know the rules of how they work till the problems needed to make them work are solved. One assembles Microsoft IIS blocks, the other works for Google on image processing. Which are you?
All the advice to rewrite it is misguided. Maybe rewrite small parts that you need to to keep it working on new hardware, or whatever, but if it works, I would think that wholesale rewriting is asking for trouble. The Ars article is full of great advice about what you should do to manage a large codebase going forward, but actually it doesn't really address the question of what to do about a large legacy codebase that wasn't written with best practice. The best software is written by incremental improvement of what went before (no matter how badly written, as long as it meets its specification) - big projects written from scratch usually fail.
If things go well, your bank account will know, for example.
What does it even mean "my boss doesn't really care about" a project? Is his vision somewhere else, but has given you the job of guarding the crucial machinery on which everything else builds?
(And being homeless sucks badly, by most accounts.)
200k isn't something you're going to rewrite in a couple of weeks. I think the absolute maximum you could get (for one very skilled person) would be about 5k-10k per week. Rewriting would take on the order of half a year.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
I have spent most of my career as a software developer inheriting and updating such spaghetti code bases. Here are few remarks and some of my experiences around this:
In summary, don't be too scared of a legacy spaghetti code base. These things can be understood well enough in time to refactor or port to a new platform.
The difference between software development and most manufacturing is that they produce the same or very similar product thousands or millions of times where we produce a different product every time. This "building an app is like mass producing a chemical" philosophy is one reason why most software shops have insane amounts of unneeded documentation and overhead. I certainly agree that some standards, processes and documentation is needed but it should be kept to the bare essentials as every bit of work done that doesn't directly build a product could well just be a waste of time.
Rewrite it from scratch using the spaghetti code version to run correctness tests to verify you haven't changed the behaviour.
And just how are you supposed to write "tests for correctness" when the very concept of what is "correct" is embedded in the code?
Any such tests would embody your own notions of what is correct based on your understanding of a codebase that cannot be understood.
Furthermore, Doom is quite a different thing. You have an end result that can be somewhat different and it doesn't matter - it could render textures such that they appear rather different but if you find it visually OK then it's fine.
No such luck with business software which usually has extremely rigid and exactly output, often output other systems are depending on being just so. There is no room for alteration of behavior, yet as I said no-where exclaims all of the features of the output you cannot possibly understand....
I agree with a few responses that the only way to proceed is to re-write tiny parts, that at most affect one other system in the company - with the explicit buy-in from those other groups something may change, and the understanding you may have to back out your changes wholesale if you cause too much disruption.
Can't get buy in to proceed? Then quit or work with the code as is.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
I advise printing it out and posting it in the hall in printed form, annotated with contributors. Paper the walls with it. You would be surprised how much more efficient that makes a coder even if it's not ever read.
Also, for debugging sometimes you have to get down on the floor with 50 pages of fanfold and work it out. Decorum be damned. A PC only has so much vertical resolution and sometimes you need more than to peek at the code through a sliding window.
You can also flip through fanfold like a book and see two pages at a time with an option to turn several pages at once.
Usually when the inxperienced programmer does the rewrite, "initial results are promising".
That is, it is possible to quickly build a basic framework with badly designed error handling and
only part of the requested features.
When it is tested in real life, lots of omissions and problems are found.
Once these are fixed, the result is as messy and unstructured as the initial system was before
it was rewritten.
Your employer probably isn't interested in spending the time and money on a re-write. Nor are the clients going to be interested in waiting that long for new features, either.
You will be made to figure it out and add features, or you will be shown the door.
In the past years I have been several times in such a predicament. Huge amount of code and the function of the system is not completely clear. The original developers are gone, the system isn't well documented and only a handful of people know how how it should behave. As a matter of fact, tomorrow I will start coding on one system we can no longer support as hardware, OS and used libraries and frameworks are outdated and/or discontinued.
Reengineering and rewriting is usually the best option. However, you need skills and experience in order not to make the same mistake the previous developer did. Of course, management must trust and approve your actions.
A few dos:
* Learn at least UML use cases, components diagrams and sequence diagrams.
* Make use cases and check these with affected parties.
* Start of with a rough component model of the new system.
* Make a clear picture which nodes (hardware + OS), subsystems (units performing a function), software components (modules containing data, modules performing a function, etc...) and agents (users, triggers/schedulers) are involved.
* Draw the interactions between the subsystems and/or software components.
* Clearly document which interactions are on-line and which ones are batch/background/off-line.
* Specify interfaces. (Used file formats, protocols, software library interfaces if you will.)
* Slowly refine your model until you feel comfortable with it.
* Make a rough class model and keep usability and maintainability in mind. Backtrack if necessary.
* Divide software components between "dumb" containers of information (e.g. plain Java beans) and components performing functions (business logic if you wish.)
* Decide which interfaces to make public and which not.
* Describe restricted/private bits of code just enough for maintainers to understand them. And nothing more than that.
* Make as much unit as necessary for your components. Unit test enough functionality.
* Communicate your results regularly and refine your model where applicable.
* Define integration tests and do these very seriously.
* Define regression tests and perform these very seriously.
* Make involved parties accept parts of the system according to performed integration and regression tests.
* Try to plan gradual decommissioning of the legacy system.
* Document the system "enough". System architecture (from UML), references from architecture to code, installation manual and operational manual are the most important ones.
* Try to achieve longevity in the documentation. Abstract details and convince involved people that that is a good thing.
* Define 1st, 2nd and 3rd level support. Preferably you should remain 3rd level support to better enjoy sleep.
* Conform to standards and practices if they reduce discussion and enhance clarity.
* Use well established techniques. E.g. JPA and JAXB.
* Allow well established component manufacturers to make your programming life easier. E.g. Apache Commons.
* Be tidy.
A few don'ts:
* Avoid OO pattern overkill.
* Don't take the quick and dirty option too quickly. Those decisions will haunt you eventually.
* Avoid making everything public. Documenting and maintaining public interfaces is more expensive.
* Try to avoid big bangs.
* Avoid less well established component manufacturers. My next project did use components from less established component manufacturers and their sell by date has generously expired.
* Don't allow babling "architects" to make a mess of your system. But don't alienate them either.
I may have forgotten a few things but this is all stuff I consider even for smaller projects.
I hadn't the slightest objection to his spending his time planning massacres for the bourgeoisie... (P.G. Wodehouse)
If that's the case, the stated or implied directive is don't break this. Which means probably no major rewrite.
That depends on whether it actually needs to be that complex. Implementing a project that is 200KLoC is not the same as implementing something that is functionally equivalent to an existing codebase that has grown to 200KLoC. If the project is as described, I wouldn't be surprised if it could be implemented in under 20KLoC, possibly less if there are some existing libraries that can be used.
I am TheRaven on Soylent News
If that's the case, the stated or implied directive is don't break this. Which means probably no major rewrite.
This should be +5 Insightful. It's the bottom line for production software.
You've never seen real spaghetti code if you believe this. In a really nasty ball of spaghetti, there's nowhere to make the cut; any significant section of the codebase essentially depends on the entire codebase. Among other sins: code at low level will make decisions essentially based on which high-level caller ultimately called it.
Working Effectively with Legacy Code http://www.amazon.co.uk/Working-Effectively-Legacy-Code-ebook/dp/B005OYHF0A/ref=sr_1_1?ie=UTF8&qid=1344190021&sr=8-1