Slashdot Mirror
Lessons From Six Software Rewrite Stories (medium.com)
A new take on the age-old question: Should you rewrite your application from scratch, or is that "the single worst strategic mistake that any software company can make"? Turns out there are more than two options for dealing with a mature codebase. Herb Caudill: Almost two decades ago, Joel Spolsky excoriated Netscape for rewriting their codebase in his landmark essay Things You Should Never Do . He concluded that a functioning application should never, ever be rewritten from the ground up. His argument turned on two points: The crufty-looking parts of the application's codebase often embed hard-earned knowledge about corner cases and weird bugs. A rewrite is a lengthy undertaking that keeps you from improving on your existing product, during which time the competition is gaining on you.
For many, Joel's conclusion became an article of faith; I know it had a big effect on my thinking at the time. In the following years, I read a few contrarian takes arguing that, under certain circumstances, it made a lot of sense to rewrite from scratch. For example: Sometimes the legacy codebase really is messed up beyond repair, such that even simple changes require a cascade of changes to other parts of the code. The original technology choices might be preventing you from making necessary improvements. Or, the original technology might be obsolete, making it hard (or expensive) to recruit quality developers.
The correct answer, of course, is that it depends a lot on the circumstances. Yes, sometimes it makes more sense to gradually refactor your legacy code. And yes, sometimes it makes sense to throw it all out and start over. But those aren't the only choices. Let's take a quick look at six stories, and see what lessons we can draw.
For many, Joel's conclusion became an article of faith; I know it had a big effect on my thinking at the time. In the following years, I read a few contrarian takes arguing that, under certain circumstances, it made a lot of sense to rewrite from scratch. For example: Sometimes the legacy codebase really is messed up beyond repair, such that even simple changes require a cascade of changes to other parts of the code. The original technology choices might be preventing you from making necessary improvements. Or, the original technology might be obsolete, making it hard (or expensive) to recruit quality developers.
The correct answer, of course, is that it depends a lot on the circumstances. Yes, sometimes it makes more sense to gradually refactor your legacy code. And yes, sometimes it makes sense to throw it all out and start over. But those aren't the only choices. Let's take a quick look at six stories, and see what lessons we can draw.
Nowhere in this article is any flaw in Joel's pronouncement to never rewrite found. The only of the six stories that was a rewrite of an existing product is Netscape and that was a disaster. The fact that Firefox came out years later doesn't make it any less of a disaster.
In the rest of the cases, no one is re-writing their existing product and letting the competition catch up. No, what they are doing it making a new product and moving users over if possible. That makes sense and that is what I learned from Joel's original article.
Making this seem like some kind of intriguing rethink of rewriting legacy code is false and click-baity.
I disagree with your disagreement. Rewriting a working application is juggling with a loaded gun. You say "if you carefully learn from past mistakes" but this is almost impossible in any organization I've ever seen. All those snippets and dirty patches are not documented properly. If there are comments, they may (or may not) be out of date. Maybe there is a reason they chose a weird data structure but that guy retired 15 years ago. So you do something "proper" and find it is 10X slower. Oops.
If I've learned anything, I've learned you gotta let sleeping dogs lie. More patients are killed by doctors in this business than anything.
Sometime you need to disconnect your software from your ego.
A lot of full rewrites are primarily due to the fact the developer wants to be the Hero, toss out the old broken code, and make a new superior on in their own image. (Which then over times becomes old and broken)
Also trying to keep bad code on life support, because you had invested so much into it over your time there, is just as bad, as the application barely meets the business needs anymore, and every fix is getting increasingly more complex.
Sometimes what the code really needs, is just some reorganization. (Move your function and methods in more logical locations in your code so you visualize the problems more easier. Beautify your code, make white spaces consistent. ) Create a script to see where each function is called and where, Delete unused functions. Identify duplicate logic and merge them, or make a parametrized function of of them. It is actually amazing how a little cleanup work can turn a mess into a manageable application for decades.
Also if designing software really try to split Logic and Interface/System call into separate portions. The routine that checks the required field, should be straight logic. Not generating HTML or Form boxes, giving the errors. Despite what your bosses say, assume the platform it is running on will be retired in 5 years, and try to plan to port the application over.
Finally, really try to use basic core components as much as possible, avoid 3rd party tools as much as you can (this includes semi official repositories such as with pip or cargo). Don't be a Hero and make a ground breaking new interface, just use normal component.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
Technical debt is a thing...
Best place I ever worked knew that, and worked to quantify it. That helps a lot when you're trying to figure out whether or not to refactor. In a couple cases, it became apparent that a partial refactor was going to accomplish something like 90% of a full one, with 10% of the effort.
Quantifying your pain-points with even back-of-the-envelope calculations can really help you make good decisions about how to move forward.
Velociraptor = Distiraptor / Timeraptor
Maybe add this as a seventh cautionary tale.
I developed an application for reserving lab equipment for one of the largest server manufacturers in the world. It was capable of automated inventory, software and image deployment, and automated permissions handling that worked with an out-of-band IPMI interface and was capable of getting data as detailed as PCI bus info, CPU and memory information of virtually any server that runs IPMI. I wasn't the first to attempt an inventory management system like this within our company, but it was the most ambitious and long-lived solution by far and was used by 4000+ employees for thousands of servers and virtual machines at its height in over a dozen sites world-wide.
Unfortunately, it was never an official job duty. I did it during free time at work and during a painful point in my personal life where I sunk most of my waking hours into the project. It was developed on a LAMP server when I was still a very novice programmer and despite how gross and hacky the code was, the software ran exceptionally well, had a professional-looking UI, didn't require any kind of management beyond ensuring the hardware was functional and any auto-generated tickets by my application were solved and gained it gained a lot of positive attention. I had an instance at each location and a master one collecting data from those for backup purposes.
I'd always intended to rewrite the code, but life priorities and job duty changes made progress very slow on that, so despite the issues with the code, I continued updating it and maintaining it to support new products and features for nearly 8 years. The issues with my earlier novice programming skills started creating more and more problems however as I didn't develop it to be modular initially and every change I made to the UI to support the new features I was adding was piling up the problems I had to fix each time. I decided to begin learning NodeJS as I knew by this time that PHP's limitations in presenting real-time data would eventually be an issue and that it wouldn't scale very well as-is.
I began the slow and arduous process of learning a language I'd dabbled in, but never been a huge fan of. Javascript, especially when it comes to programming non-blocking code for things that must happen in a very specific order, but where some parts must be run in parallel, can be an incredibly obtuse language to program with. Javascript starts great until you start using hundreds of modules and libraries and see how splintered the community has become. It's like modding with Skyrim. I spent more time testing mods than actually playing the game. Here, I spent more time learning and testing how to make all the modules work together smoothly than actually programming the software itself, frequently finding certain combinations didn't work properly as documented and requiring extensive troubleshooting.
I'd made steady, but very slow progress as I didn't have a lot of free time to learn the language and I was basically learning as I was developing. The final nail in the coffin was that the company decided a spreadsheet-based solution without any automation, inventory or user management was the way to go... They felt that the IT labs functionality being tied to a single developer's presence was a bad idea. To be fair, I'd been trying to hand over the project to others and act as more of a project manager for awhile, but no one stuck with it long enough to even submit a single commit. The move to the new solution was a complete and utter disaster. Hardware and pieces parts gets swapped, moved, used, etc and as time goes on, the data just gets more and more stale as people complain they can't find what they need. I was happy to just be done with it as it was mandated that all IT were supposed to move away from my application, but I've found myself in an odd scenario as people are going behind IT's back to run my software and I'm having to continue providing support for it despite my desire to escape it since I have no interest in developing an application that will eventually just get hard-blocked by IT.