Slashdot Mirror
Lessons From Six Software Rewrite Stories (medium.com)
A new take on the age-old question: Should you rewrite your application from scratch, or is that "the single worst strategic mistake that any software company can make"? Turns out there are more than two options for dealing with a mature codebase. Herb Caudill: Almost two decades ago, Joel Spolsky excoriated Netscape for rewriting their codebase in his landmark essay Things You Should Never Do . He concluded that a functioning application should never, ever be rewritten from the ground up. His argument turned on two points: The crufty-looking parts of the application's codebase often embed hard-earned knowledge about corner cases and weird bugs. A rewrite is a lengthy undertaking that keeps you from improving on your existing product, during which time the competition is gaining on you.
For many, Joel's conclusion became an article of faith; I know it had a big effect on my thinking at the time. In the following years, I read a few contrarian takes arguing that, under certain circumstances, it made a lot of sense to rewrite from scratch. For example: Sometimes the legacy codebase really is messed up beyond repair, such that even simple changes require a cascade of changes to other parts of the code. The original technology choices might be preventing you from making necessary improvements. Or, the original technology might be obsolete, making it hard (or expensive) to recruit quality developers.
The correct answer, of course, is that it depends a lot on the circumstances. Yes, sometimes it makes more sense to gradually refactor your legacy code. And yes, sometimes it makes sense to throw it all out and start over. But those aren't the only choices. Let's take a quick look at six stories, and see what lessons we can draw.
For many, Joel's conclusion became an article of faith; I know it had a big effect on my thinking at the time. In the following years, I read a few contrarian takes arguing that, under certain circumstances, it made a lot of sense to rewrite from scratch. For example: Sometimes the legacy codebase really is messed up beyond repair, such that even simple changes require a cascade of changes to other parts of the code. The original technology choices might be preventing you from making necessary improvements. Or, the original technology might be obsolete, making it hard (or expensive) to recruit quality developers.
The correct answer, of course, is that it depends a lot on the circumstances. Yes, sometimes it makes more sense to gradually refactor your legacy code. And yes, sometimes it makes sense to throw it all out and start over. But those aren't the only choices. Let's take a quick look at six stories, and see what lessons we can draw.
Sometimes you have to completely rewrite everything from the beginning.
Other times, changing a few words here and there, or a paragraph or two is all that is needed.
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
Nowhere in this article is any flaw in Joel's pronouncement to never rewrite found. The only of the six stories that was a rewrite of an existing product is Netscape and that was a disaster. The fact that Firefox came out years later doesn't make it any less of a disaster.
In the rest of the cases, no one is re-writing their existing product and letting the competition catch up. No, what they are doing it making a new product and moving users over if possible. That makes sense and that is what I learned from Joel's original article.
Making this seem like some kind of intriguing rethink of rewriting legacy code is false and click-baity.
The articles own facts don't support the article's conclusions.
"Netscape’s slide into irrelevance wasn’t entirely due to the rewrite—a court agreed that Microsoft had deliberately abused their monopoly. But the rewrite was certainly a contributing factor"
The graf accompanying this section shows Netscape's market share dropping from about 80% to 50% BEFORE the rewrite. Now that drop continues from 50% to near 0% during and after the rewrite, so the rewrite certainly did not save Netscape. But the slope of the decline barely changes pre- and post-rewrite. Basically, unless there's other evidence not presented, the best conclusion is the rewrite had no effect.
Also, "what finally ended the IE6 era wasn’t Firefox but Google Chrome."
Except your own graf shows IE market share dropping starting in late 2002 in mirror to the rising Mozilla/Firefox. Chrome doesn't even show as a factor until 2008. The articles own facts don't support the article's conclusions.
What really killed Netscape was releases a lousy product. 4.0 suuuuucked. (Folks on the web in '96/97 remember.) And IE at the time was releasing it's first good version, a better version. Fact is, at that time IE was better than Netscape.
What Netscape needed was a better product, and if it took a ground-up rewrite to get that better product, then a rewrite was necessary.
What we know now is the rewrite did not save Netscape. But we'll never know if there was some other course of action that could have saved the it. What we really should be doing is examining what was going on in '95/'96 to produce such a bad product and lose that market share in the first place, not at what rearrangements of the deck chairs was done as the ship went down.
If the person recommending the rewrite is the person who wrote the original code, they are usually right.
If the person recommending the rewrite understands the code and has years of experience with it, they are often right.
If the person recommending the rewrite is the new guy fresh out of uni, they are almost certainly wrong.
Disclaimer: I was once that new guy.
I disagree with your disagreement. Rewriting a working application is juggling with a loaded gun. You say "if you carefully learn from past mistakes" but this is almost impossible in any organization I've ever seen. All those snippets and dirty patches are not documented properly. If there are comments, they may (or may not) be out of date. Maybe there is a reason they chose a weird data structure but that guy retired 15 years ago. So you do something "proper" and find it is 10X slower. Oops.
If I've learned anything, I've learned you gotta let sleeping dogs lie. More patients are killed by doctors in this business than anything.
Sometime you need to disconnect your software from your ego.
A lot of full rewrites are primarily due to the fact the developer wants to be the Hero, toss out the old broken code, and make a new superior on in their own image. (Which then over times becomes old and broken)
Also trying to keep bad code on life support, because you had invested so much into it over your time there, is just as bad, as the application barely meets the business needs anymore, and every fix is getting increasingly more complex.
Sometimes what the code really needs, is just some reorganization. (Move your function and methods in more logical locations in your code so you visualize the problems more easier. Beautify your code, make white spaces consistent. ) Create a script to see where each function is called and where, Delete unused functions. Identify duplicate logic and merge them, or make a parametrized function of of them. It is actually amazing how a little cleanup work can turn a mess into a manageable application for decades.
Also if designing software really try to split Logic and Interface/System call into separate portions. The routine that checks the required field, should be straight logic. Not generating HTML or Form boxes, giving the errors. Despite what your bosses say, assume the platform it is running on will be retired in 5 years, and try to plan to port the application over.
Finally, really try to use basic core components as much as possible, avoid 3rd party tools as much as you can (this includes semi official repositories such as with pip or cargo). Don't be a Hero and make a ground breaking new interface, just use normal component.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
So you rewrite your code with a new sets of corner cases snippets, dirty patches that stick out from it.
The problem is old code while ugly, has been made to work over the decades, you are not going to learn from the past mistakes, because in the rewrite, you will undo years of patching which explains the changes in thought.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
You forgot:
If their title is Manager, or Consultant, they are ALWAYS wrong.
First law of people: People are generally stupid.
Two programming articles in a row? How in the world am I to satisfy my deep inner thirst for bleeding-edge news about global warming, basic income, the Model 3, and Ajit Pai?
Keep this up and I'm going to have to go back to HuffPost. Sad!
And rewriting XUL into Web Extensions, making the rewrite mistake twice gave Chrome the web. We could have had a strong Mozilla, now we got a weak one. Imagine a Firefox released in 2000, not 2004. We could have defeated IE earlier and avoid IE6 all together.
I hope in five years time, when Chrome has 90%+ market share Mozillians take a hard look at themselves and see that they were responsible for the chromageddon.
His article hinges upon the assumption that Netscape was screwed over by the rewrite. In reality, they were almost certainly screwed on the business side to the extent no amount of technical effort could overcome their position.
To the extent they had technical woes, it may well have been the case they couldn't sort out how to make the improvements they wanted to make given their current design.
Now there are valid points, that 'old code' may look crusty but there is a good chance it is crusty for a reason and that sort of thinking should be ever present while making such a call, to try to understand *why* it is crusty before throwing it out. However sometimes it is for bad reasons:
-Written against a once-presumed 'winner' of the market that becomes defunct. Your shockwave website has to be rewritten because the supporting technology is toast, and you better be scrapping your flash website soon if not already.
-Maybe the runtime is still around, but *your* ability to find willing developers is difficult, so you have to switch languages/runtimes to align with the labor market
-The people doing it didn't know what they were doing and did it incorrectly. Optimally, this is the same team that recognizes they painted themselves into a corner so they know what to do next time.
-The code is full of workarounds for third party libraries that no longer apply. True this doesn't scream 'rewrite', but one of his points was that the ugliness of code is due to fixes for things long forgotten that still matter, but it's frequently the case they do still matter.
In short, like all opinions be informed and influenced, but no simple answer is ever 100% correct no matter what. Internalize the points and evaluate in your scenario.
XML is like violence. If it doesn't solve the problem, use more.
The classic answer to that is 'job security' == 'unpromotable'
But that assumes you aren't going to change companies, on your schedule, and 'fuck them'.
The truth is 'job security' == 'not promotable without an employer change'. But that's just true for many employers in any case. If that's your boss, there is no reason not to play 'knowledge is power', just don't be slow blatant about it the PHB can see it.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
I do think refactoring is a must and needs to be part of the design cycle, just as much as getting the features that marketing already has sold to customers.
Technical debt is a thing, and getting that heap of hacks, kludges, workarounds, and other junk out of the codebase is just as critical as doing months long sprints to appease marketing. In fact, the Agile cycle in theory has room in between sprints for doing just this and fixing stuff. However, every place I have worked at who tried to do Agile had a "permanent sprint" in place, which not just burned people out, it made code bases which were all but impossible to maintain. Especially with management always threatening to outsource or offshore the devs, to the point where the devs knew they were going to get offshored, so they didn't care about anything except making their deliverables, even if it meant that they left gaping security holes, or just horri-bad code, because that mess would be cleaned up by someone else.
If a company actually values its code base, they will treat it just as they do any other machine... take time to take it down and service it, fix glaring problems, add actually useful code documentation (no comments about "don't touch this routine, we don't know how it works, neither will you.") Then, start seeing if you can do things like performance testing and code optimization, perhaps a pass for security checking (strncopy(), not strcopy(), etc.)
Technical debt is a thing...
Best place I ever worked knew that, and worked to quantify it. That helps a lot when you're trying to figure out whether or not to refactor. In a couple cases, it became apparent that a partial refactor was going to accomplish something like 90% of a full one, with 10% of the effort.
Quantifying your pain-points with even back-of-the-envelope calculations can really help you make good decisions about how to move forward.
Velociraptor = Distiraptor / Timeraptor