Slashdot Mirror
Learning and Maintaining a Large Inherited Codebase?
An anonymous reader writes "A couple of times in my career, I've inherited a fairly large (30-40 thousand lines) collection of code. The original authors knew it because they wrote it; I didn't, and I don't. I spend a huge amount of time finding the right place to make a change, far more than I do changing anything. How would you learn such a big hunk of code? And how discouraged should I be that I can't seem to 'get' this code as well as the original developers?"
So you have been handed the steamin' pile o' code, it is great that you are very cautious and deliberate when modifying it. Make a set of regression tests, that is, make a set of test data and procedures and expected results to ensure original functionality that is still desirable is still working and no other errors introduced. It is hard, much more tedious than just creating new code with few constraints.
Doxygen is your friend. run it over the source code and keep the HTML handy for searches and cross references.
First of all, 30-40,000 lines of code is not lots of code. Try, 250,000 of code.
To start, use a good programming editor/environment (Xcode, Vslick, Visual Studio, etc.) that gives you the ability to easily go to definition or references to variables, functions, structs and such. Run some sort of profiler or flowchart type program on it to get a high level view of the code and how it fits together. If you can get the person(s) who worked on it before you to give you an idea of it fits together.
Fight Spammers!
I find that if the other programmer wrote it in such a way where it's too complex for me to follow, I'm not the one who's a moron.
I didn't get this one until I switched to my alter ego, the assembler programmer.
I'll echo some earlier comments.
Set up an execution environment with debugger, and run several typical scenarios and trace them with debugger. Get the feel of the big-picture execution scenarios/paths.
It will take time for your brains to get comfortable with it, though. And the details, when you look into them, will throw odd stuff at you. But that's the nature of our work.
Fuck systemd. Fuck Redhat. Fuck Soylent, too. Wait, scratch the last one.
For culinary folks...
The time and money you spend tracing and inserting noodles in the spaghetti will end up being larger than the time it takes to cook a new batch (no pun intended).
For auto folks...
The time and money you spend bondo-ing, welding, rewiring, duct-taping, and C'n'Cing parts for the car will end up being larger than the time it takes to design and build a new car. (Although restoring an old/vintage car for the sake of nostalgia is a much more pleasing experience than buying a new one).
Gain an understanding of the purpose of each pivotal region. Know what your desired result should be, then begin the rewriting endeavor.
'We are trying to prove ourselves wrong as quickly as possible, because only in that way can we find progress.' RPF
Ha, ha! Just 4 months ago I joined a project with a code base of about 500k lines. I would call that (the 500k lines one) intermediate in size. There are code bases with many millions of lines. I now feel pretty comfortable finding things in it. And I mostly use find and grep.
When 1person suffers from a delusion,it is called insanity.When many people suffer from a delusion,it is called religion
I have inherited projects and do my best to convince management that a pause is needed to document the code. Personally I try to flowchart the functionality and cover a couple of office walls with Visio printouts. Later on I can use such work to add detail and further documentation.
I inherited some code where the developer used names of girlfriends in variable names, it was just dumb and completely unprofessional. I didn't worry so much about keeping track of those, I was more worried about a change in one spot having unintended (and perhaps unknown until too late) consequences. Rather than spend time fixing problems, I thought it best to do some up-front documenting to at least provide a path to successful maintenance.
When I left the project, the manager had a binder of documentation and almost cried.
I had an English professor who always said, "Structure is the key to understanding." He was talking about literature, but I think the same is true for programs as well.
Try to understand the structure of the program. What is the basic flow? It should have an initialization routine, a main loop, and a shutdown routine. Find out roughly where they are, then focus on the main loop. Usually there will be one piece of code that is central, and it will occasionally pass control into other large pieces of the program. Sometimes there will be more than one main loop, and control switches back and forth between the various main loops. If the program is event drive, this will make a difference in the structure.
If you are just trying to make a small change, try to find the sequence of events that will lead up to where that change needs to be made. Follow the sequence of execution until you get to the line you need to change. If you are changing a single variable, sometimes it's helpful to do a search and find all the places that variable is used, to make sure your change won't have any side effects. This may seem time consuming, but it can save 10 times more in debugging.
Learn to follow code execution with your eyes, without running a debugger. One thing that separates good coders from not so good coders is the ability to follow code that isn't being executed.
Qxe4
I currently maintain several million lines of perl. It's not hard, it mostly just works, and when it doesn't, it's not that hard to figure out where it's broken IFF there is a consistent repro case for the problem.
If you have a proper development/production divide, there shouldn't be any weird production issues unless you or your predecessor missed some test cases. If you don't have test cases, that's a problem, if you don't have a properly firewalled and complete development environment, that's a problem, the code itself? Shouldn't be a problem.
Realities just a bunch of bits.
Everyone, including me, always wants to go for the clean rewrite. But in my experience it almost never turns out for the best. There's a reason for all that messy code. Much of it was bug fixes that real-world users needed. Other complexities were needed in the first place to make the user experience simple (natural, giving it that "hey, it's just works like I expected" feeling).
The reason you don't understand the code is that you weren't part of the original design discussions, in which weeks or months were spent learning, debating, arguing, etc., about many different design decisions at many different levels of abstraction. You don't know why the trade-offs were made. You just see the finished product.
Rewriting the code won't give you insight into any of this. Learning the code the hard way, fixing bugs, rewriting *small* pieces and seeing what breaks the regression tests, etc. will eventually help you to understand it.
There is no point in rewriting it before you fully understand it. Attempting that can kill a product. Conversely, by the time you fully understand it, there won't be any need to rewrite it, because you'll own the code.
What do you think about intermediate variables that are not strictly necessary?
Use them if they make things clearer for someone reading the code, otherwise don't. For example, you can write:
screen.displayName = user.firstName + user.lastName;
or you can write
String fullName = user.FirstName + user.lastName;
screen.displayName = fullName;
Thus making it more clear to someone reading that you are trying to use the full name. That is probably not the best example because anyone would probably understand that user.firstName + user.lastName is the full name, but I think you can see the main point, that sometimes it can be easier to read a few meaningfully named intermediate variables than a long equation. If it isn't easier to read, don't do it. But really when I read code, or even write it, I am willing to conform to either way of doing it if someone else feels strongly about it, because that is far less important than things like flexibility of major structures in the code.
Qxe4
Mine's 12 inches.
I call that a module. Large is anything over 1,000,000 LOC. Step up.
well that depends on how many developers we are talking about. The original question seems to indicate that the author has inherited the codebase. The need for this question wouldn't exist if the person were on some large team.
For one or two or five people, 40K lines is a sizable codebase, especially if it has been poorly maintained / designed.
blah blah blah
I am currently working with a mission-critical codebase, which is written in PHP and has absolutely no cohesive design to it. Well, unless you consider making everything static and unnecessarily inheriting other classes and overwriting static variables willy-nilly a cohesive design. There are business rules just everywhere and API requests everywhere and all kinds of calls that overwrite static variables. If you don't methodically trace logic it's really easy to get lost. What makes it worse is that there are many many variables that are named very similarly and you don't really know which one is right and which one is just going to get overwritten in some method call you are not looking at right now. And if this software fails, the worst case scenario is that my company makes no money. It really has made my life over the last few weeks pretty horrid. Fortunately I enjoy the job and the co-workers and am well respected there. Otherwise, it wouldn't be worth the aggravation.
My advice: communicate your difficulties to everyone who will listen (refrain from complaining or bellyaching, just communicate). If you inherit something like this, and it is mission critical, then you need to take as long as it takes to get it right. That's right, AS LONG as it takes. Take the time to document everything. Bother the crap out of anyone who can help you. You are responsible for doing your job, and part of doing your job is figuring out how to maintain this beast. And in order to do that, you need to use every resource at your disposal. If anyone wants to rush you along, you need to communicate the difficulty and the importance of the task. If you have been working at a place for a while and have done a good job to date, then they should trust you. If you're brand new, then you'd better hope someone there values your opinion and doesn't merely think you are incompetent. If you are asked to make enhancements, don't refactor until you understand the code. So make enhancements, leaving the potentially crappy code in place, even copying it if necessary. Steadfastly resist the temptation to refactor until you understand the entire piece that you are trying ti refactor. Don't remove seemingly unnecessary variables, and don't reduce seemingly redundant database calls. That comes later when you actually know what you are doing in there. IOW, if you have to navigate a lion's den by touch, don't stop to groom the sleeping lion (unless of course, that is your given task.)
The word inherit seems to imply that either the original maintainer no longer works there or has moved on to a different position. This means that it's you on the hook to figure it out. You've gotta dig in, buckle down, and get to it.
blah blah blah
That *totally* depends on the code base and the way the OP thinks. Sometimes they're a complete waste of time. Others...not so much.
I've worked with plenty of programmers who see pretty much every software problem in terms of FSMs. One size does not fit all.
It somewhat depends on the language used - some languages are easier to penetrate than others. And some languages does more in 10 lines than other languages do in 100.
But anyway - to learn the code you may have to find a starting point (there is usually at least one logical point to start) and then make a flowchart in PowerPoint or something for the general structure. It's no point trying to get into the finer details, just a general sense of flow. You will get things wrong in the beginning, but don't worry. And you may end up finding a lot of dead code too.
When you have a satisfactory overview of the code it's time to really swim and drink the code. Many programmers have a tendency to accept that "it works" and stop there. By throwing the code into the compiler at maximum warning level and then try to fix all warnings you will be even more involved. And if you aren't satisfied you can take on the code with code analysis tools like Splint (for C) or FindBugs (for Java).
And don't forget that the commands "find" and "grep" in *NIX are your friends. Other environments usually have other tools, and IDE:s have their own, so you don't have to install Cygwin or something to get a grip on things.
And if you think that you don't understand the code well enough - try to port it to another operating system or other language.
Of course - this takes a lot of time and consumption of your favorite hacking beverage.
And yes - I'm involved as a single developer in a system with about 400k lines of code written in Java, and it was ported from an older system written in C, C++, Basic, Java, DCL...
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
There is no point in rewriting it before you fully understand it.
I fully support this statement.
I recently worked with a guy new to contracting. He came onboard to a project that had a lot of problems. He argued for re-writing it thinking he could do it quickly and simply; I didn't dispute that the system could use significant changes, and I asked him to read through and understand the existing code.
He never did.
Consequently I suggested to senior managers that he should be let go. Reading other people's code, particularly undocumented code, is painful - even for experienced coders. But it is necessary and failure to do so before recommending changes is unprofessional, dangerous, and lazy.