Organizing Source Code, Regardless of Language?

og_sh0x queries: "I'm looking for a source of information dedicated to organizing source code. I see a lot of books and other resources covering syntax and various syntax-related philosophies, but I can never seem to find a good resource for organizing source code in general. For instance, at what point do you split that massive source file into multiple files? At what point do two functions approaching similar functionality need to be merged, despite the cost of digging through the source and making changes to call the new function? These are problems that plague many programming languages. Are there such resources that cover these issues?"

Comments

Read Stroustrup

[ObviousGuy]

Programming C++'s first couple of chapters discuss this very topic.

refactoring is what you're after

[Anonymous Coward]

www.refactoring.com - or any other good refactoring books should help loads to get you started. but there's nothing like experience :)

Refactoring

[fingal]

I would strongly recommend that one reads "Refactoring" (Martin Fowler - Addison-Wesley - 0-201-48567-2) (after reading Design Patterns) for the solid techniques that it introduces for clearly defined manipulations that change the shape of code without changing the functionality. However, even with this it doesn't completely resolve the issue of "when" that you raised, as summed up in the introduction to chapter 3 "Bad Smells in Code":-

By now you have a good idea of how refactoring works. But just because you know how doesn't mean you know when. Deciding when to start refactoring, and when to stop, is just as important to refactoring as knowing how to operate the mechanics of a refactoring.

Now comes the dilemma. It is easy to explain to you how to delete an instance variable or create a hierarchy. These are simple matters. Trying to explain when you should do these things is not so cut-and-dried. Rather than appealing to some vague notion of programming aesthetics (which frankly is what we consultants usually do), I wanted something a bit more solid.
I was mulling over this tricky issue when I visited Kent Beck in Zurich. Perhaps he was under the influence of the odors of his newborn daughter at the time, but he had come up with the notion describing the "when" of refactoring in terms of smells. "Smells" you say, " and that is supposed to be better than vague aesthetics?" Well, yes. We look at lots of code, written for projects that span the gamut from wildly successful to nearly dead. In doing so, we have learned to look for certain structures in the code that suggest (sometimes they scream for) the posibility of refactoring.

One thing we won't try to do here is give you precise criteria for when a refactoring is overdue. In our experience no set of metrics rivals informed human intuition. What we will do is give you indications that ther is trouble that can be solved by a refactoring. You will have to develop your own sense of how many instance variables are too many instance variables and how many lines of code in a method are too many lines.

The book then goes on to describe the various types of "abstract smells" and what sort of correctional techniques can be considered to correct them, for example:-

Iappropriate Intimacy
Sometimes classes become far too intimate and spend too much time delving in each others' private parts. We may not be prudes when it comes to people, but we think our classes should follow strict, puritan rules.

Overintimate classes need to be broken up as lovers were in ancient days. Use Move Method and Move Field to seperate the pieces to reduce the intimacy. See whether you can arrange a Change Bidirectional Association to Unidirectional. If the classes do have common interests, use Extract Class to put the commonality in a safe place and make honest classes of them. Or use Hide Delegate to let another class act as a go-between.

Inheritance often can lead to overintimacy. Subclasses are always going to know more about their parents than their parents would like them to know. If it's time to leave home, apply Replace Inheritance with Delegation.

I have frequently found that just reading through this short (~15) collection of abstracted "smells" gives a very good way of supplementing the "experience" that you speak of and helping you to make decisions with the benefit of a) a bit of third party support in making these decisions and b) a clearly defined set of rules as to how to apply each of the refactorings including test cases to prove that the functionality has not been changed in the process and, more importantly, a clean roll-back procedure for those times when the olfactory senses get a little bit confused...

Re:Refactoring

[gaj]

I'll second that recommendation.

When I first saw "Refactoring", I said to myself: "Self, now I've seen everything". I thought it was yet another book enshrining process and procedure over good working code.

I was wrong.

This book is really good for those who haven't yet learned what Stroustrup refers to as "taste". Hell, I've been coding for many, many years and I certainly thought it was worth a read!

My only caution is the same as the one I give about GoF, UML, OOA/OOD/OOP, or any other codified programming "methods": Don't blindly follow them w/o taking your own experience into account.

Basically, the less time you've been coding, the more seriously you should take these concepts. Over time, and with many KLOC, you'll develop your own "taste"; your own sense of what works. This is not to say that learning new methods is useless to someone who's been coding for a long time; far from it. Just that, in most cases, a hacker will develop a pretty good idea of what works for them and what doesn't. The dirty little secret is, whether your like the language he "accreted" or not, Wall is right: TMTOWTDI. And knowing which way to use in any given circumstance can only come with experience. Reading books like "Refactoring" can help a lot until you get that experience, though!

Book recommendation

[Read The Fine Manual]

Have a look at this book by Steve McConnell: Code Complete: A Practical Handbook of Software Construction.

Yes, McConnell is a Microsoft guy, but this book is completely operating-system and programming-language agnostic (even though examples are in C, Fortran, and Pascal, IIRC). It is an excellent guide to software construction, covering every aspect from design, over coding practice, style issues, to project management. I highly recommend it.

This book answers in detail

[dant]

This question is the subject of Large Scale C++ Software Design by John Lakos.

Don't let the title fool you--although he uses C++ for his examples, the concepts he talks about (splitting code into components, why each component should be in its own file, levelization of components, etc.) make sense in any OO language.

I consider this book a must-read for anybody working on large programs.

Other recommendations...

[PinglePongle]

Programming Pearls by Jon Bentley - old as the hills by now (he talks about the location of data on tape....), but full of very good insights into writing "good code"TM.

You might also like "The pragmatic programmer" - Hunt and Thomas - which is another "meta-programming" book with a lot of ideas and insights you could actually sell to your pointy-headed boss.

The section on "zero-tolerance" coding is a great "why and when to refactor" argument. There's also a good section on how to design the units of which your software is composed, how to reduce the coupling between those units, and how to test em when (you think...) they're done.

Nev