Organizing Source Code, Regardless of Language?

og_sh0x queries: "I'm looking for a source of information dedicated to organizing source code. I see a lot of books and other resources covering syntax and various syntax-related philosophies, but I can never seem to find a good resource for organizing source code in general. For instance, at what point do you split that massive source file into multiple files? At what point do two functions approaching similar functionality need to be merged, despite the cost of digging through the source and making changes to call the new function? These are problems that plague many programming languages. Are there such resources that cover these issues?"

Comments

This

[psavo]

is called 'Experience' part of your CV.

I've yet to find a simple way to determine any of those. It's just that feeling when you get while looking at the code 'damn, not again..'.

The wrong questions

[p3d0]

These sound like the wrong questions to me. It reminds me of someone's (perhaps Dijkstra's?) story of the response he received when he recommended abolishing gotos. Someone said "ok, I'll buy that; so what do I do if I'm at this point in the program, and I want to get to that point?"

The trouble with such a question is that it has no answer. Dijkstra's argument was not that one should take existing programs and remove the gotos; rather, that programs written using only structured elements (sequencing, conditionals, loops) are more comprehensible, and don't require any gotos because there is a more elegant way to achieve the same effect. Thus, as you can see, there really is no answer to the question; the questionner's approach was fundamentally flawed.

Likewise, software organization is not done in terms of functions; rather, it is done in terms of information-hiding modules. To ask when one huge function should be split into to, or when two similar functions should be merged, indicates to me that the design might be flawed. Sometimes that's unavoidable; for instance, if you are involved in a project written by someone else. In that case, you do indeed need to make this kind of decision.

However, true modular programming does not mean taking huge lumbering hunks of code and splitting them into modules. It means writing modules using the principles of information hiding to avoid making huge lumbering hunks of code in the first place.

This, of course, is easier said than done. It's not that hard to avoid gotos, because the use of Dijkstra's structured programming techniques makes them unnecessary. In contrast, writing good modules is hard, and without superhuman foresight, some modules are bound to be pretty crummy. These will need to be rewritten in order to achieve good information hiding properties.

So, there's your answer: don't put the cart before the horse. Don't expect that someone will tell you that you need to split a function when it gets beyond X number of lines. Rather, look at the integrity of the system's modules. If I can leave you with one piece of advice, I hope it is this: design module interfaces not according to what services they provide, but what information they hide. Modules for which you can't find a succinct statement (12 words or less, with no ifs, ands, or ors) of what information they hide are poorly designed, and need an overhaul. A symptom of this may be that your functions are redundant, or too long, but the core problem is one of poor module design.

Re:The wrong questions

[elflord]

So, there's your answer: don't put the cart before the horse. Don't expect that someone will tell you that you need to split a function when it gets beyond X number of lines. Rather, look at the integrity of the system's modules. If I can leave you with one piece of advice, I hope it is this: design module interfaces not according to what services they provide, but what information they hide.

Actually, the questions he is asking are indeed very important. It's all well to say that code "should be well designed", and indeed, most books spend a lot of time talking about design principles for people with clean slates. Unfortunately, very few people have a clean slate to work with. Using a good design up front is not an option if you're not the one who did the upfront design. We are either stuck with maintaning poorly designed code, or even code that was designed well up-front, but needs a change in design to meet changing requirements. What a book like refactoring brings to the table is the process of incremental redesign. Redesigning code without rewriting it is a fine art, and refactoring basically explains how to do it.

Re:Always!

[sohp]

Having a thousand 10 line files does nothing to improve maintainability.

My obligatory plug for The Mozilla Project. Not quite one function per source file, but definitely lots of very small source files, each implementing a very narrow slice of functionality. Mozilla is pretty well factored code, and maintainability is enhanced by the separation of responsibilities. It makes it possible to enhance or fix problems in one area, say the in nsFTPChannel, and know that all the thousands of other lines in the program will be largely insulated from those changes.

Yes, it does take a while to get familiar with the entire Mozilla codebase. The flip side is that you only have to look at and understand a small fraction of it to start becoming productive.

If you are using C++, Large Scale C++ Software Design is definitely a recommendation I can second.