Slashdot Mirror
What Are the Genuinely Useful Ideas In Programming?
Hugh Pickens DOT Com writes "Computer Scientist Daniel Lemire has had an interesting discussion going on at his site about the ideas in software that are universally recognized as useful. 'Let me put it this way: if you were to meet a master of software programming, what are you absolutely sure he will recommend to a kid who wants to become a programmer?' Lemire's list currently includes structured programming; Unix and its corresponding philosophy; database transactions; the 'relational database;' the graphical user interface; software testing; the most basic data structures (the heap, the hash table, and trees) and a handful of basic algorithms such as quicksort; public-key encryption and cryptographic hashing; high-level programming and typing; and version control. 'Maybe you feel that functional and object-oriented programming are essential. Maybe you think that I should include complexity analysis, JavaScript, XML, or garbage collection. One can have endless debates but I am trying to narrow it down to an uncontroversial list.' Inspired by Lemire, Philip Reames has come up with his own list of 'Things every practicing software engineer should aim to know.'"
are the most useful thing I learned in the last 5 years.
Forget about having to learn any specific language or environment. You should be able to pick up any language or environment on the job.
You need to learn how to plan, estimate how long that plan will take to complete, and finish it on time. Very few programmers I've worked with are any good at estimating how much time they will take to complete anything. The worst offenders take double the amount of time they say they will.
Forget about specific computer science trivia. You can look that all up, and it's all available in libraries with various licenses. When you're starting a new job, refresh yourself on how that problem is already being solved. If you need a refresher on a specific computer science concept, take some time and do so.
With this advice you won't burn out at age 25.
The most useful concept I've ever come across is the notion of a closure in Lisp. The entire operating state of a function is contained within that function. This, and the McCarthy lisp paper (1955!) where it is explained how a lisp interpreter could be created using only a handful of assembly instructions is well worth the read. It is from the fundamental concepts first pioneered in lisp that all object oriented programming paradigms spring; if you can understand and appreciate lisp, the notions of encapsulation, data hiding, abstraction, and privacy will become second nature to you.
Furthermore, if you actually put forth the time to learn lisp, two things will become immediately apparent:
In Stroustroup's "The C++ programming language", there are numerous examples of concise, elegant code. These spring from the concept of deferring the details until they can be deferred no more - the top-down approach results in code which is easily understood, elegant, efficient, robust, and maintainable.
Many years ago, a poster commented that the work necessary to complete a particular project was the equivalent of writing a compiler; he was trying to emphasize just how broken and unmaintainable the code was. The irony in his statement is that most professional projects are far more complex than a compiler needs to be; because he didn't understand how they worked, he thought of them as necessarily complex. However, the operation of a compiler is actually quite simple to someone who understands how they work; the McCarthy paper shows how high level code constructs can be easily broken down into lower-level machine language instructions, and Knuth implements a MIX interpreter in a few pages in the "The Art of Computer Programming." Neither building a compiler nor an interpreter are monumental undertakings if you understand the principles of parsing and code structure. i.e., what does it mean if something is an operator, versus, say, an identifier.
Ideas are powerful; the details, temporarily useful. Learn the ideas.
The society for a thought-free internet welcomes you.
I think he was missing input validation from his list. The idea that you can never trust user input and you must validate it. The idea that you should white list what you want instead of black list the things you don't want. Ideas that consider the security of the system and not just the working condition of it.
Im a gamer, not a grammer major. This post is full of spelling and grammer mistakes.
I would say that one of the most important thing in programming is to break down a problem into parts that are useful and easy to manage. It doesn't matter which language you code in. It's very much like building with Lego - you have more use for all those 4x2 bricks than any other brick. The humongous large bricks are "use once". A right sized brick can be copied and pasted into future code as well, possibly tweaked a bit to suit the new environment. In the process of breaking down a problem - define interfaces. Make a design of the important interfaces to make sure that they can remain stable over time. That can make maintenance easier.
The second most important thing is to learn what compiler warnings means and how to fix them. In this case strong typing isn't your enemy - it's your friend since it will tell you about problems even before you get them when executing the code.
Third is to learn about which known algorithms that are out in the wild so you don't have to invent them yourself. Quicksort is already implemented a thousand times, so there's no need to implement it again, just find which library you need. If you are developing a commercial app you shall start with the Apache project since that license is permissive when it comes to how the libraries may be incorporated. The LGPL is also good. But leave the license headaches to someone else to sort out if you aren't sure.
These are the useful ideas I try to follow, the rest is just a mixture of ingredients and seasoning to get something running.
Remember: You can build a great machine with simple stuff or a stupid machine with expensive stuff.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
But never make the prototype too good.
"You need 12 weeks to turn it into actual software? this works fine!"
Surely any programmer ought to know the underlying principles that make databases work (ie ACID etc) even if they never intend to go anywhere near multi threading. Even in single threaded programs knowing what and how ACID works can help. Have you never done a write() and wondered where the data you sent to disk went?
Perhaps the relational calculus might not be strictly necessary, however if knowing the theory behind relations helps engineers from naively treating databases as data garbage dumps, it'd be worth it.
Plan My Week for iPhone
When it was part of our CS required curriculum, I suspected I would never use it, but it turns out that the vast majority of projects I've been involved with have used databases in some way, and one of them even involved some pretty serious database query optimization. As far as I can tell, unless you pretty much code exclusively down at the kernel level, you're going to eventually be asked to work on some project involving databases. They're the glue that holds technology together. Outside of a handful of niche fields, I'd be surprised if any programmer managed to go more than five years out of school without having to work with one.
Also, once you understand databases conceptually, everything starts to look like a special case of a database. This is a good thing. C data structures? Table records. Pointers? Relations. And so on. It ends up helping you understand complex problems even if you're one of those rare people who never ends up touching an actual database.
Check out my sci-fi/humor trilogy at PatriotsBooks.
IMO, a function should be as long as it needs to be, and no shorter or longer. If the most easily understood way to express a concept is as a 5,000 line function, then you should write a 5,000 line function. Splitting up a function based on some arbitrary length limitation can only lead to less readable code.
For example, my record is almost 5,500 lines. The entire function is basically a giant switch statement in a parser (post-tokenization). The only way you could make that function significantly shorter would be to shove each case into a function, and all that would do is make it harder to follow the program flow through the function for no good reason. At any given moment, you're still going to be staring at exactly one of those cases per token (plus a little code on either end), so having each case in a separate function just adds execution overhead without improving readability, and it makes debugging harder because you now have to pass piles of flags around everywhere instead of just using a local variable.
One of the data structures for the function in question is almost 1200 lines long by itself (including anywhere from two to fifteen lines of explanation per field, because I wanted to make sure this code is actually maintainable). By itself, the initializer for that data structure cannot meet your "fits on one screen" rule, even with most of the fields auto-initialized to empty. And there's no good way to shrink that data structure. It is a state object for a very complex state machine. The code to interpret the end state is over a thousand lines of code by itself.
In short, those sorts of rules simply don't make sense once the problem you're trying to solve becomes sufficiently complex. They're useful as guidelines for people who don't know how to write code yet—to help them avoid making obscenely complex functions when the functionality is reasonably separable into smaller, interestingly reusable components, to keep themselves from shooting themselves in the foot by repeating code where they should call a shared function, and so on. However, IMO, if you're still thinking about rules by a few years out of school, they're probably doing you more harm than good, causing you to write code with unnecessary layers of abstraction for the sake of abstraction.
Check out my sci-fi/humor trilogy at PatriotsBooks.
I'd be interested to know what line of work you do, programming wise. My experience tells me that a lot of programming that is being done is meant to be powerful and meant to be built quickly. Running quickly and with low tolerance for faults is a little less important because very few things are mission critical. While anathema to the academic, it demands a certain skill set, which is the ability to very quickly assimilate new arbitrary knowledge about libraries, software, and code, that the programmer hasn't seen before. The result is a fragile sort of knowledge that often lacks formality and granularity but is sufficient enough to accomplish a task very quickly.
This skill is not exclusive with the ability to write everything from scratch, but understanding a system through and through seems to often undermine the coder's speed at getting the product out because they want to do it 'correctly'. This specific tug of war, between programmatic idealism and pure ease of use often ends up being a major concern in the work I do because I am rarely satisfied with the 'easy' solution. Sure, sometimes doing it 'correctly' amortizes quickly and thus the up front cost should be paid, but other times it really just badly trades one resource (dev time) for another (execution time, time of the humans managing the software, time to market).
I've met coders who talk like you, and when they do, they're so impractical in their overarching decisions that often the software is DOA because it doesn't do enough, or it took too long to write, or it was endlessly being refactored.
You're probably doing work at a more basic systems level than I am. And, maybe that is why you have this philosophy. I used to think work at the level I'm doing now was uninteresting, but the problems are shifted into ones of integration, forward planning, and multi-discipline and away from algorithm, pure transaction, etc. There are also a buttload of programmers working at this level, and while I would love to say that we must hire only candidates who can write a minheap backed up by a btree... I think by far the more important qualification is that skill I was talking about, which is a lack of 'not invented here' syndrome and an ability to read other people's code / technical documentation very quickly and see how their architecture can fit into our needs with the minimal of fuss.
To put it another way, most of the low level problems have been solved adequately enough for this level of work, but there are a whole slew of emergent problems that require more than just Pattern A + Pattern B to solve, and are just not that helped by understanding the machine instructions or by algorithmic knowledge. I believe that to be true for probably the many of the programmers that are gainfully employed. So, I could not agree with the notion that your hiring strategy would work well for most positions.