Slashdot Mirror

quick addition is there are programs which optimizeit for you and tell you what to cut out to reduce collection, increase performance.

Haven't had much luck finding a decent IDE for my environment (servlets->EJB->JDBC, etc, etc). What I have found that works well is a good text editor: gvim. Syntax highlighting, and my fingers know vi. CVS for source control and ant for build/distribution. Also have had good luck with optimizeit for profiling java apps. Profilier is especially nice because it lets me attach to an application server and filter out application server specific classes (which can be many). This allows me to really see what my beans are doing. Also fond of jprobe for debugging. Using many different apps for different aspects of development can be a pain in the ass, but I've found that after a little meddling around, it works quite well. Anyhow, just my 2 cents.

For quite some time before that I have been writing GoingWare's Bag of Programming Tricks, a collection of articles on the business and practice of programming. Of most interest in learning how to program well would be:

• Study Fundamentals Not Tools APIs or OSes
• Properly managing memory returned by transcode() in the Xerces library (while about an obscure problem, it illustrates more general principles in a useful way)
• Secrets of the Debug Meisters: MacsBug Tips and Tricks (MacsBug is a Mac OS assembly debugger)

• Pointers, References and Values - Passing Parameters, Returning Results and Storing Member Variables, with Musings on Good C++ Style

• Bounded Pointers for GCC
• Spotlight for Mac OS PowerPC
• BoundsChecker for Windows
• Purify for Windows and Unix (but I think it doesn't support Linux)

• OptimizeIt

Mike

I'd further like to say that I think everyone should spend less time concentrating on learning specific API's in great detail, and instead focus on improving your core skills and fundamental understanding of programming.

I used to spend a lot of time learning API's (bought every volume of Inside Macintosh as they came out, and when I was just starting out and broke and hungry, used to read them in the bookstore before I could afford to buy them).

I prided myself on knowing all the little bugs and intricacies of the MacOS so I could just know to code around an OS bug without having to research why my code didn't work. I got so good that I was hired as an OS engineer at Apple where I concentrated on debugging the MacOS system software with MacsBug (a machine level debugger) - I had the MacOS source code at my disposal but that usually didn't help when I was visiting a tester's cubicle to diagnose a machine with a hard-to-reproduce crash.

Then I moved to the BeOS, shipped a product and wrote a lot of code but got fed up with their lack of commitment to their developers. And without getting paid to write BeOS code, I never could keep up with the BeOS API's I wanted to work with, like the new Media Kit (which I do know enough about to say it is pretty cool).

A couple years back I stopped spending much time learning and mostly just cranked out routine code. I felt I didn't want to learn anymore because, as I would sometimes say:

I feel that if I have to learn another API my head will surely explode.

I read Scott Meyers' Effective C++ and More Effective C++ and as I read through each item I inspected my program top to bottom and applied the advice to it (thus fixing a lot of bugs).

I also bought Bjarne Stroustrup's The C++ Programming Language: Special Edition (I recommend the special edition to professional programmers).

I started reading the newsgroups comp.lang.c++, comp.lang.c++.moderated and comp.std.c++ and posting questions there - in one case I found a construction on the very edge of the C++ standard and as a result of a compiler bug managed to instantiate an object of an abstract base class - its pure virtual function had a nil pointer in the vtbl and my program would crash when this function was called. An engineer from the compiler vendor read my post on the newsgroup, agreed that it was a bug that his product would compile my code, and logged a bug.

I didn't used to use the Standard Template Library very much at all. I had read too many mailing list and newsgroup postings from people whose code wouldn't compile when they changed platforms.

But I figured that by now compilers must have matured enough I could reasonably start trying out the STL. I bought STL Tutorial and Reference Guide by Musser and Stepanov and actually only read a little bit of it before I realized that the STL is actually really easy to use (the API is very simple and uniform), so if you know only a very little bit you can go a very long way.

In part because of challenging myself I became overwhelmed with programming stuckness as discussed in Overcomming Programmer's Block? (sic) and I suppose grew a little bit by taking a week off without pay to rest, contemplate, study and take a broader view of architectural issues.

One thing that helped quite a bit was learning about Extreme Programming.

These things have all had direct payoff in my code, both in making my code quicker to write, easier to debug, easier to make my classes more reusable within the one program I've been writing the last few months, and I'm pretty sure more likely to make some of the code I've written reusable in most any program in the future.

It's also made it a lot more pleasant.

But don't listen to the headhunters - what they're looking for is "skill sets" and industry buzzwords (COM, COM+, DCOM, TCP/IP, Visual C++, ASP, SQL, device drivers, CORBA, Unix internals, Java, Perl, PHP, JSP) - I get recruiter calls looking for all kinds of acronyms, most of which I don't mention anywhere on my resume.

Even I advise listing every skill keyword you can legimately claim on your online resume in Market Yourself - Tips for High-Tech Consultants - but while listing skillsets may be a valuable jobhunting tool in your resume, acquiring them should not be your focus.

BTW, when someone asks me whether they should learn Java or C++, I usually advise beginners to learn Java as it's easier to get something working reliably without crashing, but emphasize they should learn both languages as well as at least one kind of assembly code. I stress that it's important to learn both C++ and Java well enough to understand the strengths and weaknesses of each (pop quiz - why does Optimizeit claim to remove memory leaks from Java programs, when Java is a garbage collected language?).

Most new programmers these days are most concerned with which language will make them the most money the fastest. I tell them that they won't go anywhere until they can pick up any new programming language as a matter of course and have at least a couple under their belt.

I've got bad news for you neophytes - friends, just knowing a programming language doesn't win you very much in the work world, you have to understand the concepts and how to apply them, and you have to know how to apply them in a production environment, working in a business under pressure, shipping working products and dealing with people who don't understand anything of substance about computers.

Your focus should be on acquiring skills that will be applicable to any program you write. You should just learn enough of a skill or tool to get the job done and then leave it. Take with you what can be applied anywhere.

BTW - learning the fundamentals and not getting too specialized enables you to develop for any platform, as I do - you can see this from my homepage. (Perhaps one reason why I prefer understanding the fundamentals is that my degree is in Physics, and I've always wanted to understand things at a fundamental level - Quantum Mechanics, Particle Physics, Cosmology and so on.)

One final word of advice - read The Forum on Risks to the Public in Computers and Related Systems. It's often entertaining and funny, occassionally tragic, usually insightful and will make you a more responsible programmer and a wiser computer user. You'll certainly learn to avoid using computers for anything of real importance and take extra caution to protect yourself when you choose to do so.

Optimizeit is a powerful profiling solution for Java developers looking to rapidly track down and fix performance issues such as memory leaks and performance bottlenecks in any Java programs. Optimizeit is essential for the detection of performance issues in application server environments. Optimizeit provides instantly integration with most popular application servers, its overhead limiting features ensure scalability and new offline profiling coming soon in 4.0 allows testing of applications in production environments
The Queue Principle

I've spent most of my career fixing somebody else's broken code.

This is not to say that my own has always been of the very highest quality, but in fact I decided early on to try to come to a fundamental understanding of what was wrong with software development, to get very good at debugging (I say that debugging is a specialty on my homepage) and to learn to write better code.

In my early years I was initially very shocked at what I'd discover in production use at companies. Over the years I just learned that that's standard practice, in commercial software, in-house software, and even in scientific software (where I have become convinced, because of my experiences with high-energy physics data analysis, that many scientific papers are published with erroneous measurements because of software bugs).

Early on I read that something like 90% of software development is spent doing maintenance programming. Some of this is doing legitimate upgrading, but a lot of it is just spent fixing bugs, and even a lot of time spent doing upgrades would be more productive if the code were of better quality.

After reading this figure and having so many experiences with software bugs, both other people's and my own, I decided very early on to get very good at debugging.

One of the first things I did was adopt the regular use of "lint" for checking my C code. I would integrate lint targets into my makefiles and after editing a source file I would type "make lint" before compiling to objects and lint would check all the files that were out of date with the object modules. Pretty quickly I got to where I could write code that was nearly always lint-clean - but the existing code I worked on would make lint scream with hundreds if not thousands of complaints, often serious things like variables being used before they are initialized.

One of the first solid clues I got about software quality came from Robert Ward's book "Debugging C" - now out of print, it predated the common use of source code debuggers and talked about how to write your own stack crawls and other tools.

Ward emphasized the use of the Scientific Method in debugging, and because I was trained in physics, this came very naturally to me; before that I'd mostly floundered and used printf a lot.

I've gotten very good at debugging and have even worked full-time as a debugger at Apple Computer where I was a "Debug Meister" and my business card gave my title as "Cybernetic Entomologist".

I can easily get highly paid consulting work doing debugging for companies desperate to ship a product (and have in the past) but I don't really like to do it for various reasons, some of the same reasons I quit my debugging job at Apple.

One is that if I only do debugging I don't have something to point to at the end of the day and say "I wrote that". I could say "that works because of me", but sadly there's usually lots of bugs left that I didn't have the time to find so I don't really feel proud of the result. The other problem is that the bugs are usually not there because of something interesting, it's not like the code is mostly good but there's some subtle flaws, rather the code is a heap of dung and I can go in with a pitchfork and do debugging wholesale until I get tired of it and the client or manager decides the rate new bugs are being found is low enough they can feel OK about shipping it.

I don't feel good about contributing to such shoddiness. If a company is not good enough to support quality in their corporate culture I don't want to come in and put on a band-aid for them. It would be an entirely different thing if a company hired me to restructure their development process so that quality was a goal that was achieved through direct application of process but gee whiz no one has ever asked me to do that for them.

I do have to say though that the best thing that ever happened to me is that I became a "technology prostitute" as the author of the original article puts it. One benefit of this is that the process is entirely of my own creation, and almost all of the work I've been given has been to write entirely new products from scratch, so I can engineer in quality from the beginning.

Here's a few recommendations I have. Get good tools. Besides a compiler, editor and debugger, you need a static code checker and you also need dynamic testers. The ones I know about are (I haven't used them all yet):

• PC-Lint static code checking for C and C++. It runs on Windows but Flexe-Lint comes as shrouded source code and is highly portable.
• Spotlight dynamic tester for Mac PowerPC - I use this every day and recommend it highly
• BoundsChecker dynamic tester for Windows
• Purify dynamic tester for Unix (but apparently not Linux) and Windows NT
• Optimizeit dynamic tester for Java - do you know many Java programs have memory leaks? Can you understand why? Not just Java but any garbage collected program.

Finally, to really come to understand the software quality problem in the industry and what you can do about it, read The Forum on Risks to the Public in Computers and Related Systems also available on the Usenet News as comp.risks. The book The Software Conspiracy exposes the complete disregard the commercial software industry has for serving the consumer by providing quality products - I haven't read it yet but it looks interesting.

A very interesting methodology that emphasizes personal responsibility and puts the fun back into programming as well as maintaining quality from the very start is Extreme Programming. I'm starting to adopt extreme programming (the the extent a one man operation can - I can't work in pairs :-/ ) and find it a tremendous benefit.

The Forum on Risks to the Public in Computers and Related Systems

The Software Conspiracy

While there will always be quality problems in software, current practice in many companies is to not even try to do the basic things that tend towards improving software quality. Until the public wakes up and realizes they're being ripped off, and their safety and corporate information being put at risk, we will always have this problem.

One solution is to get every programmer in a company a copy of some good quality tools, static analysis tools like PC-Lint and dynamic (runtime) analysis tools like Spotlight (for the MacOS) or BoundsChecker (for Windows) or Purify for Unix (but apparently not Linux) and NT.

As a Spotlight user and a long-time reader of the Risks forum, I wouldn't dream of shipping a Mac product unless it tested absolutely cleanly under Spotlight and had zero memory leaks.

But it is amazing to try Spotlight on a mature commercial product for the first time. Think you're program's free of bugs? Guess again. I proposed using Spotlight to my manager, on our program which had been shipping for several years and cost $600 retail. It was a serious product for high-end users. My manager said it would be a waste of time because "Our program has so many bugs, Spotlight would keep finding them and progress would be very slow." And you know, he was right. I persisted anyway, and spent three months ferreting bugs out of that program with Spotlight.

There's a lot of tools out there (and there's tools like these for Java too, like OptimizeIt - do you know many Java programs have memory leaks?). You don't have to pick the tools I recommend, but look out for what's available there and make sure you have something for every developer seat in the house.

It will be the best investment you make. The $199 for Spotlight will be paid for in the day it's first used.

And free software writers, I suggest writing free software versions of these. It would be possible in principle to write a special version of gcc, or an command-like option to it, that when your program is linked to a special library all your memory accesses are boundschecked. Note that Spotlight can validate memory reads as well as memory writes.