Slashdot Mirror

1. Blog your progress. Whatever you did today, blog it. Let people know what you did that worked, or what was faster (Nginx vs. Apache), or what wasn't (ColdFusion?). Don't reinvent the wheel, use WordPress, regardless of whether you like PHP/MySQL or not.

2. Use a subscription/payment management company. You're just a small group of nerds, not accounts receivable clerks. Fastspring, Plimus are free; Chargify, Subsify, Cheddar Getter, BrainTree, Spreedly charge; and Zuora is expensive.

3. Use Google Docs and Slideshare to share documents.

4. Chat. Don't just rely on email. Emails can often read like "this way or the highway". Be collaborative. You can often accomplish more with 15-30min collaboratively as opposed to composing and responding to long emails. Skype, Jabber, SIP

5. Take notes on what you did. Made a server configuration or a setting change in your CMS, your compiler, or whatever? Copy and paste from xterm so you don't have to guess about those commandline switches next time. Take screenshots and make them available to others. Zim, Projly, DokuWiki.

6. Have a phone numbers. If not bog-standard landline phones, take advantage of Google Voice and SkypeOut and SkypeIn (people can call your Skype line on a normal phone number). I realize Google Voice might not be available in South Africa yet.

7. Someone mentioned version control. Use git if you're a cool kid. Or svn if you're old and busted. Read the RedBean book. I've had success in having non-tech colleagues using graphical clients like TortoiseSVN (integrates into Windows Explorer).

8. Write tests. Any member of your team, sitting anyplace, should be able to push a button and run all your tests. Tests document how you're supposed to use a given method, class, etc., especially valuable when you're so far flung. Use JUnit, PHPUnit, FooUnit for your language. Write the tests before you develop, and you're doing Test Driven Development.

9. If you're writing tests, that implies loose coupling, which might require dependency injection. Can be difficult to climb that mountain, but it's worth it when you can just run a test and be sure your project works.

10. Development processes: Scrum, Extreme Programming. UML lets you communicate graphically about objects.

I don't see any reason why you can't get some programming experience while working on QA. You should at least be able to get read access to the code of the projects you are working on. When you find bugs during you QA work try tracking them down and/or writing unit tests for them. Depending on what languages are used there, you should probably get to know JUnit or NUnit or search for other testing frameworks that might be relevant to your project. Then create a patch with the test and attach that to the issue. It will make the developer's life easier and prevent the issue from occurring again later. Obviously, not all test can be easily automated, but you're bound to find a few along the way.

If you're able to create an automated test for the issue, the next step is to see if you can fix it yourself. If you sumbit a few unit tests with fixes, the development team will probably start thinking about hiring you if/when an opening comes up. If openings do not come up within a year or two, then at least you have gained some good experience that can be used during interviews, or to help out one of the many open source projects that desperately need better automated tests.

Good advice. Mod parent up. From a technology perspective, a Version Control System, Test Driven Development, and Continuous Integration can go a long way towards improving quality. If the OP is in a MSFT shop, then you are most probably stuck with VSS or TFS. VSS is file based so it is not very good for distributed development. You will need to enhance VSS with SoS if you have remote developers. TFS doesn't have that problem and also has support for TDD's unit testing. If the OP is willing to use OSS, then there are plenty of good options available. There is plenty of good advice here as to OSS VCS. There are various unit testing frameworks for Java, .NET, Ruby, PHP, C++, you name it. Also, check out Cruise Control for Continuous Integration.

Technology alone cannot solve quality issues, however. Changes in methodology, process, and even corporate culture may also be needed. Take a look at my site for more advice on that.

Very good points. Someone mod the parent post up.

The student matters more than the teacher. If the student doesn't do the work, the teacher can't help. If the teacher is incomprehensible, the student can still RTFM. Alas, I didn't get that point through my head as an undergrad.

The one thing I'd like to add: learn how to do testing. Learn how to write testable code. Perhaps start at the JUnit website? Add unit tests, for starters, to your homework projects when it makes sense to you to do that. See if it impresses your professors.

• JUnit
• NUnit
• PhpUnit (I think there are multiple versions)
• Php SimpleTest

what strategies should a developer take to insure that the resulting program is as crash-free as possible?

Simple, easy answer: unit test it as much as possible, then tinker with the design to make more unit tests posible.

Beyond that, do regular code reviews and get into the habit of programming defensively.

this applies to any language, not just C++. There's even a port of JUnit to C++ over here

• Build Tool - Ant (http://ant.apache.org/)
• Unit Test - JUnit (http://www.junit.org/)
• Application Framework - Spring (http://www.springframework.org/)
• Security Control - Acegi (http://acegisecurity.sourceforge.net/)
• UI Framework - JSF (MyFaces) (http://myfaces.apache.org/)
• J2EE/Web Server - JBoss/Tomcat (http://www.jboss.org/)
• OR Mapping - Hibernate (http://www.hibernate.org/

Using automated unit test frameworks such as JUnit is one of the most important engineering practices in the whole suite of agile practices. Many agile practices that don't explicitly ask for test-driven development still will insist that Agile is enabled by technology that allows for fluid, testable work. I'm very interested in Agile applied outside of software, and the underlying principles and practices of Agile. The situations where it can apply best have some sort of technological support for automation, and in particular automation of tests. I have seen situations where spending >$50,000.00 to build a custom automated test harness is money well spent in order to avoid mistakes in the first place. One of the main principles of Lean (a heavy source of influence for Agile), is that you can only go fast if you have exceptionally high quality. Any source of defects slows a project/team/production line down.

I strongly disagree with "C) Realize that this is going to take a long time." In my experience high quality and speed go hand-in-hand. Several times I have produces bug-free enterprise use code in complex environments (multi-threaded distributed messaging, high-volume data processing). The way to do fast and high quality: Test Driven Development. Do your tests before you write your code and you always know you have perfect code. You can go damn fast if you do that. Use both unit tests (e.g. JUnit) and acceptance tests (e.g. Fitnesse). Do continuous integration with something like Cruise Control. Don't settle for a single bug and you will go faster than you ever thought possible.

My previous project used JUnit very successfully, and had it integrated into their nightly build process. At the end of each day, all the developers checked in their modules and found out via the regression test reports the next morning, whether their changes caused someone else's module to blow up! :D

I only use assertions in unit test code such as here and here.

• C++
  • jEdit is a Java programmer's editor with excellent C++ support. I do development on Linux, Win32 and MacOS X, so it's very nice for me to have one editor I use on every platform. jEdit's not as featureful as, say, Emacs, but it's considerably more friendly to use.
  • Boost. If you're writing C++ and you're not using Boost, you're committing a crime against yourself.
  • Python. With Boost's Python library, it's easy to make your C++ applications scriptable. Write the heavy lifting parts in C++, then make those parts callable from Python. Do the rest of your development in a far safer, more sane language. You get almost all of the speed of C++, and far fewer headaches.
  • SWIG is another tool that's excellent for creating scriptable C++ applications.
  • Subversion for your version-control needs. Nothing else will do.
  • Doxygen for all your documentation needs. Learn it, love it. Your code's not done until every public part of the API has been doxygenated.
  • The GNU Autotools are really, really awful. They're also far better supported than Scons or pick-your-Autotools-replacement. Get ready to feel the pain of m4 macros. Sorry. :(
  • The GNU Compiler Collection started getting a good C++ compiler around version 3.0. I've been quite favorably impressed with 3.3, and I'm looking forward to 4.0. I don't recommend it for Windows, but for Solaris and x86 Linux it's beautiful.
  • I haven't found a good C++ unit testing framework yet. If you find one, please let me know.
• Java
  • Eclipse is an excellent Java IDE. jEdit also fits the bill nicely, if all you want is an editor. I use both frequently, and am quite pleased with both.
  • Subversion again for your version-control needs.
  • jUnit for unit tests. Your code's incomplete unless you've written unit tests for it.
  • Javadoc for documentation. I would recommend Doxygen, but it's quite possible you'll be deploying your applications on machines that don't have it installed.
  • Ant for all your build needs.

Windows is also in fact an extremely modular OS and your comments about modules being tightly coupled togeather is pure speclative BS (And completely wrong).

The reality is either one of us can only speculate all day about whether or not Windows is all that modular. Based on reports from both family and friends who have worked on Microsoft products, I would disagree with you. This is besides the point. My original post was written in the general case.

On the matter of tightly coupled systems being difficult to test, yes, that does make unit testing more difficult. For example, say I am building a Java web application. If I have code in the back-end which depends on objects provided by my servlet container, that means I have to have a servlet container available in order to test the component. I cannot simply stand it up in a completely isolated environment and test it. So, not only are there practical issues that make unit testing difficult, I am also trying to quantify the behavior of a much larger system (and one I specifically do not want to test).

The problem with unit testing (or ONLY doing unit testing) is that it does not test system interactions. A very large number of IE flaws that allow things like installing tool bars or other trojans were interactive flaws that involved multiple systems. Each system worked independantly as expected, but hook them up and the true behavior of the complex system become much harder to test.

Unit testing will find buffer overflows and such, but not worm holes through complex system interactions.

If I define the model by which pieces of my system interact, I should know how those pieces cannot (are forbidden to) interact. My unit tests, therefore, should include tests I expect to fail, not just tests I expect to succeed. I should test functionality in both the context in which it is permissible (verify it works) and in contexts where it should be hidden (verify my security model works). So yes, unit testing can be applied to test system interactions as well which can help reduce security holes. As for trojans and the like, I agree. Unit testing is not the end-all-be-all of security analysis, but it can take a fairly large chunk out of it.

In fact, modularizing actually makes the situation worse, since it presents many more opportunities for module combinations.

Your statement just about leaves me speechless. How can you tell me I am coming up with bullshit and yet write something like this? I am not going to go into depth about why modularity in software architecture is good (that is a topic for freshman) but I will tell you that your statement is flat out wrong. Try reading up on the subject some time, okay?

The real world is a hell of a lot more complex than you seem to be thinking it is having not done any of it.

And you base this statement on...? My point was that well designed software architectures can be broken down into very small pieces that are easy to unit test and that unit testing can be applied across an architecture. I find you've hardly done any work in refuting my claim.

And finally, never mind that Open Source does virtually no testing of any kind. FF is STILL full of invalid memory access bugs when http data is garbage that simple unit testing would have found...

I don't know how you manage to extrapolate a flaw of a Mozilla browser sub-project to encompass all open source development. This is somewhat ironic considering some of the big name unit test frameworks come from the open source community.

Several of the signatories at agilemanifesto.com use Java -- for example, Kent Beck (along with Erich Gamma) wrote JUnit.

So Java's must be pretty damn good for agile programming...

The big question was how to handle all the stuff that would normally be considered GUI code-specifically, the hit-testing: Where do users click, and what should happen when they click there?

I suspected that hit-testing would be the most complex part of the application, so I wanted to ensure that I had tests for it. It also didn't belong in the model, since that should contain only basic game concepts. I needed an intermediary class somewhere: The Swing code would get a mouse event and delegate it to this intermediary class. The intermediary would interpret the mouse coordinates and determine if something had to happen. It would possibly do this by interacting with the model. If anything did happen, an event would be broadcast back to the Swing class.

In MVP, the View actually serves as both the view and controller (presenting output and managing input). The Model is the Model, as in MVC. The extra middle layer is considered a bridge between the View and the Model. It's specific to the application and is often considered throwaway if the View needs to change (for example) from Swing to HTML. In this situation, the Model would remain unchanged.

Our application is just about all Java, written in a J2EE application container, and almost all of our tests are written in some form of Java as well. We use Test Driven Design to drive our API design, and end up with unit tests written with JUnit. We run these unit tests continuously, after every batch of checkins, using Cruise Control which performs an automated build.

To perform functional tests, we use JWebUnit which sits on top of HTTPUnit. The combination emulates a web browser, performing clicks and form sets against the actual server. We automate the running of these tests against an installed system, using Tinderbox, and we do this on every platform that we support (combinations of application servers, databases, and operating systems). The tinderboxes run 4 times a day, using the Cruise Control results.

It takes a lot of hardware (the tinderboxes for each platform) but the infrastructure is easy to set up with free software. The HttpUnit tests are tedious to write, mostly because they take a long time to run. Because of this, we don't consider a feature (user story in XP) complete until it has an automated HttpUnit test.

As for process, we have a small QA team that design test documents for each user story. We use those test documents to design the HttpUnit tests. Since we do this during the development stage, we involve the QA team on a daily basis.

When a user story is considered finished by the development team, the tests are reviewed by QA and the functionality is reviewed by the product manager that designed the feature. This way, we have no big surprises when everything is done, everyone can see the product progress day by day.

I won't say we ship a 100% bug free product, but our quality is through the roof compared to what it used to be, and it's actually much higher than we were even expecting.

You forgot to mention CVS (http://www.cvshome.org/). Usually database developers commit their changes to their stored procedures directly into the only database, where they are stored. That is not good, because other Java developers are using the same database at the same time, so they see intermittent bugs.

It is better if all code goes into CVS, even stored procedures. Not to mention that every developer must have a copy of the database in their own machine, usually MySql (http://www.mysql.com/) or HSQL (http://hsqldb.sourceforge.net/) is better for that.

Then from CVS you take out a build (you use CruiseControl to generated builds that are compiled and unit tested using JUnit http://www.junit.org/) so you know all the Java code works with the stored procedures, the database model and the example data. Then you can add your code, with your unit tests, into CVS.

My $0.02 CLP (= $0.0000318238 USD according to http://www.xe.com/ucc/)

We test using the following on web apps:

• Target browsers for intranet apps (even though we use standards as much as practically possible)
• W3C validators for HTML, CSS, and Links
• Validators within WebSphere Studio (Java, JSP, HTML), HomeSite (HTML) and TopStyle (CSS)
• JavaScript Console and Debugger in Mozilla/Firefox
• JUnit
• Cactus
• People. The users. The project owners. Us. Other web developers on e-mail lists.

• We aren't currently using an automated tool to test the front-end flow, because we haven't found any good, easy-to-use, and cheap tools that support a modern version of DOM/JavaScript usage. If you know of something that you like and works, I'd love to know about it. I've tried httpUnit, but had trouble setting it up and it didn't support all the DOM methods we were using at the time.

Is Junit any way linked with extreme programming? It sound that their goals are quite identical. However their ore informationmore informationsection doesn't mention JUnit anywhere.

Well, JUnit's website has the title, "JUnit, Testing Resources for Extreme Programming".

Let's see, from juinit.org....

JUnit is a regression testing framework written by Erich Gamma and Kent Beck...

From the credits for Extreme Programming Explained....

Kent Beck (Author), Foreward by Erich Gamma

Besides which, junit.org has links to 3 other extreme programming sites, so i'd say you're onto something...

Actually, this is VERY IMPORTANT! I think the big advantage(s) of "open sourcing" java will be seen when things such as the mess with the logging API's and the use of the assert keyword are avoided.

It is still a mystery to me why Sun developed their own logging API's when LOG4J was widely used and accepted.

Hopefully a more open approach to Java would help projects that are housed at Jakarta and SourceForge actually make it into the JDK instead of sticking us with inferior rewrites.

The logging API is just one example. Imagine if the JUNIT implementation of assert was used, and if SWT could be combined with Swing/AWT to create better/superior user interfaces. I think Java could grow in leaps and bounds with an open approach.

Another good example of this would be the JDOM project. How long has it sat in the JCP? While in the meantime Sun implemented their own INFERIOR XML libraries.

The JCP is too political, and needs to modified/done away with. Let the people decide the direction of JAVA!

Just my .02

--Ryan

An addendum: your experiments don't have to be anything spectacular. Here are a few:

• Try adding a few printk statements throughout the code, decide when you expect they will print and what they will say, and confirm it.
• Try tuning a sysctl or hardcoded value. See how it affects performance. Do microbenchmarks or benchmark real systems. Make pretty graphs. (Graphs are fun!)
• Add a likely() or unlikely() option to help the compiler's branch prediction. Again, test the performance. Or take one away - it's possible someone was wrong about the performance

• compile with -ftest-coverage to see what code is run and design unit tests to that uses every line. This will help your understand, could find existing bugs or dead code, and will help you be confident in your changes if you later make any.
• compile with -ftest-coverage -fprofile-arcs to see what code branches are very likely or unlikely to be taken, then use likely() and unlikely() to change the branch prediction. Benchmark the results.

Another project I've considered is adding likely() and unlikely() equivalents to boost. A lot of people use this code, so increasing its performance a little would be pretty beneficial.

The first wow, that's cool moment I had with test driven development was from an article on Object Mentor called The Bowling Game. It also highlights pair programming a bit.

The unique thing about the article was that it was presented as a discussion between two developers pair programming doing agile test driven development of the game. It was like watching over their shoulders.

If you want to get an idea of what extreme programming is like, I suggest reading this article AND writing the code and tests along with it, either in Java (and JUnit) or C#/VB.NET (and NUnit) or another language with a xUnit unit testing framework. Most object oriented languages have them now so you don't have to roll your own framework.

Nothing reality tests the usability of a proposed API then writing unit tests against it.

For those who develop in Java, may I propose JUnit? If you want to test the GUI of a web server, then try HTTPUnit although the value of writing unit tests to this is less since GUI is usually subject to a lot of changes over time.

No methodology is perfect for every team and project.

Absolutely, and the best approach is to pick and choose ideas from various methodologies that best suit your project and team.

There are a couple of ideas that XP encourages that I think can be beneficial to a wide range of projects.

Unit testing and testing-first is an approach that I think is a great way of testing your testing. The programmer writes a test, and then writes enough code to pass the test, then they write the next test, and enough code to pass that, and so on. This is an excellent approach because it ensures that code is well tested, and its less of a chore for the programmer because its all part of the coding and the problem solving process, rather than the thing you have to do after you've done all the interesting stuff. There are various unit testing frameworks to support this.

I also like the idea of continuous integration. Where every time a change is submitted to source control a complete build of the system is done, all unit tests are run, and any other checks are completed e.g. coding standards etc. The success of this is reported back to the programmer so any errors can be fixed.

Does tools like UML help with this?

What tools does Linux have for creating regression tests?

Debian and Gentoo does this well. I'm not certain how the others compare.

In addition to the replies posted above unit testing tools like JUnit (and its analogues for the other languages) are getting pretty popular today.

Most of the modern popular Java libraries have unit tests suites. Of course, unit tests are no panacea per se for best results, one has to use at least some of testing plan (and it will be way better than no plan at all). Tools like Clover help it somewhat.

Generally, it seems like consistent testing practices have all chances to become widely adopted by the open source community.

One word: JUnit

XP was built on the knowledge you just mentioned: That the Q&D solution is necessary, and that if you try to follow set procedure (or the "waterfall model" as it's called) where you have a requirements, design phase, implementaion phase, testing phase you will most likely fail.

Basically, what XP is all about, is to acknowledge that the specification is useless, beacuse after a couple of years when the project is finished, the end result doesn't look anything like the sepcification anyway. At least if you want to survive in a dynamic market. We have all seem that in the real world, the requirements change during development, and if they do, you need to go back and change the spec, and then possibly reimplement large parts of the code.

So, how do you go about solving this? Well, first of all you have to understand and believe in a couple of mantras:

• The implementation is the specification
• Refactor mercilessly
• Do the least thing that could possibly work
• Test driven development

Here is the list of development rules.

The implementation is the specification

This means that instead of writing a specification before programming begins, you let the application evolve (very similar to most Open Source projects actually) and if the requirements change during development, you change the code to adopt.

Refactor mercilessly

This is absolutely nessecary in order for the previous point to work. What it means is that you should not be afraid to change the layout of working code, to make it easier to add new features. With good refactoring you don't need a complex design in the beginning, which means that you get to market more quickly.

I strongly recommend you read Martin Fowlers book Refactoring, it's a real eye-opener.

This leads us up to the next point:

Do the least thing that could possibly work

If you are thinking of implementing a couple of abstract base classes and interfaces to make your object design super generic, so that it can be used for a lot of differnt things in the future. For example implementing a plugin architecture in your file parser or somehting like that, you need to ask yourself the following questions:

Am I going to need this abstraction now? and If I need it in the future, will it be easy to refactor the code so it gets that functionality? If your ansers to there questions are "no" and "yes" respectively (which they usually are) then you should not do it.

Essentially: don't do anything until you need it.

Test driven development

All this refactoring, and no solid specs can be a bit scary, especially in the beginning. A common question that pops up is: "how can we guarantee that the code works if you keep changing it all the time?". The answer is: unit testing. The rule of thumb here is: Implement the test first, then you implement the code so that the rest runs. Whenever you are going to fix a bug, write a unit test that triggers the bug first, and then fix the code so that the test succeeds. You then make sure you run the ever growing test suite several times per day. It helps a lot in catching regression bugs.

For Java, I recommend JUnit.

Now, the biggest problem you face is selling XP to your PHB's. They will more than likely feel that they are losing control, and they will be afraid that their nice Microsoft Project documents will become useless (no one seems to remember that (almost) every single waterfall project will overrun both budget and time constraints). However, there is

I work at a pretty large .com, one who actually survived the bust and maintains a profit, and has a pretty significant amount of traffic. We have used ATG Dynamo for our application server for several years, partially based upon the built-in ability it has to do an MVC architecture, personalization, pools, and so forth.

However, we just completed a web application that was built using many open source components, including Struts, Validator, JUnit, and others. By using open source components we have completely divorced ourselves from using the proprietary technologies used by Dynamo, and have opened ourselves up to the possibility of using a different, and of course cheaper, application server. This would not have been possible were it not for stable, performant open soruce initiatives.

Not only is management happy because we have (potentially) saved a bunch of money, but the developers are happy because they are much more friendly towards open source than closed technologies; it is far easier to get an answer to a question via Google than it is to pay for and go through the hassle of using a support contract of some kind.

I do not mean to denegrate Dynamo at all, because it is actually a fairly good application server. The licensing costs, however, just cannot be justified when so much of the functionality provided by it is already available elsewhere, for free.

Back in the Middle Ages, when the Catholic Church wanted a Cathedral built, they would pay a bunch of Freemasons to do it. The Freemasons viewed themselves as creative artisans, and they closely guarded the secrets they used to construct these impressive houses of worship.

The method they used, however, was less than impressive. Typically, they would start with a general design, and piece together stone and mortar until something collapsed, which happened quite often. Then they would patch the section that collapsed and keep on going until something else fell down, or they finished. Given the level of understanding with regards to Physics and Material Science, those Freemasons has no other choice than to build them this way.

Now fast forward to the 21st century. The engineering disasters on par with those medieval collapses can be counted on one hand (Tacoma Narrows Bridge and the Hyatt Regency walkway collapse are the only two I can think of). This is directly due to the fact that a civil engineer can determine if a design is structurally sound before they build it.

Contrast this with modern day software development. We can't even tell if a system is flawed after we build it, let alone before. So software gets written, deployed, and put into the marketplace that has no assurances whatsoever of actually doing what it's supposed to do (hence the 10,000 page EULA).

You can't have Civil Engineers until you have Physics. And you won't have 100% bulletproof software until you have Software Engineering. And you won't have that until someone can figure out a way to prove that a given peice of software will perform as it's supposed to. JUnit is a step in the right direction, but there's still a long way to go. It's going to take a breakthrough on the order of Newton to make Software Engineering as reliable a discipline as Civil Engineering.

Sun has been burned once when they introduced the assert keyword and broke thousands of programs that use the JUnit testing framework. The bustage occured because JUnit already used the keyword as an identifier, for the assert() methods. A token can't be an identifier, so everyone's tests broke with 1.4 I applaud Sun from learning from that fiasco and avoiding a repeat.

to the contrary. one principle of extreme programming is to "write your tests first" with a test suite like jUnit. doing this makes you think hard about the behavior of functions you write.

I interviewed Martin Fowler, author of Refactoring, last year. In this interview I asked him to define refactoring. He said:

Refactoring is making changes to a body of code in order to improve its internal structure, without changing its external behavior.

We didn't refactor JUnit's code, because we didn't start with JUnit's code and make changes. But we did do what I consider a "refactor" of the design. We started in effect with JUnit's API, and made changes. We started with knowledge of what we liked and didn't like about JUnit's API, and refactored that.

Where you can see the refactoring is not in the code, but in the JavaDoc. Compare JUnit's JavaDoc with SuiteRunner's JavaDoc. I'm guessing it was version 3.7 of JUnit that I had in front of me when I decided I would start over. It may have been 3.6.

Not necessarily true.

If you start building your prototype with solid testing and you apply effective refactoring your little prototype can grow into a solid and clean system.

Java has great tools for these jobs (junit and eclipse), but you can find similar tools for almost every other language. Give them a try.

Fh

The inadequacies of debugging (not just the dubuggers themselves) as a method of ensuring quality is why methodologies like agile programming and extreme programming are garnering so much interest from developers. It's also why there has been an explosion of tools that can help with the creation of good software with fewer bugs, easier maintenance and a better chance of actually solving user problems. In the Java world, there is junit, mockobjects and log4J , that can all be used with ant so that tests can be automatically run with every build and source control systems can be automatically updated. One approach is the Naked Objects framework, which is a combinination of a design methodology and tools in one package that has advantages like the easy rapid prototyping and automatic user story docuemtation.

For example if you were using the Naked Objects framework to create a system, you might take the following approach (simplified here for posting):

1. Start with an Exploration phase, which includes:
   1. Meeting of developers, administrators and end users to discuss the nature of the problem and brainstorm possible ways to implement the solution.
   2. Create a rough prototype immediately (Frameworks like Naked Objects makes rapid prototyping like this easy).
   3. Go back to the meeting, have everybody play with the code and make more suggestions and changes. Demonstrate the prototype frequently to elicit feedback.
   4. Repeat (often) till people are mostly happy with the results.
   5. This prevents the all-to-common situation where a written specification results in excellent software that doesn't actually solve any of the problems that the users have.
2. Specification Phase:
   1. Use what you've learned to write a specification that includes lots of special case scenarios (use-cases, user stories etc.)
3. Delivery Phase:
   1. Throw out all of the code you've already written... maybe keep the class and method names, but no actual functional code
   2. Write the tests FIRST. This includes unit tests, acceptance tests and corresponding documentation.
   3. Write the code to implement various user stories and test frequently.
   4. Auto-generate user training manual, which is based on the acceptance test code.
   5. Deploy to beta testers and be amazed at how happy they are with the product and how few bugs there are.

4. Notice that approaches like this emphasize getting the design right and doing constant testing to ensure that the code that is delivered actually does the job that is expected of it. Debugging is rarely required when such methodologies are used, though profiling can be of some use.

You write tests to encompass these larger pieces of functionality. In general, debuggers are usually used at the lowest levels: checking variables and references, and sometimes as far as processor registers. At higher levels of abstraction, the debugger becomes less useful, as you're generally not looking for null pointers and the like.

When unit testing, once you identify a bug, you write a test case that fails, that you would otherwise expect would pass. Fix the bug such that the test passes, and repeat. This works pretty well in the real world, and the proliferation of unit testing tools (both free (JUnit, CppUnit) and commercial (Parasoft)) is a testament to the usefulness of this technique. Many professional developers I work with haven't dusted off the debugger in years...

[I] have yet to see a reason to abandon debugger use.

I'm not advocating abandoning debugger use, if you find it truly productive. However, over the course of a project I spend less time writing unit tests than I would if I had to fiddle with a debugger.

Sometimes you need to see the cogs in motion.

And this is where configurable logging comes in. It's an automatic process. It's also a lot less painstaking than remembering what watchpoints and breakpoints you had to use when you saw this bug happen last week while fixing somethig else.

I have found Extreme Programming techniques to be very useful. I do much less debugging now, because I write unit tests as (or even before!) I write the methods. I use the JUnit framework for all the Java code that I develop.

I'm sure that debugging technology can and should be improved, but I think that better development methodology reduces the need for debugging.

I bought the book, and that's saying a lot because I'm very cheap. It's a great book that doesn't just teach you how to use Ant, it teaches you how to do lots of things with Ant, like build web services using the excellent Apache Axis, which can automatically create a web service from any Java class source file. It also teaches you how to use JUnit to do automated unit testing with Ant, how to use CruiseControl, etc., etc.

A lot of posters have complained that XML isn't very human-readable. I use and highly recommend a great little tool called Pollo. It has built in support for creating Ant build.xml files, as well as Cocoon sitemap files, XML schemas, etc. IMHO, it's got the best XML editing GUI I've found.

I've never used it, but it seems like it would probably be pretty helpful.

-NiS

Is your Websphere app unit tested? Use JUnit, and potentially add-ons like Cactus and HttpUnit so you've got some way of verifying your port.

If you decide you need a consultant, remember that the only thing worse than an expensive consultant is a cheap one.

I think JUnit-style testing works great, and I plan to start using it more often.
Testing is good to verify that your code does exactly what you think it does; a lot of the time I produce code that I "think" works, using JUnit allows me to verify that it actually does.
Check out junit.org.

For those of you who are sceptic about unit-testing, you should try it. Setting up the tests are not as tedious as one might think, they force you to think your problem through, and maybe most of all: they make your build look cool :)

This is exactly why methodologies like Test First have been gaining popularity, along with their supporting tools like JUnit and NUnit, exist. When you're coding strictly against you're compiler, and perhaps some defined design, the only errors you'll see are compile time, and in the case of java or (insert scripting language here), run time errors. Logic and design errors are revealed through testing your software, usually in the form of unit and/or integration tests, figuring out if your software does what it's actually supposed to do.

Test First methodology actually takes this process one step further, elevating logic errors to the level of obviousness that compiler errors have. This is key, because it helps break the mindset of the assuming programmer that if it compiles, it must be fine.

Unfortunately, the tools I've seen that support test first only work in the context of testing Java or .NET (C#, VB, etc) code. Because of the complexities involved, I doubt a generic CUnit or C++Unit could ever be engineered for generic distribution. The best support I've found is just to get the developers to think within the guidelines of test first.

I personally think this methodology, like any other should be viewed as a possible tool to use to solve some problem, in your specific project environment and schedule and what have you. But thinking along the lines of trying to test your software before you build your software will help you to write more robust code.

-ds

References:
http://www.junit.org/news/article/t est_first/
http://junit.sf.net
http://nunit.sf.n et

Most bugs are found by the developers themselve while doing code review, this help if their schedule have time geared toward review and not only coding.

Then using the right programming langage and the right engineering practice help.

Most modern langages don't suffer from buffer overflow and have eliminated most memory leaks. Increased support by tools af design by contract and the like will help.

Good testing practices like those used in JUnit (unit testing) help tremendously too.

What's shocking to me is that almost no open source authors or advocates give a hoot about automated testing of any kind. The only free software I've found with a test suite is gcc.

Bullshit

What's shocking to me is that almost no open source authors or advocates give a hoot about automated testing of any kind. The only free software I've found with a test suite is gcc. As much as I hate to say it, there's a good chance that the relative inexperience of most open source authors is a factor here.

Perl is really good about this. The Test::Harness and Test::More modules make it very easy to write test suites, so CPAN modules have lots of automated tests. It might even be a requirement to get a module into CPAN; I'm not sure.

PostgreSQL has regression tests.

There's a really nice test environment for Java code called JUnit. Lots of stuff is using it. Lots of articles about how to write effective tests. There's a project to develop mock versions of common objects (servlet requests, SQL queries) that fail in interesting, predefined ways. I'm using a C++ workalike called CppUnit in one of my projects.

The Boost code has automated testing.

There's a project called qmtest.

The Wine people have recently started using regression tests.

I've never used assertions, but they seem a great complement to unit testing. Unit testing allows you to write code to test your functions and easily see if something breaks, the major problem is that they lack an easy way to look inside objects to keep an eye on internal consistency. Assertions can be great to catch those silly little boundary mistakes.

A good unit testing framework for Java is JUnit, they are available for other languages as well.

BTW, you can create your own assertions with Log4J, so even JDK 1.1/1.2/1.3 users can use them:

if(logger.isDebugEnabled())
if (bla>10)
logger.warn("bla>10, bla=" + bla);

This uses almost no CPU-time if debugging is disabled. Log4J is a very good logging package, it surely beats System.out.println, check it out!

I would like to see a book on some of the Jakarta projects, specifically: Ant, Velocity, and Turbine.

also a book about JUnit, especially if you can tie it in with Ant, etc, using real examples of setting it all up.

and again, dead tree, please, until that whole 'digital paper' thing pans out.

-rp

nobody's figured out a way to quantify software quality and automate software QA

Yes they have. It's called unit testing.

It might be better to say: nobody's figured out a way to make software managers allocate the extra time up front to set up a decent unit testing framework.