Unit Test Your Aspects

An anonymous reader writes "The widespread adoption of programmer testing over the past five years has been driven by the demonstrable productivity and quality of the resulting code. Find out why and how to do it, as this article introduces you to the benefits of testing aspect-oriented code and presents a catalog of patterns for testing crosscutting behavior in AspectJ."

Comments

Unit Testing In The Schools...

[__aaclcg7560]

Why does it seem like unit testing is only taught in Java programming courses? I have never seen this in any C/C++, C# or Visual Basic courses.

Re:Unit Testing In The Schools...

[j-tull]

Unit testing, in my limited experience, isn't really taught in any programming course. Testing methodologies are typically taught in software engineering courses.

My software engineering course was fairly language independent (though most students used C++ since that was the primary language taught at this particular institution). Testing, in general, is a language independent craft.

I guess what I'm saying is that I disagree with your premise of unit testing only being taught in Java courses. I would guess that it's typically (but not always) taught in the same language as the given institution's introductory/intermediate programming classes.

Re:Unit Testing In The Schools...

[TrappedByMyself]

Why does it seem like unit testing is only taught in Java programming courses? I have never seen this in any C/C++, C# or Visual Basic courses.

Blame it on JUnit. Since it provides a way to make unit testing easier, it's easier to get people to actually do it. When I was in school, we talked about unit testing, but never really did it very well. By employing the latest buzzwords such as JUnit and Spring in your design, it's much less work to build your unit tests that it would be with other languages.

I also think, for better or worse, the the Java community is much more open to the latest toys than users of the other established languages are, which may explain why the XUnit variants in other languages haven't taken off as well.

Re:Unit Testing In The Schools...

[tcopeland]

Dr Stephen Edwards teaches about this in his classes. He's written an interesting paper "Using Software Testing to Move Students from Trial-and-Error to Reflection-In-Action" about his experiences with test driven design at VA Tech. You can see his home page here and that paper is the third one in the list.

I've recently been working on a BlueJ extension for PMD and he's quite active on the bluej-discuss list.

Re:Unit Testing In The Schools...

[Coryoth]

Blame it on JUnit. Since it provides a way to make unit testing easier, it's easier to get people to actually do it. When I was in school, we talked about unit testing, but never really did it very well. By employing the latest buzzwords such as JUnit and Spring in your design, it's much less work to build your unit tests that it would be with other languages.

At this point I'd like to point out the existence of JML, an annotation syntax and set of associated tools for Java that makes a lot of things easier and more convenient. JML allows you to do design by contract in Java and if you compile with the provided compiler jmlc it will automatically include runtime checks based on your pre and post conditions and invariants. Another tool, jmldoc, will generate JavaDoc documentation augmented with the extra specification your provided with JML annotations, making for much better documentation. You also have jmlunit which will automatically generate JUnit test classes based on your annotations.

That means you get automatically generated unit tests, the constraints of which are automatically included in your documentation (as well as in the actual code as annotations) and the option to use those constraints as runtime checks for debugging purposes. That's an awful lot of cheap benefits just for writing annotations to your code as you go.

General examples of using JML for DbC can be found here, and examples of jmlunit generating JUnit test classes can be found here.

Jedidiah.

Re:Unit Testing In The Schools...

[timcrews]

I don't know anything about the teaching side, but the JUnit framework has an equivalent C++ version, known as CppUnit. See http://cppunit.sourceforge.net/cppunit-wiki/FrontP age for details. NUnit has also been noted elsewhere as a .NET alternative.

Re:Unit Testing In The Schools...

[JSR $FDED]

Actually, take a look at TestNG if you want to get a sense of where testing in Java is headed (and I mean testing, not just "unit testing", which is just a subset).

Re:Unit Testing In The Schools...

[dzfoo]

>> http://frayspace.com/toomuchtv/isntthatgood/merryf unday.html

Simple reason:

C/C++ programmers: It works, dammit! We don't need no stinkin' testing.

C# programmers: The compiler would warn or fail if it didn't work; if anything goes wrong in production, we'll patch it in the next release.

Visual Basic programmers: Uh... testing? wuzzat?

          -dZ.

hmph

[Dominatus]

My unit requires no futher testing, thank you very much!

Re:hmph

[slackmaster2000]

I try to test my unit at least twice a day. So far no problems have been found, knock on wood (pun!).

I unit test all the programmers I hire...

[Anonymous Coward]

...but discard most of them due to faulty design.

  I do inquire if they have any siblings that may be been properly engineered.

Less time for proper testing

[Barkley44]

As important as testing is, many clients (at least the ones I've dealt with) are willing to place testing on the back burner in turn for more output for the same amount of money. If code works right 95% of the time on the first try, that is a sacrifice they are willing to make. Obviously the more critical the product, the more testing is required.

Re:Less time for proper testing

[Barkley44]

Oh I agree - I'm refering to documented, structered testing by a third party. Even though developers can test it, it is more effective to get someone not in the development process to do the testing. You know, that one case you never thought about, no way a user would do XYZ ;) These are the cases where clients are willing to pass on since the budget doesn't allow for it. If they have $1000 to spend, they want as much physical code as possible. Spend $900 on dev, $100 on testing (as in work is done, test and fix bugs). The other option is $500 on dev, $100 on my own testing and $400 on a third party. I personally will do either, but the clients I've worked for always opt for the first option.

Re:Less time for proper testing

[misleb]

Testing, in this case, is a little different than what you are referring to. I dunno about AspectJ, but with Ruby on Rails you basically program in a bunch of use cases and mock data manipulations and run it with a "make" file. Any time you make significant (or not so significant) changes to the code, you can run your tests and make sure it all works.

And if you really want to get serious about tests, there is technique where you program the tests BEFORE you implement the code. You just keep running the tests until they don't fail and you know you are done (well, almost). It is called test driven development. From that I understand, it saves a good deal of time that would otherwise be spent clicking through an application to see if it works.

Personally, I haven't been able to get into it too deeply. Writing new tests from scratch in a project for which you've already written a big chunk code is daunting and tedious, IMO. You literally have to go step by step through the program and simulate every forseeable use case. I did write some tests and I can see the benefit. I tested what I thought was some rather simple code and found bugs right away. Bugs that might not have been obvious by simply clicking through the application. Also, writing such tests can give you ideas for features that you might not have thought of before.

At this point, I think it is more an issue of discipline than man hours. If I could just get myself to spend a whole day getting tests written, I believe I would save myself far more than that in debugging and traditional testing. And I would feel much more confident about my finished product.

-matthew

Re:Less time for proper testing

[dubl-u]

If code works right 95% of the time on the first try, that is a sacrifice they are willing to make.

That may be a possible choice with traditional QA testing (which you do right at the end), but that's not the notion with unit testing. I don't write tests to increase quality as such; I write them to increase my development speed. If the code usually works, I spend very little time debugging. When I do make mistakes, the test coverage means I generally find out right away about bugs, so the problems are easy to find. And when I'm doing new work, the existing suite reminds me of all the little details that went into the existing code base, meaning I have to spend a lot less time on design archeology before ading new features.

I don't do test-driven development just for better quality. I do it because it helps me go faster, and with less stress to boot.

Re:Less time for proper testing

[Q2Serpent]

A good idea to get you started would be to write testcases for the problems you debug and solve manually. This has a few benefits:

- You are now familiar with the functionality of that piece of code, so writing a test should be easy.

- Having a test ensures that no future code changes will cause the same bug (it is very embarassing to re-introduce a fixed bug).

- Having a test also ensures that other code changes don't affect anything related to this functionality working. This can also help uncover unnecessary coupling - when this testcase fails in the future because something seemingly unrelated changed, you can investigate what is really going on because you will be alerted to it much sooner.

Re:Less time for proper testing

[stg]

Personally, I haven't been able to get into it too deeply. Writing new tests from scratch in a project for which you've already written a big chunk code is daunting and tedious, IMO. You literally have to go step by step through the program and simulate every forseeable use case. I did write some tests and I can see the benefit. I tested what I thought was some rather simple code and found bugs right away. Bugs that might not have been obvious by simply clicking through the application. Also, writing such tests can give you ideas for features that you might not have thought of before.

A great book on the topic of adding tests to old code is Working Effectively with Legacy Code. There are a lot of techniques there that I've found very helpful while refactoring projects done before I started using automated tests (in my case, using DUnit).

Writing tests really helps reducing excessive coupling in your code, too. If you need to use and initialize a lot of classes for a simple test, it's usually a sign that they are a bit too dependent.

Ironic

[snevig]

There's a javascript error in the article. Line 8, Char 76: Unterminated string constant

Re:Is this statement a joke?

[sedyn]

"Software Engineering is making leaps and bounds"
"However, more than ever, there are a very large number of incompetent developers out there."

Do you think that these statements are related in any way? I mean, using software evaluation / developmentation method X seems to be the equivilant of having a safety net. And there's definately a lot of developers out there that need one.

For .Net Developers...

[jferris]

..take a look at NUnit, if you haven't already:

http://www.nunit.org/

Then, read Marc Clifton's series on Advanced Unit Testing in C#. The code is easily ported to VB.Net, as well, although not required. I am working on introducing the practices outlined in the article where I am currently employed.

http://www.codeproject.com/csharp/autp1.asp
http://www.codeproject.com/csharp/autp2.asp
http://www.codeproject.com/csharp/autp3.asp
http://www.codeproject.com/csharp/autp4.asp

As if CodeProject wasn't slow enough. The readthroughs on this post should bring it to its knees in no time at all. If you have a chance, look at some of Marc's other postings, as well. Very high quality stuff.

In regards to Unit Testing in general, I don't know why it isn't given more weight in college coursework. Honestly, it would make a great course, or series of courses. I've been out of school for just a wee bit though, so maybe some are offering it already. ;-)

Aspect-oriented?

[jshaped]

ok, I'll take the hit and be the first.

wtf is Aspect-oriented programming?
how does it relate to object-oriented programming?

Re:Aspect-oriented?

[Trevahaha]

From Wikipedia:

In software engineering, the programming paradigm of aspect-oriented programming (AOP), also called aspect-oriented software development (AOSD), attempts to aid programmers in the separation of concerns, or the breaking down of a program into distinct parts that overlap in functionality as little as possible. In particular, AOP focuses on the modularization and encapsulation of cross-cutting concerns.

Re:Aspect-oriented?

[Peter La Casse]

The basic idea behind aspect-oriented programming is that you write code that will execute whenever a certain part of your program is reached. The most common example I've seen is that of a debug message that prints when every function is called; you could write the debug message in a separate function or file, and then via aspect-oriented programming not include code to call it in every single function. Instead you'd define an aspect, or something, and have your debug message print whenever the situation that defines that aspect occurs.

The promise of this is that you can save some effort by reducing repetitive function calls. The risk is that control flow can be very difficult to trace. Some people argue that aspect-oriented programming is similar to programming with goto, and I'm inclined to agree with them. It's useful under certain tightly defined circumstances and harmful otherwise.

Re:Aspect-oriented?

[Anonymous Coward]

aspect-oriented programming is a sure way to write spaghetti code.

What you do is you write a piece of code where part of it is a regular expression that matches one or more methods in your existing code, and another part is either the symbol "before", "after", or "both".

When you compile the code, the aspect's code is inserted before, after, or both before and after, the existing code. Your source isn't changed, just the binary.

So if every time the method Foo() is called you want it to call Bar() before and Baz() afterwards, this is a great way to do it.

The downside is, if you look at the source code at the place Foo() is called you will have NO IDEA that Bar() and Baz() are being called, because the code is inserted at compile time. If you debug the executable the call becomes obvious, but it's caused many a WTF on my part.

Lisp has had this sort of thing for years, with it's :before and :after directives.

Re:Aspect-oriented?

[archevis]

>> wtf is Aspect-oriented programming?

Aspect Oriented Programming is actually just a remake of the old comefrom pun. However, AOP is not a pun. It's a real life atrocity that should have been slain at birth. It can be visualized as having a piece of code "steal" the instruction pointer from somewhere inside another piece of code. And some morons seem to think that this constitutes good software engineering...

No, I'm not shitting you here, it's actually true! It just sounds like a joke.

And for the record: No, I don't hate Java. Quite on the contrary, I love Java. Furthermore, I am a heavily certified and devoted Java-only developer, and have been for 7 years. But AOP is a big-time fuck-up.

>> how does it relate to object-oriented programming?

It doesn't. Then again, most developers don't even know exactly what OOP really is, and what conceptually separates it from other paradigms. Hell, most "developers" don't even know what a programming language paradigm is.

So it all boils down to the buzzword factor.

Re:Aspect-oriented?

[ggwood]

There is an attempt at a jargon-free explanation here.

Re:Aspect-oriented?

[ajs]

"modularization and encapsulation of cross-cutting concerns."

And it's just that kind of buzzword-to-noise ratio that makes people ignore AOP.

In reality, AOP is a structured way for a programmer to modify existing classes an an OO system without sub-classing. So, here's AOP in Perl, just as an example:

package SomeClass;
...
sub somemethod {...}
...
# And then later in your code:
my $oldmethod = \&SomeClass::somemethod;
*SomeClass::somemethod = sub {
  print "Invoking somemethod...\n";
  goto $oldmethod;
}

I used Perl's stackless-invocation goto semantics here for two reasons: it's the most efficient way to do this; but it's also an eye-catcher that (because of the bad blood programmers tend to have with respect to C or BASIC style goto) highlights what I think the problem is.

I tend to try to avoid this kind of thing in my programs, regardless of language, except where I make it very clear that functionality can and should be added, in which case I provide a mechanism. So, I'm not sure how AOP could work well (it's supposed to be used in those cases where the original author didn't have any idea about what you want to do). Sub-classing or re-writing such code has always seemed the right way to go to me.

How, for example, are you supposed to maintain code where a substantial change to any library routine's internal behavior could cause catastrophy for someone who has tried to add behavior to it? I suppose you could lexically scope such constructs, which would be reasonable, but no... I think this is just an attempt to get a small portion of what smalltalk or ruby style mix-ins/traits would give you.

Then again, I guess the problem really stems from trying to use a high-level language (Java) which attempts to simulate the constraints of a low-level language (C or C++) while users attempt to use it as if it were high-level (like Smalltalk, Haskell, Python, Perl or Ruby).

In the end, it seems to make more sense to use Java the way it was intended to be used, and use high level languages where you want dynamic features like the ability to reach into someone else's code and do whatever you like.

Re:Aspect-oriented?

[Bloater]

> you could write the debug message in a separate function or file, and then via aspect-oriented programming not include code to call it in every single function. Instead you'd define an aspect, or something, and have your debug message print whenever the situation that defines that aspect occurs.

Is that like intercals "COME FROM" construct.

Re:Aspect-oriented?

[Anonymous Brave Guy]

Sorry, my mod points just ran out, or you'd have been (+1, Insightful).

Speaking of moderation, the fact that a post quoting the phrase "modularization and encapsulation of cross-cutting concerns" was modded (+1, Informative) is itself (+1, Funny). :-)

Maybe it's just me, but I've never appreciated what's supposed to be so great about aspects. Yes, it might be useful to be able to introduce some debugging diagnostics at the start and end of every function automatically. We use a pretty powerful instrumentation system in our code at work, but having to stick a macro at the start of every function to make it work is tedious (though hardly the end of the world).

But what else is it good for? I've looked into AOP a couple of times now out of curiosity, made the effort to read some of the background papers, and now placed it firmly on the level of so-called extreme programming: much hype, toy examples, few to zero real world case studies showing measurable benefits. I haven't read every single post in this discussion yet, but probably the majority, and I've yet to see anything other than the same old example (or thinly disguised variants of it) repeated ad nauseam. Maybe one of the AOP advocates can tell me what I'm missing?

Re:Aspect-oriented?

[rbodkin]

First off, it's quite ridiculous to claim that an open source technology with no marketing behind it is somehow a "buzzword." AOP is gaining interest because of the success of those who are using it. When I was with the AspectJ team at Xerox PARC we did a workshop with IBM and they decided to invest significantly in the technology for internal use (not for marketing). AspectJ is now shipping inside WebSphere.

Aspects are great because the provide a useful means to abstract the relationships for policies, so we can finally cleanly capture hard problems like performance monitoring, consistent error handling, enforcing data security, or allowing product line variability for changing features in an application. This last one is the strategic reason for IBM investing: they can open source components but still integrate with different runtimes.

Obviously you've considered this really carefully. I can just imagine your intellectual predecessors sitting there 25 years ago fuming about how polymorphism makes it impossible to tell where a call in the program goes. "Leave my line numbers ALONE"

Aspects are great because the fact is that traditional OO just doesn't do a good job of modularizing (look it up) these hard but important problems. Of course, you also should consider the real world consequences of this problem (code gen, overweening frameworks, fragile code, etc.) Aspects promote good maintainable software for the real world. Just as was the case with OO, this new level of indirection requires some learning and good tools support.

Re:Aspect-oriented?

[kavi_3]

I have to say, I can tell you've never seen a good example of AOP in action. It's a very powerful tool for certain kinds of problems. Crosscutting concerns is not just a buzzword. There are certain requirments in many software projects that cut accross many modules and sections; examples being security, transactionality, policy enforcement, the list goes on and one. One example is any problem that could be solved with the Visitor pattern can be solved with aspects in way that is far simpler and maintainable.

I work on a very large software project that uses aspect in several different ways. On example is for transaction management. We have an aspect that is able to start a transaction at any method marked with a @Transactional annotation. And you can create aspects that do things like inject a caching system objects in a way that doesn't force the object to be aware that it is being cached. Catch business events that should happen at certain times without littering your code with calls to raising that event. It an extremly powerful tool.

The problems with aspects are the lack of tool support. It can slow down compiles and mess with debuggers at times (I know, I've had to deal with this). However this will improve over time. Another improvement is runtime weaving, where the aspects are woven into the code at runtime. This allow you to run the aspects when you actually need them (production, integration testing) and not when you don't (unit testing).

And the compaint about not knowing where the flow of control is going is similar to the complaints about OO and ploymorphism. Yes, it can make it a little harder understand a program at first, but with the proper tools and some time, it will make the program far more maintainable.

You have to know when and where to use it.

Re:Aspect-oriented?

[tootlemonde]

And some morons seem to think that this constitutes good software engineering...

The "moron" in question here is Nicholas Lesiecki, author of Java Tools for Extreme Programming: Mastering Open Source Tools Including Ant, JUnit, and Cactus. This review in Dr. Dobb's Journal calls it "original and useful".

He is also a contributor to Cactus, "a simple test framework for unit testing server-side java code", part of the Apache Jakarta project.

He is currently a software engineer and programming instructor with Google.

Related Reference

[under_score]

I've written up a brief introduction to the qualities of an ideal test. The great thing about unit test frameworks such as JUnit, NUnit, CPPUnit, etc. is that they manage to satisfy all of these qualities: Decisive, Valid, Complete, Repeatable, Isolated and Automated. (Although it is possible to break some of these qualities with poor test creation practices.)

Re:Is this statement a joke?

[mcrbids]

Software Engineering is making leaps and bounds, so there are many tools available for writing good software. The people who really know what they are doing have more at their disposal. However, more than ever, there are a very large number of incompetent developers out there. It's much easier to get into programming now than it was in the machine language/assembly/C days because things are much easier to use, and the tools are much more accessible. Combine that with the .com influx of bozos and you have alot of people spewing out crap code.

Reminds me of a study done on Taxi cabs in San Fransisco shortly after ABS (Antilock Braking Systems) came out. As expected, the accident rate for those with ABS was quite a bit lower than those without ABS. But the surprise was that the death toll was about the same either way!

Seems that those with ABS tended to offset the benefits of ABS simply by driving harder, pushing themselves closer to the line for safety as a result of the certainty that they achieved with ABS.

In other words, people tend to identify "acceptable risk" and when the level of risk reduces, people then engage in riskier behavior in order to gain an advantage. This is not a bad thing - utilizing an advantage is an important part of survival in a "survival of the fittest" universe.

Edsger Wybe Dijkstra's qoute

[Device666]

"Program testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence."Quote - Edsger Wybe Dijkstra. I get the "jibbies" in knowing that code is tested but not proven correct by design (program derivation).

Re:Edsger Wybe Dijkstra's qoute

[msbsod]

Edsger Wybe Dijkstra truly understood the virtue of the Lambda Calculus. IBM software developers who advocate aspect-oriented Java code obviously still do not understand the concept, as the examples on the cited IBM Java web page show (see use of private variables).

Re:Edsger Wybe Dijkstra's qoute

[Dionysus]

I'm not sure I understand. What's the problem with using private variables?

Re:Programmer testing?

[adrianmonk]

Programmer testing? That means testing programmers, as in certifying them? Apparently not.

From the context, it seemed pretty obvious to me that "programmer testing" means testing that is performed by programmers, as opposed to by a separate part of the development team that specializes in testing.

Having the programmer do some of the testing of his/her own code makes sense to me for several reasons:

1. Coverage. The programmer has a more intimate knowledge of the code and thus has better insight into ways it can fail than people who are looking at the code as a black box, which is basically what testers do. For example, if the programmer knows the code uses his own implementation of a hash, he knows to think about adding a test where the input data has two strings that have the same hash value to see if it properly handles collisions.
2. Testing efficiency. Because the programmer knows what the code is supposed to do, what the contracts are between classes and their clients, etc., etc., it's easier for the programmer to construct certain tests than for another person to learn these things and then create the tests. (Although on the other hand, the second pair of eyes might find things that the programmer didn't think of.)
3. Development cycle efficiency. If a programmer does the tests as he's developing things, he will discover bugs much sooner than if someone else is doing them. Perhaps days sooner. In general, the sooner bugs are discovered, the better, for several reasons. One is that code is easier to fix when it's still fresh in your mind. Another is that bugs may indicate a design flaw that will need to be corrected and may affect interfaces and thus require changes to client code (which may be written by other developers), and that creates overhead. Also, it means development versions of the software will be more stable and easier to work with (fewer crash scenarios to avoid when demoing to the boss, checking out if it's as usable as you thought, etc.)
4. Tracking overhead. If the programmer fixes bugs while still in the process of developing things, then he is the only one who has to know about the bug. Which means it doesn't have to go into a bug database, nobody has to spend 5 minutes discussing it in a meeting, etc.
5. Finding the source. If the programmer does the testing, it's much easier to trace that back to the source of the problem than if the tester just notices that the software fails in a certain scenario. It takes work to go from a scenario to the point where you know whose code fails (and thus who should fix it).

I'm sure there are other reasons, but the point is this: in many cases, increasing the amount of testing that the programmer can do is advantageous. (At least up to a point -- you need a second pair of eyes, and you need someone who tests how all the modules interact to see if the system as a whole works as expected.) But still, finding ways to make it possible and to make it easy for programmers to add more tests and better tests is usually a good thing.

Re:Thank God for the 2000 tech crash!

[shmlco]

I actually had a degreed developer tell me, with a straight face, that he didn't do testing, as his professor told him it's the job of the QA department...

There is a suprise in Aspect-J

[metalotus]

Having tried Aspect-J in a production environment, I found an unhappy surprise.

When you move beyond the trivial examples, you will encounter Aspect-J's long-standing bugs that simply crash your IDE every five minutes. For example, this P1 critical bug makes Aspect-J unbearable:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=4421 5

I've heard of JBoss AOP being used successfully in production (http://www.jboss.org/products/aop).

Worked with AOP

[El_Muerte_TDS]

I worked with AOP a bit (AspectJ and Compose*) and I can tell you this. I have mixed feelings about the use of it. For example AOP is simply awesome if you want to add debug code to your whole application so you can easily add\remove call tracing\logging to find where the program craches (ofcourse you could do the same with in C\C++ with some macros, but it's a bit more work). On the other hand you will completely lose overview on how your AOP changes affect the program in a whole. Even with the visualisation tools that come with AspectJ is not easy.

Software Patents Considered Harmful

[-_broken_watchman_-]

[As Minna Kirai said]

And since Aspect-Oriented programming is a patented technique http://www.pmg.lcs.mit.edu/~chandra/publications/a op.html , basically you can not use it.

So, who really cares if its theoretically any good, when legally it is worthless?

See US patents 6,467,086 and 6,442,750 :

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PT O1&Sect2=HITOFF&d=PALL&p=1&u=/netahtml/srchnum.htm &r=1&f=G&l=50&s1=6,467,086.WKU.&OS=PN/6,467,086&RS =PN/6,467,086

http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PT O1&Sect2=HITOFF&d=PALL&p=1&u=/netahtml/srchnum.htm &r=1&f=G&l=50&s1=6,442,750.WKU.&OS=PN/6,442,750&RS =PN/6,442,750