Slashdot Mirror

esnyder queries: "I find the pragmatic/agile/XP hype about test driven development compelling, but find it hard to see how to test first (or even unit test at all) in some situations. I would like to explore some extended examples of it in a moderate to large scale real world codebase to improve my test design skills. Can anyone recommend some F/OSS software projects that consistently use test driven development processes that I could check out? Preferable over 50K lines of code, but I'd welcome pointers to anything that people think would be helpful."

bucketman writes "Saw this book at Chapter's and bought it and read it straight away. I've been kind of sitting on it for a month or two because my reaction to this book was so strong that I kind of wanted to see if time would mellow it. Well, it hasn't and I'm ready to post now. Note that this book has already been reviewed on Slashdot once, but I've decided to send in my review anyway, as it presents more detail as well as an alternate point of view and might be of interest." Read on below for bucketman's take on the book (and the methodology). Agile Software Development with Scrum author Ken Schwaber and Mike Beedle pages 158 publisher Prentice Hall rating 1 reviewer bucketman ISBN 0130676349 summary Presents the Scrum software development process.

Extreme Programming (AKA XP) has been an interest of mine for some time, as I struggle to find ways to make it easier to say "yes" to in my domain (embedded systems) and it is in the study of this other development process that I first heard of Scrum. Both XP and Scrum are development processes under the "umbrella" of the Agile Alliance.

First and foremost, let's cover what Scrum is, as presented in this book. I will compare Scrum with XP as I go, for reasons that will become obvious later on.

You have the Scrum Master, who is more or less half technical lead and half project manager. He is defined as being responsible for the success of Scrum. I think the closest thing that XP defines to this is the Coach, but it's a poor fit at best.

You have the Product Backlog, where all the stuff that people want in the product is written (called Work) along with assigned priorities for each item and an estimate of some form indicating how long it will take to do. The assumption is that items with high priority will have more accurate estimates and more precise specifications and the low priority stuff will be more or less SWAGs. The Product Backlog also contains Issues, which are more or less problems that gate one or more Work items. Issues are turned into Work by the Product Owner (see below) at his discretion. The analogous concepts in XP is the Story and the Tasks.

You have the Product Owner. He owns the Product Backlog document and the prioritizations within. While he may need to consult others in the effort to do his job, it is his responsibility to maintain the list of work and issues, to decide the prioritizations and to decide what work will be done in the next Sprint (see below). The Product Owner also owns the estimates in the Product Backlog. It is expected that he consults with development to derive these numbers. These estimates are said not to be binding on the Scrum Team (see below). The XP role akin to this is the Customer, with the exception that, in XP, estimation comes solely from development.

The Scrum Team is just the set of technical people working on the product. So testers, documentation people, developers etc. Should aim to be around seven people (i.e. the "optimal" group size).

The Sprint is the development iteration. It should typically aim to be 30 days. It has a Sprint Goal, which defines the overall objective of the Sprint (think of it as a mission statement just for the iteration), and a set of items from the Product Backlog to develop during the iteration. These items are chosen during the Sprint Planning Meetings mainly by the Product Owner. A secondary meeting is held with just the Scrum Team and Scrum Master to decide who will do what and in what order within the Sprint. Also done in this meeting is the breakdown of Work to Tasks, which are smaller units no longer than 4 to 16 working hours in duration. The XP Sprint equivalent is simply the iteration. To my knowledge, there is no equivalent concept to the Sprint Goal. XP's Planning Game does the work of both the Sprint meetings noted earlier.

Because the duration of the Sprint is respected first and foremost, the Sprint Goal is used to determine what content to remove from the Sprint in the event it is discovered that the deadline is at risk. So, we move work from the Sprint to the Product Backlog while attempting to honor the Sprint Goal in order to satisfy our (e.g.) 30 day schedule. If that is not possible (and in some other situations, like irresolvable organizational impediments) the Sprint is canceled and redone from the Planning Meeting.

You have the Sprint Signature, which tracks the expected Work done against the actuals and provides a day-to-day description of what happened in the Scrum Team (similar to, nut much smaller than, the project log spoken of by Steve McConnell).

Once the Sprint is on, no one outside the Scrum team has anything to do with the Scrum team. XP, again AFAIK, does not state this, but the fact that iterations are, if anything, even shorter than Sprints means that in all but the worst cases, the stakeholders will likely be willing to wait until the next iteration to redirect the effort. In any event, because both processes return so frequently to the customer for guidance and because both processes allow the customer to introduce change throughout the lifecycle,, there is less risk that the customer will feel their wishes are not being respected. Lastly, XP's customer and Scrum's Product Owner both funnel all requirements to development. All the project stakeholders then direct their requests to this person. In so doing, the stakeholders never get told "no" per se, but rather are told that at worst their request won't be evaluated until the next iteration/Sprint planning meeting.

At the daily Scrum Meeting, each Scrum Team member answers the following questions:

• What did you do since the last Scrum Meeting?
• What will you do between now and the next Scrum Meeting?
• What, if anything, impeded your progress?

So, you say how you did yesterday against your commitment from yesterday. You say what you plan to do today. All this is recorded in the Signature which is kept by the Scrum Master. Lastly, you note any obstacles that got in your way. It is the Scrum Master's job to note these and remove them immediately. The Scrum Master is expected to report on his progress in this area at each meeting as well. If the obstacle is a failure to make a decision, the Scrum Master is responsible for making that decision immediately - failing that, to ensure it gets made that day. I really like that, BTW, as experience has made clear to me that, on balance, the risk of making the wrong decision has a much lower negative effect on a team than does leaving it in limbo frequently. In the Scrum meeting, no one is allowed to speak save for the Scrum Team and Master. Others may attend but may not speak. This is intended to keep to meeting focused and short. I like this part but suspect that to do it successfully, you'd be wise not to invite anyone that's not intended to speak.

The Sprint Review is held at the Sprint's end and is intended to be a half-day forum where the Scrum Team presents to the stakeholders what it accomplished during the Sprint. The loose equivalent in XP is the final (for that iteration) successful execution of the Customer Tests (nee Acceptance Tests). I say "loose" because the XP concept is stronger - the iteration's deliverable is not only presented but accepted.

To get the Scrum process to scale, a Scrum of Scrums is used where Scrum Masters from multiple Scrum Teams attend a Scrum Meeting run by a Scrum uber Master.

So, AFAICT, that's pretty much it.In reality, I don't see a whole book's worth of content coming out of this.

So what else do they discuss? There's a fair bit on work environments, the ill effects of heavy weight processes and so on - basically restating stuff in less detail that most everyone who will read this book would be likely to have read elsewhere.

There's a lot on the characterization of Scrum in terms of Process Control Theory and why it would be expected to be a better fit for software development than would other processes. This left me cold, frankly, because I tend to read these books looking for what to do, how to do it and what benefit I should expect to see. The underlying science of it is of some interest but not nearly so important to me as the answers to those three questions. Also I already implicitly favor these iterative approaches. So do many, many others - after all, the evolutionary lifecycle model and the "mini-milestones" proposed by McConnell in "Rapid Development" echo both XP and Scrums concepts.

We also get stuff like this:

"Overlapping development phases: In an environment where some of the requirements are discovered while simultaneously something is created with the information at hand, it is imperative that the phases of discovery, invention, and testing overlap to drive the creation of a new product to completion through self-consistency. Most problems in new product development arise when the phases of the project are separated. Empirically, this overlap in phases enhances shared responsibility and cooperation, stimulates involvement and commitment, sharpens a problem-solving focus, encourages initiative taking, develops diversified skills, and heightens sensitivity toward market conditions."

Other than "don't use the waterfall lifecycle," what exactly does all that mean? The end of it is completely unsubstantiated. It drives me nuts when development books do things like this. That paragraph doesn't say much of anything AFAICT but nonetheless manages to set expectations way too high.

Here's another (about the psychological effects of Scrum on the Scrum team):

"They become deeply involved in their work. Scrum drives individuals to focus, commit and excel.
They focus on the work and lose concern for themselves.
They experience an altered sense of time.
They consistently produce at high levels of accomplishment.
Scrum allows developers to concentrate most of their time in developing software, and by doing so developers enter 'flow' state."

I don't know what any of that means and I'd be scared to find out.

Out of nowhere, you get statements like this:

"Scrum requires a balance of individuals with at least 50% of them to be experts ..."

Well, I guess that's that then. I think I could define a process which would have demonstrably improved efficiency over the lifecycle with just that one statement - worse comes to worst just sack the other half and away you go :) Anyway, this requirement appears for the first time on page 121 in a book with all of 154 pages. It's hard to say if it's intended to be taken seriously - if it were, you'd think it would have been a lot more front and center.

But let's go back to what Scrum is, as shown earlier. IMHO, it is not a lot more than XP with at least the following removed (this from memory, please don't be too harsh):

• Continuous integration
• Refactoring
• Unit testing
• Paired programming
• Collective ownership
• Customer-defined acceptance tests
• The "yesterday's weather" estimation practice
• Sustainable pace (nee "the 40 hour work-week")

So, all the hard/useful technical bits from XP are removed. In return, we get the Daily Sprint Meeting, the Sprint Goal, the Sprint Signature, and the emphasis on decision-making by the Scrum Master. The remainder of the Scrum practices have analogues in XP, as I've shown above. It's for this reason I found myself smiling when the book notes success in industry in wrapping XP in Scrum - from what I can see, XP wrapped in Scrum *is* XP :)

The book is careful to point out that any and all of these "missing" practices (refactoring, unit testing et al) may be used but that Scrum does not prescribe them. And that's fine, but I'm evaluating it on what it actually does prescribe - if Scrum can take credit for what it does not prescribe, they it can lay claim to infinite credit after all. This is maybe a small point but it's important to what follows. Because Scrum does not tell us what to do in these areas, I'm assuming that they are free to vary.

So, if you accept my characterization of Scrum as XP with all the hard technical bits removed. what might you see in a Scrum project?

If you follow Scrum's iterative path with no refactoring to offset the inevitable software entropy, the software would tend to lose architectural unity quixkly. It would be easy to break and hard to fix. - and so on and so on, for all the ills which refactoring is intended to address. If Scrum does not prescribe refactoring explicitly, then we must assume that not doing it is an acceptable Scrum path. If Scrum requires refactoring as a key practice, it should say so. XP and Scrum turn every project into a maintenance project but XP bolsters that position with all the technical practices that ensure the team is *really* good at maintenance. Scrum does none of this. Combine the lack of refactoring with no prescribed regression (unit) testing and I think it becomes clear that Scrum projects that do neither would risk devolving into code and fix affairs.

Because we have no review process prescribed in Scrum, which XP provides via paired programming, and no stated ownership model, we would expect to see Scrum projects which explicitly prescribe neither ending up with code like little fiefdoms where the divisions between one guy's code and another's are painfully obvious. We also would expect to see de facto individual code owners with all the gone-to-greener-pastures risk that entails.

Because we have no estimation practice prescribed we would expect to see poor estimates. Because the Product Owner owns these estimates, there's no reason to expect them to be meaningful at all.

And in return for all this, what do we get? We get the Sprint Goal, which is a fine idea, IMHO. It defines, a priori, a way to sensibly cut work within an iteration. We get the fast decision-making from the Scrum Master. This is also fine by me. We get the intense project progress tracking of the daily Scrum Meeting and the Sprint Signatures. Hmmm ....

And so, at long last, and very much in my long-winded way, I'm finally ready to state my real conclusion about Scrum ...

But first here's another quote from the book :) It's an example Sprint Signature description:

Day 16-17 The team is discouraged by all the work remaining and didn't work during the weekend. No changes in estimated time remaining.

Day 18 The team works more and its remaining work declined. The team then met with the Product Owner and the Scrum Master to determine what tasks could be reduced or removed while still meeting the goals of the Sprint. Some Sprint backlog was dropped; other estimates were lowered because not as much functionality had to be supported. Overall estimated work remaining reduced to 1400 hours. If all this work is completed, the team will still meet the Sprint Goal, although with functionality implemented less completely.

Day 19 The team continues work toward the Sprint Goal using the new Sprint Backlog. Estimated work remaining declines.

Day 20-30 Team is motivated because it can still meet the Sprint Goal if it works hard. The team works regularly including during the weekends. Estimated work remaining declines to zero as the team meets its Sprint Goal by the 31st day."

Here's one thing I've learned in the working world - death marches don't just happen on their own. It's not easy keeping a real death march going. You need to brutally cut and slash the work items in the finest detail to ensure that what you're committing to is actually what's absolutely necessary. To do this, you need to get stakeholders interested enough to grovel around in these nasty details. To really run a death march (and sustain one), you need to track everything everyone's doing down to the granularity of hours so you can tell immediately if someone has returned to a normal level of effort (people will tend to do this long before their actual *hours* spent at work decline). You need to make this tracking of effort very public so people feel intense peer pressure to keep their effort as high as their colleagues. You need to keep the goal always seemingly in reach and make sure this perception is held by everyone on the team because once the expectations are obviously unrealistic, most people will slack off. As long as the goal looks possible with significant effort, people will still buy into it.

And, in the end, that's what I think of Scrum: if you take away all those XP practices and don't replace them with anything, I think you end up with a really good way to run a code and fix death march. And if you put back all those practices, you're pretty much doing XP. I believe parts of Scrum can be lifted and applied in other processes (particularly XP) but that you'd be insane to adopt Scrum as defined in this book without addressing the risks detailed above."

You can purchase Agile Software Development with Scrum from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

RickHigh writes: "I've been waiting for a book like this. If you are doing software development of any kind, you should read this book. Especially if you are doing Extreme Programming and you errantly believe modeling has no place in XP. Or, your doing the Unified Process and you feel that your models and documents are more important than a working system, or you feel you are bogged down in documentation and required artifacts (more likely). Agile Modeling (AM) is a modeling methodology that enhances your modeling endeavors, whatever your process methodology inclination. Agile modeling will help you effectively incorporate modeling into your organization." Read the rest of Rick's review below. Agile Modeling: Effective Practices for Extreme Programming and the Unified Process author Scott Ambler pages 400 publisher Wiley, John & Sons, Incorporated rating 7.5 reviewer RickHigh ISBN 0471202827 summary A worthwhile melding of XP techniques with modeling of general interest to programmers of all stripes.

The title is only partially accurate as the book covers a lot more than modeling. I know from experience that picking titles is tricky (you can't please everyone). You don't need to be a UML expert to get value of this book. Any software developer should be able to appreciate it. Modelers, software developers and Yes, managers will find this material useful. In fact you may want to buy a copy and put it on your manager's desk. This book is original and well thought out. It is also well written and very readable. I wish there were more examples of applying the different artifacts in different phases of the XP life-cycle, but there has to be room for the next edition. The depth is appropriate.

Many UP developers (and those working with other prescriptive processes) get bogged down in the tonnage of documents and artifacts that are required. They wonder if they are ever going to have time to actually write the code.

XP offers a methodology for building high quality software fast. However, many XP developers, and I've spoken to them about this very subject on many occasions, find that XP does include time to do models. This books shows how to integrate XP and modeling.

This book sets the record straight about design and Extreme Programming. Actually, Kent Beck set the record straight with the first book on XP when he said 'In the time it would take you to code one design, you can compare and contrast three designs with pictures.' Kent Beck's views were corrupted over the years for various reasons that Agile Modeling explains, as it clarifies once and for all the relationship between XP and modeling.

The book emphasizes that you should model to understand the problems and only apply the right artifacts, i.e., that not all modeling must lead to writing code.

Despite the title, he AM book transcends just being a book on modeling; the book covers many aspects of developing software. Agile Modeling endeavors to be real. By real I mean it talks about real issues and how things are handled in the real world, not the perfect world covered by most books. For example, the chapter on documentation is an excellent coverage of the subject, but not idealized beyond all usefulness.

Like the original XP book, the AM book lists values, core principles and practices. It also adds supplementary principles like 'Content Is More Important Than Representation.' A key lesson from the book is that models are important if they help you understand and solve problems. And, models do not have to be perfect -- in fact they can be thrown away when you are done with them. After all is said and done, 'Software is your Primary Goal.'

The author, Scott Ambler is the author of numerous books (and by numerous I mean a lot) for the Unified Process, and UML. He also contributed to the Mastering EJB Book, and the Java Elements of Style book. All his work in UP seems strange since AM seems to have closer ties to XP than UP, but that is probably my own warped misconceptions of the world. Bottom line, Scott has mastered the craft of writing and I really enjoyed his writing style. The first few chapters seemed a little slow, probably because their content has been covered before in other books. The chapters on AM and XP, though, were informative and useful (as was the chapter on agile documentation mentioned earlier).

If you are doing software development of any kind, you should read this book. It is an informative and enjoyable read.

What's covered? Here are the chapter headings:

• Table of Contents:
• PART I: INTRODUCTION TO AGILE MODELING.
  • Introduction.
  • Agile Modeling Values.
  • Core Principles.
  • Supplementary Principles.
  • Core Practices.
  • Supplementary Practices.
  • Order from Chaos: How the AM Practices Fit Together.
• PART II: AGILE MODELING IN PRACTICE.
  • Communication.
  • Nurturing an Agile Culture.
  • Using the Simplest Tools Possible?
  • Agile Work Areas.
  • Agile Modeling Teams.
  • Agile Modeling Sessions.
  • Agile Documentation.
  • The UML and Beyond.
• PART III: AGILE MODELING AND eXTREME PROGRAMMING (XP)
  • Setting the Record Straight.
  • Agile Modeling and eXtreme Programming.
  • Agile Modeling Throughout the XP Lifecycle.
  • Modeling During the XP Exploration Phase.
  • Modeling During an XP Iteration: Searching for Items.
  • Modeling During an XP Iteration: Totaling an Order.
• PART IV: AGILE MODELING AND THE UNIFIED PROCESS.
  • Agile Modeling and the Unified Process.
  • Agile Modeling Throughout the Unified Process Lifecycle.
  • Agile Business Modeling.
  • Agile Requirements.
  • Agile Analysis and Design.
  • Agile Infrastructure Management.
  • Adopting AM on an UP Project.
• PART V: LOOKING AHEAD.
  • Adopting Agile Modeling or Overcoming Adversity.
  • Conclusion: Choose to Succeed.
  • Glossary of Definitions and Abbreviations.
  • References and Suggested Reading.
• Appendix A: Modeling Techniques.

Sites of interest

• The Agile Modeling site
• The XP programming site
• The Agile Alliance site
• Scott Amblers Website
• Applying XP to J2EE

You can purchase Agile Modeling from bn.com. Want to see your own review here? Just read the book review guidelines, then use Slashdot's handy submission form.