Questioning Extreme Programming

David Kennedy writes with the following review of Pete McBreen's new Addison-Wesley title, Questioning Extreme Programming. It's one of the few books which casts a questioning look at the buzzwords and concepts of Extreme Programming; read on to see how well XP fares under McBreen's examination. Questioning Extreme Programming author Pete McBreen pages 186 publisher Addison-Wesley rating 9 reviewer David Kennedy ISBN 0201844575 summary A critical but fair re-examination of all of XP.

In short:

This is bound to a controversial and widely read title -- it is a critical but fair re-examination of all of XP's assumptions and core practices. It provides a much needed comparison of XP with other, less popular, methodologies. Overall, XP emerges favourably, with one serious caveat -- the author concludes that XP is only suitable for a very narrow range of projects, and those that can fulfill all requirements probably stand a significant chance of succeeding using any of the similar methodologies. As with programming languages, there is no silver bullet -- put XP in your methodology toolbox, know when it is appropriate and only use it then.

A couple of interesting specifics:

• The author specifically argues, and I agree, that what the XP literature badly needs is a DSDM 'suitability filter' to advise project leaders as to whether XP is for them.
• In the preface, Kent Beck describes the On-Site Customer role as a team, and not an individual role.

Overall, this is an intriguing and well-argued book. It's based on solid, reasoned arguments, with conservative conclusions drawn based on the evidence. It's a must-read for those interested in XP and other Agile methodologies. The Extreme Programming Series

This is the 8th title in The Extreme Programming (XP) Series from Addison-Wesley, surely the most widely read series on a software methodology ever! (If that isn't achievement enough, XP also made testing sexy again. I hear that accountancy firms are looking for Kent Beck to do Public Relations work ...) For those of you who have been living under a rock for the past couple of years the previous titles are:

• Extreme Programming Explained (Beck)
• Planning Extreme Programming (Beck & Fowler)
• Extreme Programming Installed (Jeffries et al)
• Extreme Programming Examined (Succi & Marchesi)
• Extreme Programming in Practice (Newkirk & Martin)
• Extreme Programming Explored (Wake)
• Extreme Programming Applied (Auer & Miller)

This new addition to the XP library feature a foreword by Kent Beck. This is important as many of the reactionary XP fan-club will not like this book -- it challenges XP, and I am delighted to see this title as part of the series. Beck admits he doesn't agree with McBreen's conclusions, but asks you to read the book and decide for yourself, conceding that the arguments are fair and reasoned. I come from a scientific background and distrust anything except wide open debate, a position many who welcome XP will surely agree with. A book challenging XP can only help persuade people to give it a go, by addressing their fears and explaining how to manage any real risk.

Check your sources

Pete McBreen is the author of the excellent Software Craftmanship: The new imperative, a 2002 title from Addison-Wesley. In it he outlines an alternative to the software factory model behind much of traditional software engineering thinking. He proposes a collaborative model with small teams, where the software coder is seen as a craftsman in constant dialogue with the customer. Sound familiar? It should, this is a cross between a methodology and book of advice for career programmers, and fits squarely within the values proposed by the Agile Alliance, and arguably popularised most by XP.

I highly recommend Software Craftmanship, and can think of few authors who are as well positioned to give an analysis of XP as it currently stands.

What is the book about?

Questioning Extreme Programming does just that -- it's the first title in the series to take a skeptical look at the rise of this popular methodology and question some of the key assumptions. Arguably there was material like this buried in Extreme Programming Examined, but it suffered from a fragmented, detailed view, due to it being a bound set of conference papers.

The author tackles XP in a fair way -- he's extremely excited by the methodology, and it's clearly in accord with his own preferred approach. What he does is tackle each of the XP tenets in turn, questioning their validity, and then moves on the compare XP to other Agile methodologies and asks how XP stacks up against the competition. He also has a look at the common mis-conceptions (from both sides) about XP, and tackles the key arguments against its adoption in the same way.

Let's have a look at the contents to give you an idea of the structure:

• Introduction.
  XP: Hype or HyperProductive?
• What is a methodology?
  
  • What do methodologies optimise?
  • What are XP projects scared of?
  • What do other methodologies consider important?
  • What is important for your projects?
• Questioning the Core XP Practices
  
  • Planning incremental development
  • Truly incremental development
  • Are we done yet?
  • Working at this intensity is hard
  • Is that all there is to XP?
• Questioning XP concepts
  
  • The source code is the design
  • Test first development
  • Large-scale XP
  • Is the cost of change really low?
  • Setting the dials on ten
  • Requirements: Documentation or a conversation?
  • Is oral documentation enough?
  • Playing to win?
• Understanding the XP community
  
  • ReallyStrangeSayings
  • Feel the hostility; experience the joy
  • Transitioning away from XP
• Your choice
  
  • Is XP for you?
  • Do you have a suitable first project?

What's good?

The whole thing. Let's start with the basics, the high standards of the XP series are maintained, with flawless editing and layout. Moving on, the author's position is admirably neutral -- he is knowledgeable about the field, and although he wants to be converted, he argues only from first principles, and only from the evidence. Similarly, at no point did I think he set up a straw man, or tackled the opposing issues in a different manner. I particularly admired the way he avoided polarising issues -- "All models are lies." -- dismissing them as unhelpful in his current investigation. (He points out that much of the fire in the XP debate has resulted from the use of deliberately polarised opinions as a unambiguous goad to further debate within the XP community. Fine within the gang, inflammatory outside.)

The structuring of the book is of particular interest -- the argument could easily sprawl, but is restrained into very short sub-sections, with each section sporting a clear list of summary bullets. As much as is practical, each challenge to a tenet or practice of XP is discussed independently. (This comes across as simple and straight-forward and you may wonder why I even mention it, but I think it's a fine piece of editing and worthy of praise.)

The sections of the book that I enjoyed most were those dealing with the SmallTalk culture that XP grew out of -- he presents an interesting analysis of why XP works within that environment but discusses how that environment is NOT typical of most development. I have some bias here due to my own experience, see below, but had to agree strongly with his contention that XP is weakest when it comes to team resourcing, and the on-site customer. In particular, he argues that while XP restores dignity and human rights to the programming team, it does so at the expense of the poor frazzled customer. Similarly, he argues that the pre-conditions for XP, in terms of the programming staff, are so high that almost any methodology could be made to succeed with that team.

Don't get the impression that this is a negative work -- it's not. Most of XP emerges intact, and I felt that the author genuinely wanted only to restrain people from adopting XP in inappropriate situations -- not to persuade people to avoid XP. In doing so, he actually protects XP from bad press due to teams failing when trying to adopt XP, them blaming XP itself, rather than their own inappropriate circumstances.

What's bad?

Er, not much. He sort of pulls in some material re Open Source early on, but fails to particularly build on that comparison, moving instead to a comparison of Agile methods in closed source circumstances. This is a feeble objection -- but really, it's all I've got this time!

Anything to declare?

I should probably give a quick sketch of my background before I finish as I am slightly biased -- most of my work has been in the telecommunications sector, where I either worked with a large code-base (legacy) or a large, distributed team (new development). In no way was I working in the XP style, although I became test-infected easily I found it difficult to even imagine how to apply some of the XP practices to my workplace.

XP changed the way I code and work, for the better, but in a large environment, with no contracting customer, and few experts on a complex domain involving simultaneous hardware development I couldn't see any way to do XP. I'm not saying what we did was good, I'll be the first to admit it was broken (hmm, and I'm unemployed now, time to go and think about cause and effect!). The key assumptions of pure XP just didn't fit the industry I've seen most of. (To be fair, McBreen would have called these projects "systems engineering" and placed them outside of the discussion.) Now that I'm seeking employment again, I'm a lot more aware of methodologies and am much keener to work within an Agile framework, as I believe that all of the methodologies, XP included, offer a much better way of developing software. However, the key point remains -- XP cannot be applied everywhere.

Lastly, I should make it clear that I received a review copy of this title from the publisher and did not pay for it. I paid for my own copy of Software Craftsmanship.

You can purchase Questioning Extreme Programming from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

Re:Working in pairs is a bad idea

[crazyphilman]

Didn't this get addressed in the old file "The Tao of Programming"?

(to paraphrase, since I don't have the original text handy):

A manager approached a programmer and asked how long it would take to build an important project. The programmer said, "Six months."

The manager said, "that's too long. What if I assigned two programmers to it?"

"Then the project will take a year."

"That's terrible. I'll give you four programmers!"

"Two years."

"Aigh! I'll give you a hundred!"

"Then the program will never be completed."

Dig?

Re:Working in pairs is a bad idea

[thasmudyan]

(I can see why you posted AC, because you would likely have been modded down.) Anyway, your observation matches my experience: at our shop we tried pair programming and it not only resulted in a fair slowdown, there were also other effects that we noticed:
- the pairs spent a lot of time talking to each other about off-topic things.
- it lead to the formation of "expertise" domains, where one developer would do a specific task because he was treated as the expert, when the other just sat nearby and stared.
- the number of bugs was not significantly reduced
- bad design was not eliminated

So what we do know is, we build small groups of programmers and give them a modular task. They can do as many meetings and talking and planning as they like, but then they distribute the work amongst themselves and get it done alone. The code is then later reviewed by a "review partner" from the same group. It works nicely!

Planning

[Lumpish Scholar]

Nothing beats a well orchestrated and well executed plan - i.e., a written and documented plan. If software specifications are not worth formalizing on paper - it isn't worth creating.

XP proponents agree with you.

They'd point towards "the planning game," the XP deliverable for project management. (See Planning Extreme Programming for details.)

They'd point towards "user stories" (very similar to use cases), one XP deliverable for requirements gathering.

They'd also point to projects where the requirements weren't well defined up front and changed very quickly, and the cost of maintaining detailed requirements deliverables would have slowed the project down enormously.

And what project doesn't fall in that category? It's a fantasy to think, ala classic "waterfall" development, that the requirements can be set in stone before any of the design, coding, and testing is done. Requirements grow and change as the project progresses, in ways peculiar to the "softness" of software development. (No one would start a project to build a footbridge over a creek, then halfway through want a rail bridge over the Mississippi River.)

XP may not be the best way to address this, but it's one way; and even as McBreen concludes, it's a good way for some projects. For all projects? No one's saying it is.

Re:Working in pairs is a bad idea

[Anonymous Coward]

The time that is used up when the better programmer is slowed down does not get repaid through gains in productivity of the pair. The slower programmer will simply drag the programming pair down.

I've been involved with pair programming for about 2 years now, and while we don't do full XP style development, it has been extremely beneficial. We are both expert programmers, but I do all of the typing. The benefits are the ability to have conversations about issues the moment they are encountered. Rather than coding a workaround or the first thing that comes to my mind, we discuss it and the results are spectacularly better. We decide upon a solution and code that, generally much better than what someone would come up with alone. Ideas are questioned and assumptions are challenged.

He is also able to be giving me a constant just-in-time code review. We catch dumb mistakes that the compiler wouldn't, simply by having more eyes looking at the code. When writing test cases, we both come up with test scenarios that the other hadn't considered... as a result, the test suite is far superior to what it would have been.

Additionally, working in pairs means that more than one person knows the code. This is good for lowering the "bus count". (That is, how many people can get run over a bus before your project is seriously in jeopardy.) With more people understanding the code, I can go on vacation without worry of a critical bug being discovered, and trying to tell them how to fix it from a phone halfway across the state.

Yes, it does slow us down a bit, but I wouldn't consider it a waste whatsoever. It's an incredibly bug-free, totally reliable system. That means little to no time wasted trying to figure out crashes. The time is more than made up for when you consider debugging time.

My experience with non-paired programming does not have as good of results. Even if you threw out everything else about XP, the testing and pair-programming have been the two most valueable ideas it offers.

A reveiw of the review

[CresentCityRon]

Everyone is posting about XP but I would like to address the review.

It was rather thin. I didn't see much insight into the book's ideas or even the writing style. I didn't come away with any sort of idea if I should buy this book or if it would be of use to me.

I like book reviews. I think this was more like a very short book description and summary of the table of contents. Reviews talk about a view and not just a summary.

Methodology as religion

[Aron S-T]

The problem with all software methodologies is that they are usually associated with some software guru who is trying to make big bucks as a consultant selling his brand. Also methodologies very often become religions, with the head guru being the leader of the cult. This is understandeable since most software developers depend on a song and a prayer to get their code to work. XP is a good example of both trends.

I wrote a white paper on software methodology which you can download here. I am an agnostic when it comes to methodologies, but there is alot you can learn from what's out there.

Memory Lane (was: Another fad runs its course...)

[Tablizer]

In the case of structured programming, I wouldn't say that it's come and gone; rather that it has simply been pretty generally adopted, and hence isn't trumpeted or bragged about.

Some IT fads eventually work and some don't. In my observation the success ratio is roughly about 1-in-5. Structured programming (no Goto's) and relational databases seem to have pretty well "stuck". OOP and Client/server are kind of in the same middle-land boat. They seem to work okay in some niches but not others. I remember the tail-end of the CASE craze. Then there was the Expert System craze. In the late 1970's was the "extreme" top-down craze, which was actually the extremist end of "structured". It later mellowed out to being at a routine or module level instead of application-wide. Thus, even good things can be taken too far.

Now we have web-applications, web services, UML, and XML-everywhere fads that are having their try. (Curiously, I just saw part of an article that claimed very UML-like approaches were tried, and failed, in the 1970's IIRC.) Most of these I would not personally bet on lasting in their current form, beyond minor roles. I think B-to-B web applications will be replaced by "remote HTTP-friendly GUI" technology of some sort (like XWT or SCGUI).

XP from a survivor...

[Sparks23]

I joined a company a year and a half ago, becoming part of an Extreme Programming development team. And I was pretty skeptical at first.

I mean, here I am, coming into a compiler-and-assembler design team and I'm being told that we have to /share/ computers? That we have /weekly/ design meetings? And all sorts of other strange and unfamiliar things.

But to my surprise, it worked. We each had our own 'personal space' computer for e-mail and everything else, and when we were working we went into our lab as a group. We'd look at the board of current 'stories' (overall larger tasks) and pick a 'task card' from the story. (Literally, taking an index card off the corkboard and clipping it into a little holder by our computer.) And then you sat down at a computer with two monitors, two keyboards and two mice and you pair programmed. If there were an odd number of people, you had to have someone else go over your code with you before you checked it in. All code had to have tests written for it /before/ the code was written, and the code couldn't be checked in until the tests -- and all existing tests -- passed.

And to my surprise, it worked. With two people looking at code, the little mistakes were harder to make. Design problems were more easily tackled. Because we all shifted around and changed partners a lot, we all learned all the areas of the code and had a better understanding of the system as a whole. The way tasks and suchnot were partitioned out worked very well. Meetings didn't interrupt the flow of things, because almost all meetings /had/ to be held standing up. (I.e., everyone starts to get tired, and when people start sitting down a meeting had to end.) We took a fifteen minute break every two hours to keep from getting into fugue states, and because of that productivity we were never working overtime (thus keeping us from burning out). It was one of the most rewarding development experiences I've ever had.

BUT...this also became a story in how Extreme Programming can /fail/. We lost support within the rest of the company for the XP process, and that hurt a lot; XP relies on getting weekly or bi-weekly feedback from your 'customers' (i.e. the target for your project), as well as having them set what the more important tasks for the next two weeks were. Suddenly, we were having to plan out things six months in advance and operate in a vaccuum, which really hampered the Extreme Programming method. In addition, our team was expanded...and we learned the difficult way that while XP works really well for an 8-person team or so, it does /not/ mesh well with larger teams.

So, I actually would probably agree with the book's assessment; XP is well-suited for certain situations (small team, active customer feedback and support, quick dev cycles), but will fail miserably (and I do mean miserably, as in ruining the morale of the team) if you do not have the full support infrastructure.

Admittedly, some of XP's practices -- tests written before code, meetings held standing up to keep them from dragging on indefinitely -- work pretty well even outside XP. But the system as a whole works well only within a specific target range.