Questioning Extreme Programming

David Kennedy writes with the following review of Pete McBreen's new Addison-Wesley title, Questioning Extreme Programming. It's one of the few books which casts a questioning look at the buzzwords and concepts of Extreme Programming; read on to see how well XP fares under McBreen's examination. Questioning Extreme Programming author Pete McBreen pages 186 publisher Addison-Wesley rating 9 reviewer David Kennedy ISBN 0201844575 summary A critical but fair re-examination of all of XP.

In short:

This is bound to a controversial and widely read title -- it is a critical but fair re-examination of all of XP's assumptions and core practices. It provides a much needed comparison of XP with other, less popular, methodologies. Overall, XP emerges favourably, with one serious caveat -- the author concludes that XP is only suitable for a very narrow range of projects, and those that can fulfill all requirements probably stand a significant chance of succeeding using any of the similar methodologies. As with programming languages, there is no silver bullet -- put XP in your methodology toolbox, know when it is appropriate and only use it then.

A couple of interesting specifics:

• The author specifically argues, and I agree, that what the XP literature badly needs is a DSDM 'suitability filter' to advise project leaders as to whether XP is for them.
• In the preface, Kent Beck describes the On-Site Customer role as a team, and not an individual role.

Overall, this is an intriguing and well-argued book. It's based on solid, reasoned arguments, with conservative conclusions drawn based on the evidence. It's a must-read for those interested in XP and other Agile methodologies. The Extreme Programming Series

This is the 8th title in The Extreme Programming (XP) Series from Addison-Wesley, surely the most widely read series on a software methodology ever! (If that isn't achievement enough, XP also made testing sexy again. I hear that accountancy firms are looking for Kent Beck to do Public Relations work ...) For those of you who have been living under a rock for the past couple of years the previous titles are:

• Extreme Programming Explained (Beck)
• Planning Extreme Programming (Beck & Fowler)
• Extreme Programming Installed (Jeffries et al)
• Extreme Programming Examined (Succi & Marchesi)
• Extreme Programming in Practice (Newkirk & Martin)
• Extreme Programming Explored (Wake)
• Extreme Programming Applied (Auer & Miller)

This new addition to the XP library feature a foreword by Kent Beck. This is important as many of the reactionary XP fan-club will not like this book -- it challenges XP, and I am delighted to see this title as part of the series. Beck admits he doesn't agree with McBreen's conclusions, but asks you to read the book and decide for yourself, conceding that the arguments are fair and reasoned. I come from a scientific background and distrust anything except wide open debate, a position many who welcome XP will surely agree with. A book challenging XP can only help persuade people to give it a go, by addressing their fears and explaining how to manage any real risk.

Check your sources

Pete McBreen is the author of the excellent Software Craftmanship: The new imperative, a 2002 title from Addison-Wesley. In it he outlines an alternative to the software factory model behind much of traditional software engineering thinking. He proposes a collaborative model with small teams, where the software coder is seen as a craftsman in constant dialogue with the customer. Sound familiar? It should, this is a cross between a methodology and book of advice for career programmers, and fits squarely within the values proposed by the Agile Alliance, and arguably popularised most by XP.

I highly recommend Software Craftmanship, and can think of few authors who are as well positioned to give an analysis of XP as it currently stands.

What is the book about?

Questioning Extreme Programming does just that -- it's the first title in the series to take a skeptical look at the rise of this popular methodology and question some of the key assumptions. Arguably there was material like this buried in Extreme Programming Examined, but it suffered from a fragmented, detailed view, due to it being a bound set of conference papers.

The author tackles XP in a fair way -- he's extremely excited by the methodology, and it's clearly in accord with his own preferred approach. What he does is tackle each of the XP tenets in turn, questioning their validity, and then moves on the compare XP to other Agile methodologies and asks how XP stacks up against the competition. He also has a look at the common mis-conceptions (from both sides) about XP, and tackles the key arguments against its adoption in the same way.

Let's have a look at the contents to give you an idea of the structure:

• Introduction.
  XP: Hype or HyperProductive?
• What is a methodology?
  
  • What do methodologies optimise?
  • What are XP projects scared of?
  • What do other methodologies consider important?
  • What is important for your projects?
• Questioning the Core XP Practices
  
  • Planning incremental development
  • Truly incremental development
  • Are we done yet?
  • Working at this intensity is hard
  • Is that all there is to XP?
• Questioning XP concepts
  
  • The source code is the design
  • Test first development
  • Large-scale XP
  • Is the cost of change really low?
  • Setting the dials on ten
  • Requirements: Documentation or a conversation?
  • Is oral documentation enough?
  • Playing to win?
• Understanding the XP community
  
  • ReallyStrangeSayings
  • Feel the hostility; experience the joy
  • Transitioning away from XP
• Your choice
  
  • Is XP for you?
  • Do you have a suitable first project?

What's good?

The whole thing. Let's start with the basics, the high standards of the XP series are maintained, with flawless editing and layout. Moving on, the author's position is admirably neutral -- he is knowledgeable about the field, and although he wants to be converted, he argues only from first principles, and only from the evidence. Similarly, at no point did I think he set up a straw man, or tackled the opposing issues in a different manner. I particularly admired the way he avoided polarising issues -- "All models are lies." -- dismissing them as unhelpful in his current investigation. (He points out that much of the fire in the XP debate has resulted from the use of deliberately polarised opinions as a unambiguous goad to further debate within the XP community. Fine within the gang, inflammatory outside.)

The structuring of the book is of particular interest -- the argument could easily sprawl, but is restrained into very short sub-sections, with each section sporting a clear list of summary bullets. As much as is practical, each challenge to a tenet or practice of XP is discussed independently. (This comes across as simple and straight-forward and you may wonder why I even mention it, but I think it's a fine piece of editing and worthy of praise.)

The sections of the book that I enjoyed most were those dealing with the SmallTalk culture that XP grew out of -- he presents an interesting analysis of why XP works within that environment but discusses how that environment is NOT typical of most development. I have some bias here due to my own experience, see below, but had to agree strongly with his contention that XP is weakest when it comes to team resourcing, and the on-site customer. In particular, he argues that while XP restores dignity and human rights to the programming team, it does so at the expense of the poor frazzled customer. Similarly, he argues that the pre-conditions for XP, in terms of the programming staff, are so high that almost any methodology could be made to succeed with that team.

Don't get the impression that this is a negative work -- it's not. Most of XP emerges intact, and I felt that the author genuinely wanted only to restrain people from adopting XP in inappropriate situations -- not to persuade people to avoid XP. In doing so, he actually protects XP from bad press due to teams failing when trying to adopt XP, them blaming XP itself, rather than their own inappropriate circumstances.

What's bad?

Er, not much. He sort of pulls in some material re Open Source early on, but fails to particularly build on that comparison, moving instead to a comparison of Agile methods in closed source circumstances. This is a feeble objection -- but really, it's all I've got this time!

Anything to declare?

I should probably give a quick sketch of my background before I finish as I am slightly biased -- most of my work has been in the telecommunications sector, where I either worked with a large code-base (legacy) or a large, distributed team (new development). In no way was I working in the XP style, although I became test-infected easily I found it difficult to even imagine how to apply some of the XP practices to my workplace.

XP changed the way I code and work, for the better, but in a large environment, with no contracting customer, and few experts on a complex domain involving simultaneous hardware development I couldn't see any way to do XP. I'm not saying what we did was good, I'll be the first to admit it was broken (hmm, and I'm unemployed now, time to go and think about cause and effect!). The key assumptions of pure XP just didn't fit the industry I've seen most of. (To be fair, McBreen would have called these projects "systems engineering" and placed them outside of the discussion.) Now that I'm seeking employment again, I'm a lot more aware of methodologies and am much keener to work within an Agile framework, as I believe that all of the methodologies, XP included, offer a much better way of developing software. However, the key point remains -- XP cannot be applied everywhere.

Lastly, I should make it clear that I received a review copy of this title from the publisher and did not pay for it. I paid for my own copy of Software Craftsmanship.

You can purchase Questioning Extreme Programming from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

Test First Coding

[osullish]

Where I work we've recently adopted Test First Design, Initially we all hated it as the learning curve is steep, and it doesn't really make sense the first few times round.

Initially we took an old project and applied Unit testing to it, which really turned us off the concept. But once we started a new project, and wrote the test cases before our code it suddenly made sense!

Now its instinctive for us to write the test cases first, It also allows cuts down on refactoring later as all our code is at its simplest level always. If our code is changed in the future, any problems will be identified straight away.

I'm sure in the future we'll attempt the other concepts behind XP

extreme programming mailing list

[selkirk]

XP has an active and interesting mailing list.

More information about XP can be found here or here.

This is a very valid point, actually...

[Dot.Com.CEO]

Check this out.

Re:Pardon my ignorance but...

[wantedman]

heh, XP is sorta like a formulated Build and Fix model...The REALLY basic description is below

1. Divide your project into a bunch of moduals
2. Make teams of 2 programmers and assign each modual. Program it without worrying too much about how things fit together. Both members work on the same code(2 people to 1 computer)
3. Fit the moduals together

Step 3 is the limiting facter, Large projects suffer from too many hacks, and well defined projects suffer because of unneeded hacks...

It works very well for small projects(under a year), where the requirements change a lot.

Pair programming is not Extreme Programming

[Lumpish Scholar]

XP lasted for about a week at a client site, before I got fed up with a foul smelling twit sharing my cubicle with me and quit.

Pair programming is not Extreme Programming. Perhaps XP cannot be done without pair programming (or perhaps it can); certainly pair programming can be done without XP. Some people have been doing it for decades. (I have, too, in a non-XP context.)

I've not met someone who can keep up with me when writing code. And really, that's not the time for 2 heads.

I strongly recommend the Williams and Kessler book on pair programming, which talks about this. (Maybe I should review it?)

it does work, but everything has to be in place

[matticus]

I worked as an intern this summer at a small company in Grand Rapids, MI. We were complete XP to the core. I was very nervous coming in because I have heard about how difficult XP is to do right. However, with our company, everything worked. Here are what I think a company needs for XP:

• FOOD in the programming area. So what if it'll mess the keyboards up. A hungry programmer is a bad programmer. Always have snacks. If your coders are getting fat, give them something healthy. Always have food.
• People who get along and have similar levels of experience. Pairing with a person who knows much more or less than you do is counterproductive. However, pairing with someone who knows their stuff like you to is great.
• Smart boss/project manager. This person won't do much of the coding, except pair when needed, but the team really needs someone who knows their stuff to be the leader.
• Clients who understand what XP is. If they do, they will be incredibly happy with your work. If they don't, they will be very confused.

We used these four concepts and since the code we put out was so much better than anything our clients had ever paid for, we had more work than we knew what to do with. It was incredible.

Oh-and Boss, take the guys out for a beer at lunchtime every once in a while, and encourage a 5 minute break every hour or so. A happy coder is a good coder.

Re:Extremely uninterested

[truth_revealed]

When's the last time a customer told you on day 1 exactly what the program should do, how it should work, what features it should have -- and then never changed their minds once?

It sounds like you are not managing your customer's expectations very well. I find that a formal legally binding contract with a customer (complete with a product specification) highlights the importance of advance planning and gets the customer involved with the process at an earlier stage to avoid these last minute 'oversights'. When the customer knows there will be a financial penalty imposed when changing the plan (and breaking the contract) they will make damn sure that it is spec'ed out well to begin with. Projects without plans are doomed to fail or cost several times what they were budgetted for. Advance planning helps both you and your customer in the long run. Nevermind the fact that in the absence of a formal plan your customer never feels like the product is completed.

Give pair programming a try.

[bharlan]

Here are some comments on pair programming I put together for coworkers.

What are the basic mechanics? Usually, one programmer is typing and thinking out loud about what needs to be done. The other sits and offers suggestions. Sometimes, the keyboard changes hands.

The first few times are not typical. You will spend time discovering tricks with editors and discussing personal work habits. You will find it exhausting, and neither will be unable to type for more than half a day each. Remember to pass the keyboard back and forth, and keep your paired days short.

After the novelty has worn off, you find that every partner is different. Nevertheless, the benefits seem not to depend very much on whom you work with.

What is one guaranteed benefit? All code written by two people is at least understandable to two people. Two people can maintain it, so others are more likely the find the code maintainable as well. Upgrades and bug fixing will always consume more of your time than the negligible first draft of a program. If your code can't be maintained, then you really cannot afford to ship it, no matter how useful the functionality.

Better, the two of you are thinking about the code in different ways. While the typer is thinking about one screenful of code, the companion is thinking about the effects on other code. Maybe this bit of functionality belongs elsewhere. Maybe this information can be encapsulated. Maybe we're making a bad assumption here. You catch many opportunities to simplify your code and improve flexibility.

It's hard to think about the big picture and the details at exactly the same moment. While you're thinking about one, you mess up the other. With twice the brain capacity, one of you can focus on the details, while the other checks the context and adjusts priorities.

I also find that we get more than twice the work done of either of us working alone. A speed-up isn't guaranteed, or even necessary, but there are good reasons why it happens.

Much of the time you spend in front of a terminal is wasted by context switching. "OK. Where was I?" You decide something must be fixed or refactored before you can make any progress. You spend twenty minutes making the extra change and you lose the thread. Your original idea goes cold. Or you postpone and never get back to the necessary refactoring. Worse, you may spend an hour restructuring code, then realize it was not necessary after all. There was a simpler way to solve the same problem, or you misunderstood the problem. Two people will stop each other from wasting many hours this way. One or two such insights can justify the entire session.

Pair programming is also constant code review. You get to discuss and and explain the design at the most optimum moment, when it is being written. Alone, you might get ninety-five percent of a design perfect, so that no one could possibly improve on it. But that stupid five percent could have been avoided by almost anyone you pull in from the hallway. You could have avoided it yourself on Tuesday, but today is Thursday, and you were using a different part of your brain. Two people are less likely to have the same blind spots.

Each programmer has a high tolerance for complexity in certain types of code. Certain strange idioms are second nature to you, but to no one else. You won't know for sure, unless someone is sitting next to you asking "What the heck is that?" Then you'll break the one clever line into several readable lines, and move on.

You'll like your work better, others will like your work better, and you'll get more done. That should be enough reason to give pair programming a try. Maybe it won't work for everyone, or everyday. Don't force yourself or others to participate. You must be relaxed and receptive. As with any new skill, it works better with practice. Try with different partners and different problems. If you find yourself staring blankly at the screen, then surely you can't do any worse with someone else in the room. Try it right then.

What if your programmers work in different cities? Try using an instant messenger, such as IRC, Jabber, or Yahoo's. These allow you to pass files and snippets of code back and forth. You can archive your conversations.

Embrace Change

[Dog and Pony]

http://c2.com/cgi/wiki?EmbraceChange

That is something that should be read by all "Waterfall" developers like you. It is very useful, even if you don't change your opinion.

Damn. IHBT. :)

Re:Extremely uninterested

[Dom2]

Test First is a godsend!

I've started doing test first for all my programming now and I've found that it helps in several areas:

* Forces you to think upfront about what you're doing and helps catch stupid errors right there.

* Tends you towards a better, more separated design. This is a natural consequence of testing.

-Dom

Another review from the Extreme Programming camp

[Nikos]

Here's another review from the Extreme Programming camp that takes the book to task on several issues.

Ken Causey

XP explained

[Anonymous Coward]

It is a management tool made to look like a geek toy.

The name attracts geeks: they like 'extreme', they like programming, so XP must be great, right?

Wrong...

In XP, the idea is that two programmers are constantly working together. One of the two types code, the other watches his every move. "Don't forget that semicolon!" "No, don't do it like that!" "You don't need to do that right now!".

Having a slavedriver who puts the whip on you whenever you stare out the window (or browse /.) may sound nice to management, but it doesn't make for a nice working environment - in fact I'd say it drives people very quickly to being burned out.

Another mantra of XP is "never write anything you do not need right now". As with top-down or bottom-up programming, this tries to force the way a programmer thinks into a narrow pattern.

Personally I've found that I'm at my most productive when I have a good flow going. That may mean writing some support classes first (and completely) and then building something on top, or it may mean going chaotically all over the place adding bits, but it never means doing things precisely in one order.

I've also noticed that it is not possible for two people to sync their 'flows' when they're coding. Yeah, bring forth the lame jokes...

Re:XP explained

[theOnlyTPC]

Try this link for a well laid-out explanation of what XP is and how it {does | doesn't} work.

Re:XP explained

[Rolker]

This page explains what XP is, yet after reading it, I'm more confused...
They must have written this using the eXtreme Writing technique.

Re:Pardon my ignorance but...

[angel'o'sphere]

Someone with three questions like this:
I read the whole article trying to figure out what he was on about. eXtreme Programming? Can anyone give a definition of it? What exactly is it that XP is, and why does this book challenge it? How did XP change the way he codes and works? What is this "popular methodology?" Because that's what I'm most interested in. The author of the above book review assumes everyone has read the book, or knows all about XP. gets modded as Insightfull?

He is not "cluefull" enough to make a google search?

Anyway: XP is set of about 12 "best habits" you should follow in programming.

Whereas other software developmnet processes focus on a lot of work wich is not programming, XP focuses on doing a lot of programming and incorporate the other work: analizis, design, as good as possible into the programming "phase/workflow".

The most known "buzz words" are programming in pairs and unit tests.

But: there are 12 :-)

-- Have your programming team at the customer side, or the customer in your house.

-- refactor, if you see a strange couple of lines of code, refactor it

-- only program what you imediatly need, Saying: "You ain't wanna use it". So don't program overabstraced "framework like" code as it likely will get refactored away or won't be needed anyway

-- use storry boards, similar to use cases, well in fact similar to Scenarios, but a lot of literatue mixes up the difference between Use Case and Scenario

-- imediat feed back: make a story board with the customer, see above, prepare a unit test for it, see above, code it, refactor every existing code in your way to support your new story, run the unit tests of teh old story boards to be sure nothing broke during refactoring, run your new unit test to your new story board, show the result to the customer, adjust storyboard and repeat if needed

-- plan the architecture, "play your way through it", buzz word: "planning game", e.g. use CRC cards

-- short releases, plan a day per "story board" or major feature

-- collective ownership, all code belomngs to the team, ther eis nothing like YOUR file/class or code

-- have a system metaphor, thats an architecture pattern, basicly have an other system you can compare your current work with

-- learn and communicate

Well, read a book, get a consultant and DO it.

You won't be able determine if it WORKS or not by trying it half hearted for 3 weeks.

angel'o'sphere

Re:sacrificial lamb

[almeida]

I've never heard of it either. Google found this for me. You might be interested in going straight to the What is Extreme Programming page. Based on my quick look through their pages, it seems like just another buzz word for something that isn't really exciting (ooooh, listening to customers and testing throughout the development cycle!? How revolutionary!)

Re:I just wish it weren't called that

[rossifer]

XP is really one of a larger group of "Agile" methodoligies.

The basic tenet of agile development is to throw out the big waterfall and shrink the development cycle down to lots of tiny little cycles. The big benefit here is to eliminate the "convergence" system integration step. After a certain point, the product is always ready to ship so the cost of "premature convergence" is eliminated and you get lots and lots of opportunities to reorganize requirements. You'll have a list of a few hundred things to do, pick the most important two or three, get them done, put it in front of customers (who will give you more things to do), repeat.

Feature creep is automatically managed because you also get lots of chances to see how well your estimation is working and when marketing makes new demands, they get to help decide where those new demands get put into the todo list. When someone has a chance to immediately understand the impact of a new feature, it makes it much easier to understand whether or not this is really a "gotta have" or not.

XP won't make sense unless you buy books and push them under people's noses, and even then, some won't like it because it appears to demand a complete upheaval of what you're currently doing. Be sure to emphasize that many of the practices can improve your system when applied individually. You don't need the whole thing to be better.

If you get "Rapid Development" by Steve McConnell, an older authority on development processes, most of the processes in the chapters on dev process are agile. Evolving prototype, etc. There are lots of ways to present this that won't hit the buttons of change averse people.

XP itself really is fairly extreme. There are lots of ways to improve a process that don't have to toally disrupt a culture but which can really improve the way things get done.

Regards,
Ross

Re:sacrificial lamb

[burkedrane]

Extreme Programming has little to do with computer science, so there is no reason why anyone in academia would find it all that interesting. It is, however, eXtremely pertinent to the science of software development, so those of us engineering in extreme environments, like games where the design and requirements change on a weekly basis, find its essential ideas and lexicon pretty cool. But not as cool as Adaptive or Agile development.

Re:sacrificial lamb

[customiser]

You might find Pair Programming, 40-hour week and continuous integration a little more exciting. I'm not saying that it works or even that you should have heard of it, but a "quick look" is definitely not a good reason for rejecting it.

Personally I find that some of its suggested methods are right on the spot while others a bit on the idealistic side. Still, I would like to try it and see first-hand how it works in a proper business environment.

Re:Working in pairs is a bad idea

[frank_adrian314159]

The time that is used up when the better programmer is slowed down does not get repaid through gains in productivity of the pair.

Actually, check out this paper for an experimental that shows that the extra time you spend in pairing will increase the quality of the resulting code. So you have a choice: program alone and get buggier code faster, or pair and get less buggy code a bit more slowly (but the quality gains are larger than the speed loss).

Re:sacrificial lamb

[Grendel Drago]

This is an actual explanation of the rules and practices of XP, not just a page full of buzzwords. It seems to center around extensive testing and customer feedback.

--grendel drago

Re:TDD book

[Anonymous Coward]

The title is "Test Driven Development: By Example"
It was just published November 8, 2002 which explains why a lot of us
who are usually on top of things haven't heard of it yet.

Re:please move along

[chromatic]

When you write your tests first, you get several benefits:

• a comprehensive test suite, built incrementally
• immediate feedback on the state of the system
• confidence that the test works (it should fail before you add the feature) and that the feature works (it should pass after you add the feature)
• very quick debugging (if the tests passed a minute ago and don't pass now, whatever you just did broke them)
• better API design, as you have to use the interface immediately
• simpler code, as simpler code is easier to test
• easier refactorings, as the code is already simple and tested

Bugs do happen in test code occasionally, but if you write small tests and small bits of code, it's really easy to catch them. It takes some work to get into a test-driven mindset, but I haven't met anyone who's tried it without realizing the benefits.