Extreme Programming Explained

While I've been sitting on this for a little while, chromatic has been patient. Yes, it's his review of Kent Beck's Extreme Programming Explained. The publisher is Addison-Wesley, and the book is for all those people out there who need to do programming but don't have time to do the engineering phase. Interesting book - click thru to read more. Extreme Programming Explained author Kent Beck pages 179 publisher Addison Wesley, 09/1999 rating 7/10 reviewer chromatic ISBN 0201616416 summary Extreme Programming Explained explains the virtues of the Extreme Programmer and shows you how to develop them.

The Hook

Want to write better code? How about working less overtime, getting along with your team better, meeting customer demands more quickly and accurately, spending less money, and having more fun?

Extreme Programming may be for you.

Be prepared to make some adjustments and sacrifices. Individual code ownership? Gone. Programming for the future? Slow down, cowboy. Working on your own? Grab a partner and dance.

What's the Scoop?

Extreme Programming is a way to improve software development by focusing on what really matters. If it will cost you $50,000 to implement a feature now that may not be used for two years, and it will cost you $55,000 to implement it in two years, hold off. If running test suites is good, write tests for every significant piece of the system. If multiple pairs of eyes make bugs shallower, program in pairs. If you enjoy meeting deadlines (and not working your fingers to the bone every night for weeks to do so), make shorter deadlines.

It sounds simple, even deceptively so. It may also set your teeth on edge at first.

Imagine that your customer has the time and the manpower to send a representative to sit with your programming team. He is actively involved in the design, writing 'stories' about how the system works for the end users. Every morning and afternoon, your programmers meet to decide which tasks to tackle, and they pair off, sharing one computer between them. One person codes and the other watches, and they switch off as necessary.

With every change to the system, the previous tests are rerun until they work perfectly, and new tests are added to test new functionality. Changes are not commited until all tests run successfully.

Releases are started early (six months, for a big programming project) and continue quickly after that (every couple of months). With a customer sitting in with the programmers, feedback can be instantaneous. The initial investment pays off quickly, while expenses are spread out over a greater period of time.

With no one owning a particular section of code, and with everyone working with different partners from day to day, everyone should have a good overview of the system as a whole. This can lead to better programming, from less bugs to very quick refactoring. New programmers can also be brought in and up to speed much more quickly.

What's to Like?

The book is clear and readable -- even funny. Chapters are short and to the point. Beck uses the metaphor of driving to bring his point across. (Driving is not about pointing in the right direction and maintaining that course, it's about making slight corrections all of the time.)

The bibliography is a great place to find some classic works (including books by Brooks and Knuth and even the movie 'The Princess Bride' -- no, really!).

Extreme Programming itself has a lot of promise. Some of the principles (programming for today, releasing early and often, peer review, community code ownership) fit in pretty well with open source/free software. Some of the other ones would be nice to see....

What Might Annoy You?

It's not clear where Extreme Programming fails. To the author's credit, he mentions this and gives some guidelines, but the choice and the implementation ultimately rest with the managers and bean counters. There will be some resistance at first, but Beck's enthusiasm is infectious and his clarity of explanation might be enough to overcome it.

The Wrapup

If you're a member of or a manager of a moderate programming team, you ought to read this book. It will go nicely on the shelf next to "The Mythical-Man Month". If you're curious about new ways to look at programming (especially in a group), you'll want to pick it up.

Purchase this book at fatbrain.

Table of Contents

1. The Problem
   1. Risk: The Basic Problem
   2. A Development Episode
   3. Economics of Software Development
   4. Four Variables
   5. Cost of Change
   6. Learning to Drive
   7. Four Values
   8. Basic Principles
   9. Back to Basics
2. The Solution
   1. Quick Overview
   2. How Could This Work?
   3. Management Strategy
   4. Facilities Strategy
   5. Splitting Business and Technical Responsibility
   6. Planning Strategy
   7. Development Strategy
   8. Design Strategy
   9. Testing Strategy
3. Implementing XP
   1. Adopting XP
   2. Retrofitting XP
   3. Lifecycle of an Ideal XP Project
   4. Roles for People
   5. 20-80 Rule
   6. What Makes XP Hard
   7. When You Shouldn't Try XP
   8. XP at Work
   9. Conclusion

Comments

I much rather like sole systems

[Rurik]

Get praise for the work you do right, and be ashamed for your mistakes. Community code, as such, doesn't teach each programmer his mistakes, IMO. If you feel that you have the syntax correct, and convince your partner in that, what if the next day a new pair of coders happen upon it and tear it to pieces. Granted, it was returning null values in the first place, but if the work is already done and posted up, you can look over it, but do you really learn from it?

The way I like it, each person on their own, knowing what to work towards. On routine schedules, a manager(s) will look over everyone's code and test it together. If a problem is found, the manager will discuss it with the coder, and give help on fixing it.
This may be less time effective, but you know that the skills of each programmer are rising, rather than looking at the overall skill of the community (the strong make up for the weak).

Re:I much rather like sole systems

[pieguy]

Actually, it works the opposite. Programmers who only work on their own never learn better programming techniques and algorithms beyond what they can think up just by themselves. Also, beginners don't have their bad coding practices corrected if a more knowledgeable programmer doesn't mentor them. The projects where I've worked on teams have always had better results than the projects where everyone worked independently. ---------------------------------------------

Re:I much rather like sole systems

[sjames]

The way I like it, each person on their own, knowing what to work towards.

Personally, I like a sort of hybred system. Each person works on his own subsystem as you say, and periodically, programmers review each other's code. Those reviews need to be frequent near the beginning when interfaces are being established. That is a good time to verify correct interpretations, and possably to identify design flaws or inefficiencies befor things get too far along.

It is also useful to sub-divide a large project so that each person gets SEVERAL pieces to work on. That way, it's not so painful to re-apportion later in the project when the time estimates have some real world feedback.

Re:I much rather like sole systems

[samantha]

I agree two pairs of eyes are better than one in code reviews, walkthroughs, debugging sessions and such. I emphatically do not agree that two pairs are better at design and code construction time. Program construction is NOT a team sport in my 20 years of experience and it is damn frustrating when management takes it into their pointy little heads to make it such. May as well do tag team symphony writing or fine art. I especially resent and am fed up with management notions of design by committee. I've wasted months in such attempts to "get consensus". It does not work. Let those who have notions of how to design a subsystem go do so and then review it. Don't try to review it while it is still a bunch of abstractions everyone would like to stick a ego-tag or two into.

Re:I much rather like sole systems

[Cuthalion]

I emphatically do not agree that two pairs are better at design and code construction time

I definately have had very different experiences than you, though not in terms of corporate coding (which is what I do during the day that keeps me from coding for fun all of the time).

All of the most successful hobby projects I've worked on (and some have been very successful)have been collaborative efforts between me and another coder. We've worked together productively on all parts of projects - from design (often in a coffee shop) to implementation (less often in a coffee shop) to debugging and optimizing (never in a coffee shop).

I don't know how often this kind of thing works, though. I certainly find myself talking over design issues with my co-workers (in both "my" modules and "theirs") but that's not as symmetric.

It would be foolish to argue that this always works, or that any two coders working together will compliment each other. I'm merely writing to anecdotally point out that it can happen.

end user v. manager feedback

[Pfhreakaz0id]

I like the part about customer feedback (i.e., end user feedback). That is often hard to come by in projects I work on. The tricky part is balancing it with management feedback. A typical situation is one where the users of the software want ultimate flexibility, but management wants software that adheres to certain business rules. As in "you can't set a ship date for whatever you want, it has to be within five weeks of the order date."

BTW, all this is from the perspecitive of "business apps," which some twit said wasn't "real programming" here a while back. Whatever.

cost?

[DGregory]

With infinite resources anyone can write the most beautiful, bug-free code. I agree that 2 pairs of eyes are better than one, but if they can get that 2nd pair of eyes to go program something else, they can get more programmed in the same amount of time. These are the days of companies being shortstaffed and projects having to be done "yesterday". It all sounds great in theory (just like all the good practices you learn in college C++ class), in the real world, things don't work how you'd ideally like them to.

Re:cost?

[pieguy]

It's typical of the "penny wise; pound foolish" mentality to ignore the benefits part of two eyes and just say the costs are unaffordable. I have often identified coding errors in 30 seconds after someone has worked for three weeks on their own trying to find them. (I have also been on the other end). People who work solo often spend a lot of time figuring something out that someone else already knows.

Re:cost?

[waveman]

I have been using XP for a while, and I found that the whole is more than the sum of the parts. True, no element is entirely new, but when you put them together everything works.

Pair programming is an example. The previous poster claimed without any evidence that programming alone is more productive. I don't think it is, when you take into account that with pairs you get fewer errors, cleaner code, and better distribution of knowledge across the team. And less need for doco.

The existence of unit tests for example enables you to add features and clean up code later on. One reason for 'big design up front' is that the cost of changes later on is so high. But with the tests, you don't have to fear making changes. You can then defer adding features until you need them - which may be never. Thus unit testing enables 'just in time' coding.

Look at XP as a whole before judging it.

Re:cost?

[BinxBolling]

I still think that instead of 1+1=2 you're getting 1+1=1.5 ... more productive than just one person but still not as productive as 2 people coding separately. Yes it's faster, cleaner, better than just one person sitting there and coding their stuff by themselves, but by how much? Certainly not double time.

On the contrary, I could easily imagine pair programming leading to a 1+1=2.5 increase in productivity. One of the basic points Fred Brooks makes in The Mythical Man-Month is that there is not some intrinsic amount of effort that a software development task 'ought' take. (Indeed, this is the source of the book's title.) Not all man-hours are created equal. Of course this is obvious, since not all programmers are of equal skill. But even for a single programmer, some hours are more productive than others. He may hit upon an abstraction that drastically simplifies coding for the bulk of what his program must do, and as a result accomplish a great deal in a very short period. Or he may hit a wall, get frustrated, and go off to stew and play solitaire for an hour. Though I've never tried it, I suspect that pair programming could increase the productivity of the average man-hour sufficiently to more than make up for the halving of programmer resources: With two minds working on a problem, the chances of finding a useful abstraction increases quite a bit. And when a wall is hit, the bystander's presence will motivate the coder to either keep working and find a solution, or at least go off and work on some other part of the code that isn't affected by that wall.

Of course, this all depends on both the qualities of the two individuals, their abilities to work together, and the nature of the task at hand. If the task at hand is relatively simple (i.e read a bunch of records in from a database table, generate a neatly formatted report), that second pair of eyes won't improve productivity much. But for more complex programs, it probably will.

Ahead of my time

[Relic of the Future]

In highschool, my family got its first computer. My brother and I both were real excited, and we started teaching ourselves QBasic (don't laugh... it's all we had) and we've been programming in pairs ever since. Works incredibly well... much fewer typos, fewer logical mistakes "Hey, what's that for loop s'pose to do?" "This? It's to do the... oh wait, you're right we don't need to do that..."

Now I can't stand to program alone... drives me nuts! I go a lot slower too. Really screwed up my CS grades last year, we were explicitly not allowed to work in pairs on the code :(

"God does not play dice with the universe." -Albert Einstein

Recipe for Success

[thanz]

The 'extreme' moniker is probably just marketing hype. This is isn't really all that 'out there'. This is the way most large consulting firms and code shops work. It may not be the most efficient (or enjoyable) way to develop small apps, but it keeps the pace manageable and the schedule on track for multi-million, multi-year engagements. Specifically, to address the previous poster's comments regarding non-maintainable code: This is in fact one of the best ways to write maintainable code, because everything has has to be documented because everyone has to work on it. No one person owns the code. Again, takes some of the random joy out of coding (no more pink floyd lyrics in the header), but if you're into that, you probably don't work for a consulting firm or a code shop anyway. -thanz

Planning is good!

[Woodrow]

I have flipped through this book at the bookstore and it looks to me as if they initiate a code then analyze/fix methodology. I have no problems with doing this on a project that has flexablity and no deadline. I do have a problem with the idea that you can skip the analysis and design stages of a large complex implementation and just code then go back and fix issues as they come up. Bad habits == bad results. A little planning goes a long way. I have great experience with past projects on this matter. But like I said I only have flipped through the book. Now I have to go buy it and really understand.

Re:Planning is good!

[devphil]

There is an excellent interview with the XP creator in some recent issues of _C++ Report_ (which is itself a very respected journal). Some very good questions and answers are brought up by (IIRC) John Vlissides, the interviewer...

...whose last name I just horribly misspelled, probably. :-(

Re:Planning is good!

[johnburton]

You need to read the book.

It's not about skipping the design phase, it's about doing things differently. Don't design things until you need them because the chances are that by that time you'll have either changes the specification or else have thought of a better way.

It's about breaking down a big project into lots of tiny ones where you get a bit of functionallity designed and implemented and working and with automated tests before moving onto the next bit of functionallity. So you always know where you are and how much there is left to do. You always know what you've done so far works because you have tests to prove it. You know that you can easily change what you've done when you need to because you have automated tests which will tell you if anything has broken.

It's very strong on getting requirements and testing your code against those requirements, it just doesn't require a lot of up-front effort that will probably be wasted.

What sets it apart in my opinion from a code/fix methodology is that there are strong customer driven requirements to ensure that you build the right thing, and an emphasis on automated testing so you know that what you build works.

Re:Planning is good!

[udin]

The point that Beck makes in the book is not that planning is bad, but that there is little point in planning beyond what your customer knows he wants, especially since customers tend to change their mind. Avoid flexibility overhead where the need for that overhead is entirely speculative. He is opposed to putting in flexibility that may never be used.

Extreme programming is not, however, undisciplined. It requires a commitment on the part of the customer to say what is really important to him, and constant feedback within the development group on task estimation. It is far from 'code then analyze/fix'.

Beck advocates a project management and programming style that can react to changes in requirements quickly: very short task schedules, pair programming, refactoring to keep code clean and to the point, constant testing so that one may make changes with confidence that any errors introduced will immediately be revealed...

I've tried some of this on small personal projects (except the pair part, which is a big except) mostly to become comfortable with the constant testing part and disciplined refactoring, and it is refreshing to not try to anticipate every future eventuality, but instead to try to stay 'to the point' and keep the code clean and simple. In that respect, it is not unlike hacking; the difference is the emphasis on constant detailed testing. This gives you the confidence to rearrange existing code whenever the structure gets shaky. When it does you often find that you can quickly rearrange things in a better way (and it is mostly rearrangement, not new code), test it, and go on your merry way.

In addition to Beck's book, I recommend Fowler's book: "Refactoring: Improving the Design of Existing Code". Taken one at a time, the techniques in this book seem straightforward, even trivial, but as a group go a long way to keeping code in a maintainable, extendible state.

Extreme Programming not so extreme

[Anonymous Coward]

Maybe my shop is fairly progressive, but I didn't find Beck's techniques to be that revolutionary or controversial.

Bascially, he embraces a hacking mentality instead of rejecting it, and then looks for ways to improve this process through peer review and heavy testing.

As it stands, I think his approach is both good and bad - good in that it doesn't try to turn your world upside down and fore a new methodology on you, but bad in that it seems to deny the usefulness of some valid quality engineering practices.

Sounds Promising

[Anonymous Coward]

This method sounds like a programming corollary to the Cabal system of design. (The method used to design Half-Life.)

If it's anywhere close to being as effective, it's definitely worth a look.

Pair programming works!!!

[Da VinMan]

I learned about XP (Extreme Programming) a while back and while we don't follow all the practices Beck recommends, the pair programming has worked well for us! This should be obvious to Open Sourcer's since it's just peer review on an immediate level. The real advantage of this for us is that it produces code which is understandable to more than one person, right off the bat. This alone is a good thing. But if you do something devious like pair a programmer with a more knowledgable person, you're also getting some training in. Add to that the fact that more than one person at a time is actively designing the system, and you've got a pretty solid system in place.

This is something I highly recommend that everyone try at work. Only people with huge egos shouldn't bother, but you did remember to check that at the door, right?

Re:Pair programming works!!!

[pieguy]

Programming in pairs is one way to accomplish something I've believed in for years. Peer review of code is often considered optional. One result of non review is that Junior programmers never get their bad coding practices corrected. I wouldn't be concerned about New Year's except for all the crappy code I've been skipping over as I fix Y2K bugs.

It's not bad at all

[greendot]

After doing "normal" coding for the past 7 years, I thought I had it down pretty good. We started switching over to "XP" about 12 weeks ago and our company is on it's 2nd project using it. One difference, we split our teams out into 3 week chunks. The teams are small and we work tight. After 6 months of this, each programmer really has a feel for the system and because they work on each piece for 3 weeks, they're not leaving a mess of bugs for the next team. This "XP" thing isn't as bad as it seems. When run side by side with a "normal programming", the XP team always comes in under budget, ahead of schedule, and with fewer bugs. And the constant client feedback on each 3 week iteration is a lot better than the old school "design sessions" and "requirements gathering" phases, which left out a lot of developer/client contact.

It's NOT extreme, just practical

[Da VinMan]

I don't like the name either, it gives managers the willies and makes them think we're going to program their pet project while snowboarding. We just call it "pair programming", makes it sound boring and safe.

"You got that routine done yet?"

"Not yet, got snow in my goggles!"

"OK. hey watch out for that ..."

*aiiiieeee*

"...tree." *wince*


:+>

Library Coding Tip

[Eamonn O'Synan]

When coding a library, write just those functions you actually need to support the application, and put the tests for that functionality in the test program alongside the library source, which must run successfully before committing the code.

Then, as the functionality develops, watch for factoring opportunities and allow yourself two days to re-factor, running the identical test program successfully to ensure you haven't messed it up.

Every so often, allow four days for a major new-functionality-free re-coding.

This approach gets results quickly and above all allows re-use for the next application with just those extra functions needed being added as required.

Don't spend forever in meetings discussing library design. Let the application drive the libraries and watch as successive application coding efforts get shorter and shorter with greater re-use.


Just my opinion. Flame on.






--------------------------------------

Re:Recipe for Disaster

[bbutton]

Why would you say that? Keep in mind that the review only mentioned the briefest description of the practices involved with XP. There are a lot more.

For example, maintainability is maintained through several cooperating practices, the first being PairProgramming. Every line of code is written with two sets of eyes looking at it. There are reasons for this that I won't go into here, but you can find out about them at the URL I mention below. Next, there is the rule about OnceAndOnlyOnce. This says that code can only appear in one place in a system. If you have the urge to replicate some portion of the code, then you are obligated to refactor the system to provide for a single place for that code to live. This above all else serves to create an elegantly designed system. This leads to RefactorMercilessly, UnitTestsFirst, and a few other of the XP practices.

In the end, though, XP-produced systems tend to be exactly what the customer wants (PlanningGame), thoroughly debugged (UnitTestsFirst), and easily maintainable (OnceAndOnlyOnce et al.).

Check out the Extreme Programming Roadmap for a thorough discussion of the XP principles, and subscribe to Rational's Object Technology User's Group for a spirited discussion with many of XP's founders.

Spiral rehash?

[Mr Neutron]

This sounds much like the "sprial" lifecycle model, which has been around for some time. Risk management is very important, and with the spiral model you want to identify the risky-but-critical parts of the project and tackle those early. If you can't implement those critical areas, your project isn't going anywhere.

Comments about user feedback are right on the money. Frankly, if you're developing a mission-critical app and you *don't* have constant feedback from the users during development, you're asking for failure. Software engineers aren't domain experts, and we will screw it up we lock ourselves away from the users. Yes, you'll have management constraints but you must always present options to the user. "We can implement the 100% solution using X time/money, or we can give you the 80% solution in X/2 time/money and also have resources to attack Y and Z. You pick."

Neutron

Re:Recipe for Disaster

[The Variable Man]

I had a little chuckle when my brother introduced me to this 'new' concept. I've been working like this for a number of years. The key points to me are:

• no-one 'owns' their own code
• peer review early in the cycle
• test sooner rather than later
• end user involvement
• shared responsablity (?)

The words 'open-source' spring to mind...

Extreme Programming web site

[dsplat]

I can't believe that no one else has posted a link to the Extreme Programming web site yet. A former coworker pointed it out to me. It is worth the time to read. Some of the ideas are excellent. The one I particularly liked which I have seen used by a group I work with is coupling an automated regression test suite with a requirement that all code changes must pass that suite before they can be committed.

Too many eyes

[dmorin]

Reminds me of a time back in college when three of us were assigned projects in an AI class. One guy had no experience, so he did nothing. Two of us spent all nighters working on the code. One night, around 4am, our Lisp code wasn't working and we were stuck. We'd stared at it for hours, no luck.

"Wait a second!" shouted my partner, leaning over my shoulder and beginning to type.

"WHAT?" I asked, watching him move the cursor.

He cursored over to a piece of Lisp code, and pressed the space bar. "Sorry," he said, "That was bugging me. Now it's indented properly."

Communication and Specification

[adimarco]

One of the things I dream about is detailed specifications.

I used to dread the paperwork and meetings and monotonous never-ending corporate BS involved in "requirements gathering" and the like, but the last several jobs I've done have demonstrated their value beyond any shadow of a doubt.

People don't understand how programmers work, much less the computers they use :) Time and time again the people I write applications for turn out to have no real idea what they're asking for, or how it should work. "uuh, I donno, make it work" really doesn't do much for me as a programmer, which is why the following bit caught my eye:

Imagine that your customer has the time and the manpower to send a representative to sit with your programming team. He is actively involved in the design, writing 'stories' about how the system works for the end users

The jobs that have been done correctly, the ones where you don't spend 1/2 the time "adding features" that should have been included in the initial design specification (requiring changes to the fundamental architecture, blah blah blah), have all included this kind of a buffer between the "client" and the programming team. These are the people that hammer out 80 page requirements documents, which may at first appear to take all the fun out of it, but in fact save a whole lot of work in the long run.

"The Client" always seems to know which "little changes" require you to rewrite the whole application. "Oh, what you've done is *great*, but could you just make this *little change*..." So you have to rework the whole thing from the bottom up, break half of it in the process, and "the client" can't understand why it takes you so long...

Those kinds of headaches can be avoided *only* by getting a detailed specification before you even start...

Anthony

Re:Communication and Specification

[richieb]

One of the things I dream about is detailed specifications.

The only detailed specification that can be trusted is the source code for the system. Any other spec is out of date by the time its printed.

Don't assume that the client really knows what he wants. The system can be designed to be flexible so that adding "little changes" is not a problem.

XP advocates early releases so that the client can use the system and feed back what he learns back to the developers.

...richie

Re:Communication and Specification

[DanaL]

The only detailed specification that can be trusted is the source code for the system. Any other spec is out of date by the time its printed.

But a requirements doc shouldn't really be about implementation details. It's a describe of what the software will do and should be high-level enough to present to a non-programmer.

It's true that the client may not know what he/she wants. That's what makes the the requirements so important (at least in my mind). If a project is presented which doesn't do what the client wants and modifications will take a couple of thousand dollars, someone has to absorb that cost. When 'requirements' are decided in phone conversations, 9 out of 10 times, it's our company that absorbs the cost. With a requirements doc that the client has agreed to, we can at least make a case that what he is asking for is new features beyond what we were originally contracted to do. Better still, if the client reviews the documentation and it doesn't match what he wants, he can tell us *before* we get to far into development.

Rapid prototyping is very useful (especially for web-based stuff), but I think sometimes it gives the client the illusion that all changes a simple. He often can't understand why adding a couple of new form fields might break the database structure and take several hours. And that makes him think we're trying to gouge him.

Req docs aren't perfect, but they are far and away better than nothing at all.

Dana

Also look at "The Pragmatic Programmer" book

[Woodrow]

I am reading "The Pragmatic Programmer : From Journeyman to Master" by Hunt, Andrew and Thomas, David and they have many good idea that run parallel to XP. May be a another book someone could review. If needed I can do it if I can find the time.

Where Extreme Programming Fails!

[stephend]

While I can see it working well in a number of circumstances, most of the projects that I've worked on it wouldn't work very well.

Project type 1 where is wouldn't work is any large project, say over fifteen developers (plus management). Any system that big and complex will not be understandable by everyone. Cross-fertilisation is a nice idea, but when things get too complex you need an expert or a keeper.

I guess you could have several experts, but you come up against the cost aspects very quickly.

Project type 2 are those with varying technologies. One I worked on recently used Oracle, Unix, NT, LotusScript (Domino), NotesPump, VBScript, JScript and HTML. I would argue that you can't be an expert in all of them. I do Oracle and Unix and have an appreciation of the others, but I couldn't really add much when I reviewed the work in other teams.

Again you could switch around each team once in a while, but any good team-leader would do that if they could anyway.

And that's before you even get into the 'egoless' bits, and the fact that management like people to be responsible. How can you have a scape-goat if it's everyones fault?!

Anyway, that's enough. Nice idea, but it wouldn't work in real life.

Re:Where Extreme Programming Fails!

[William Tanksley]

You claim that you've identified an area where it wouldn't work. Now go to the XP site, and observe what project it was originally used in -- a huge, multi-technological one.

I don't think this one's a problem.

Your other objections are also pretty transparent -- they're all of the nature "That's not how we do things here! It's impossible!". Indeed, if management cops that attitude you can't do XP -- but it's an attitude problem, nothing more.

-Billy

Re:What is the "engineering phase"? :)

[JordanH]

• People have time to do the enginerring phase?!?!?

The problem is that people view engineering as a phase. Engineering should permeate every phase of engineering projects. "Engineering" is not design, but it is an important part of design.

-Jordan Henderson

Re:Recipe for Disaster

[loki7]

No, you'll have programmers who understand the whole system. Where I work we shift responsibilities around fairly frequently (although not daily). This means that everyone understands how every part of the project works. And if you know that you're handing your code off to someone else in a few weeks or months you'll write maintainable code. The biggest incentive to write maintainable code is the knowledge that if you don't you're going to piss off your co-worker who takes it over from you.

/peter

Save some cash....

[Rombuu]

This book is available at Amazon for $20.97, instead of the $29.99 Fatbrain wants for it.

If you are so into the whole patent thing, well go ahead and pay more I guess. I just saved 8 bucks, myself.

I'm with you!

[DanaL]

I keep harping on my boss to present a detailed specification to our customers for approval.

While my company does medium sized, database driven websites, I still get frustrated everytime my boss comes back and says, "Oh, the customer thought it was going to do this..."

I don't usually consider work 'coding for fun', because it's nothing like what I work on when I have spare time. It's a job for me, so I like it when it is very clearly stated what it is that the code I write should do.

Dana

Re:I'm with you!

[adimarco]

I still get frustrated everytime my boss comes back and says, "Oh, the customer thought it was going to do this..."

Abso-fscking-loutely :) The customer tells you to add a particular piece of functionality, really has no idea what they're talking about, and then complain about it when you add *exactly* what they asked for.

my company does medium sized, database driven websites

Basically what I'm doing right now. The work is essentially cookie-cutter applications (or at least the type we're doing: intranet apps), so the initial time to develop is extremely small. The client is always amazed at first at how fast you're doing it.

It's later, when they "remember" those "few little things" that they want added, and you have to change the database schema in some drastic way 6 months into the project (and thus all your libs, etc. etc.), that they can't understand why it's taking you so long. "You cranked out the first revision in 3 weeks, why'd it take you so long to make that 'little change'?"

It's *impossible* to make them understand why, or why it would have been better to have a detailed specification from the outset.

I think half the problem is that most companies charge a rather hefty fee to produce a "requirements document" (whatever buzzword they're using now). I know mine charges a rather exorbitant fee in the manys of thousands. The client never wants to pay for those kinds of things...


Anthony

"Understand the whole system"

[seebs]

That's sort of like what Brooks was arguing about, when he didn't like data hiding.

He changed his mind.

You need an overview of the whole system. Detailed knowledge has a hidden threat - it inclines you to think that the internals of other parts of the system are static, and that you can depend on them.

where it fails

[dutky]

I see a some big problems with the technique:

• no individual code ownership: this seems to suggest that the design would either be imposed from the top, which implies that coders would be treated as unspecialized labor, or that the entire team would be involved in the design of every piece of the code.
  
  Treating coders as unspecialized labor tends not to work because coders are not like factory workers but more like artisans or craftsmen. When you treat programmers like cattle they tend to resent it and deliver substandard code, at best.
  
  While getting the entire programming team involved with the design of every piece of the code might sound good, it results in design by committee, which doesn't tend to work in practice. The best designs come from inspired individuals or from very small teams of people.
  
  Finally, individual code ownership minimizes the need for communication accross the entire group (the N^2 communication problem noted by Brooks in The Mythical Man Month). By requiring a very high degreen of communication between all coders in the group, this technique is almost certain to fail for any group larger than a half dozen or so.
  
• new team members: the review asserts that "New programmers can also be brought in and up to speed much more quickly." but doesn't specify how. This is a similar problem to N^2 communication problem and was shown by Brooks to be one of the primary factors in killing large projects. The fact that the reviewer brushes this off so lightly, and that the table of contents doesn't show a chapter specifically dealing with it, suggests to me that it is not addressed.
  
  Ignoring N^2 communication issues is inexcusable in any methodology postdating The Mythical Man Month.
  
• debugging in pairs: debugging in pairs is a good idea, and should be used by any programming team that has the personnel to spare, however, it does not address the central problem of debugging. The problem with debugging is that it tends to take up a huge amount of the project time, and XP seems to want to add even more debugging to the mix (through the use of automated debugging tools). Just pairing your coders during debugging sessions won't reduce the total debugging time very much since it can't increase the number of cases covered or the range of debugging skills present by a significant amount.
  
• design to throw one away: The early release schedule seems to imply that the first coded design would be used as the production system. This is not a flaw solely in XP, but is rife in the current programming market. Many systems are built, essentailly, from the initial prototype, with no redesign. Brooks identified this as a problem back in the early seventies, but it doesn't seem to have made much difference to actual practice.
  
  A methodology that doesn't address the need to redesign the production system, however, will suffer from cost overruns and reliability problems (both of which XP is explicitly trying to avoid) due to the need to work around mistakes and compromises made when designing the prototype.
  
• unwarrented assumptions: the review seems full of unreasonable assumptions. Most programming projects don't have the latitude to set prices for each feature, or even to pick and choose which features to implement (at least not after the proposal has been made and the contract accepted). Even if you can set prices for individual features or components of the design, most features will not fall into the easy choice presented in the review. Any feature worth worrying about is likely to have effects across a large part of the design and leaving it out early one will pretty much exclude it later on as well.
  
  If XP is really dependant on some of these assumptions then it seems unlikely that it is a generally applicable methodology.
  

So, it sounds a good piece, but I don't see anything that sets it apart from standard practice or from design methodologies that have already been elaborated elsewhere. Some of the ideas are good, but others just seem to be naieve or illconceived: some even seem to smack of outright voodoo software engineering.

Re:Recipe for Disaster

[the eric conspiracy]

Rotating programmers throughout a project?? Not only is that going to lead to unmaintainable code, but you'll have programmers working in areas that they know nothing about.

Rotating programmers is also very inefficient from a manpower useage point of view. Remember Brook's Mythical Man-Month? As you add people to a task, individual productivity decreases because of communications and learning time. It can get to the point where as you add people to a project, the entire project cycle lengthens.

Each time you rotate a programmer you are going to lose the time he takes coming up to speed. He is also likely to make a lot of mistakes during this period.

Debugging Party

[Midnight Ryder]

It kinda reminds me of an event that took place one day at my house - a debugging party. I was showing off a game I had written, and was getting close to releasing a new version. A couple of my friends hadn't seen it, so, I set down and showed it off to them. I let one of the guys play it, and noticed an area where it slowed down a bit too much...

We spent hours all of us staring at the screen, finding places that the code could have been a little more efficent, etc.

It worked out great, really. I don't think I would want to program like that all the time, but, it's a great way to discuss the code all at once, what the function of a piece of code is, etc. and come up with ways to re-optimize things. I figure before my next release I'll go down to the liquor store and buy some Zima and some beer, and have another debug party :-)

Midnight Ryder

Gerald Weinberg: Psycology of Computer Programing

[Anonymous Coward]

For some reason this book reminded me of Gerald Weinberg's ancient tomb. Esp the advocacy of very small teams.

I can vouch for the team of 2 while writing code. Never actually seen it used in business (how would the bean counters justify it?), but I used it on some of my hefty college projects (like writing a pascal compiler). Felt that I was never more productive than in school coding as a team of 2. Or perhaps it was that I had a crush on my partner.....

Know there's a concept: Pair a geek guy with a beatiful woman and sit back and watch him try to impress her with his technical prowess (of course it never works, beatiful woman don't care about techy stuff, she'll use him to get promoted/ get an "A" then go out with a jock, but don't tell the geek that he'll create technical marvels in record time)

:)

Keep on keeping on.....

A personal experience...

[William Tanksley]

A foundational part of XP is the mutability of code: you write the code for what you know now, and be willing in the future to totally change it. One of the best technical books I've read recently discusses the how-tos of this: "Refactoring" (by the way, the link directs you to a price comparison site which I find REALLY useful).

I tried to use this method is my latest project, a test framework, and I was extremely pleased by the results; not only did the design morph as I discovered the requirements (so the final result was a design I was pleased with), but the final result snapped into place around the item being tested and no bugs have been discovered in it!

I can highly recommend at least that part of XP: refactoring and coding for today. I'm going to try some of the other parts in small bits at a time, but from what I see they look very useful.

I certainly wouldn't describe XP as being for someone who doesn't have time to design; I would describe it as something for someone who doesn't expect to get it right the first time, and who wants to get it right anyhow. I would also describe it as being for someone who wants to be useful to customers without hurting his own productivity -- see the Bill of rights on the Extreme Programming website. In short, XP is about never having to tell a customer that you can't make that change now -- and it's also about always being able to make changes when you need to.

-Billy

Comparison shop (was: Save some cash....)

[RocketJeff]

I've been using Best Book Buys for a few weeks. It looks at quite a few online book stores and shows you the prices (including shipping and availability).

Prices for this book range from $23.92 (Buy.com - out of stock) to $33.90 (Borders.com) (both are UPS ground prices).

It's saved me a few dollars, now and then.

Jeff

Re:Comparison shop (was: Save some cash....)

[William Tanksley]

I'll vouch for BestBookBuys -- I generally use Amazon to look up reviews and do searches, then I copy the ISBN and paste it into bestbookbuys.

Saves money and time.

I'm not going to do any more reviews for Amazon, though; my reviews are now going to Fatbrain instead.

-Billy

Have the scoffers actually tried coding like this?

[vivarin]

I have, and it works beautifully. In fact, the nastier and more complex the coding problem the better. Two people watching pointer arithmetic are always more careful than one.

That said, a few commonsense caveats must precede its adoption. Namely:

• Management must allow it
• If coders are of wildly differing skill levels, then they shouldn't be paired for long periods of time or swaths of code
• For shorter sections, an individual can take a "first cut" at coding something while somebody else tracks him with test code. Then there should be a trade off and review period.

I know of no piece of code that I've written this way that hasn't been FAR less expensive than its more traditionally coded counterparts, once debugging and QA expenses are factored in. The interesting part is that design and refactoring remain part of the process long after they would normally, as ones assumptions must be continually re-examined as people new to the code come in.

Re:Today's Buzzword is...

[William Tanksley]

No, these are not code reviews -- they don't produce the documentation which code reviews do, and the documentation they DO produce is intended to document the entire project, not only the code quality.

Code reviews are definitely cool, and with enough programmer mobility this would remove *some* of the need for code reviews. I would still want them, though; they provide another aspect of managability. I would also want to use RMMM reports with spiral lifecycle model. This also seems to combine nicely with a modified Surgical Team model.

Of course, all I've done is the Refactoring part of XP -- but it alone was enormously effective. I can't wait to start pulling in more and more pieces from XP, and eventually start combining it with other proven methodologies.

-Billy

Re:What is the "engineering phase"? :)

[Eric Green]

I think the biggest problem is that "code monkeys" don't know how to write, don't want to write, don't like to write. I take a day out of the development cycle from time to time to modify and continue my write up of what I've done so far -- not the detailed low-level implementation (that's done in the code), but the Big Ideas -- protocol descriptions, overall program structure, what requirements led me to implement certain things in certain ways. Then I clarify what I'm going to be working on for the next week or two in that same way, and circulate that requirements doc around the office to make sure that I'm not missing something obvious.

Yes, I took a day to specify the requirements for one program that I'm working on. So what. That saves weeks in overall development time, since I won't need to throw away that work in the future due to an implementation that doesn't meet future requirements.

-E

Don't forget www.ExtremeProgramming.org

[cpeterso]

For a gentle introduction, check out ExtremeProgramming.org. They also have a nice on-going column about developing an "extreme"-style coffee maker.

Design Patterns

[Corrinne Yu]

Disclaimer : I hope I do not offend Design Pattern purists in my following loose interpretation of DP. I merely attempt to attribute my own ideas to others. If the following ideas deviate too greatly to be called DP, then I am grateful I have actually formulated or described something original.

There are many threads above that argue vehemently against and for a non-written-spec-based development pattern, "hacking" so to speak.

What makes a great project, and makes for great programming, is not the *winner* of the paradigm or design pattern.

It is not something as simple as "OO rulez", or
"Pedal to the Metal."

A great project, and a great programming team, distinguishes itself by the proper choice and appropriate use of paradigms and patterns.

// Flame Bait Alert Begins

Zealous proclamations like "Never have multiple function exit points", "Never use gotos" and "Never have modules exceed N number of lines" often stem from *inexperience* that never had such proclaiming coders be *forced* into such circumstances when breaking such rules lead to easier to read or more maintainable code in the long run.

// Flame Bait Alert Ends

Many development books and paradigms are written as if code and programs are created in perfect sanitized heavens that I have yet to experience in any game development house.

On the flip side, there are many game programmers or *hackers* who can so glibly discard decades worth of language or system theory research, and sincerely believe they can create thought systems of equal robustness in a matter of months.

We do not need more flame throwers.

We need accessible information and research to cloistered practitioners like us. And we need to contribute back to academia codified and scientific "development patterns" we corroborate with our colleagues/competitors so the system of knowledge can "grow" instead of "treadmill."

Post-mortem's and magazine articles, even interviews, are good baby steps. But our field needs something more scientific and codified than that.

We need to know not only what fails and what works. But what works in "which circumstances." What fails in "which other." Lots of lots of redundant information to prove one way or otherwise.

In retrospect, what I describe is more "development" pattern than "design" pattern.

I would greatly appreciate if fellow posters and programmers can point to me, or discuss from personal experiences, various "development" patterns that work, and various that fail.

The most enlightening discussions would be given the same "development" pattern, one that succeeds in one circumstances, and fails in another.

Then is the failure based on inappropriate circumstances, or improper application?

A more informative discussion is not "this paradigm fails" "this paradigm is god" but how which paradigm best works in which circumstances.

I hope to see from now on discussion goes more in that path.

Apologies of diverging from the topic of the specific book.

Disclaimer : If you are inclined to respond in flames, please at least first re-read what I am really saying. Thank you.

Corrinne Yu
3D Game Engine Programmer
3D Realms/Apogee


Corrinne Yu
3D Game Engine Programmer

A Reason to like XXTreme PProgramming.

[BillWhite]

If you pair programmers in the same cubicle, with only one computer between them, you don't have to worry that one or both of them are wasting their time reading and replying to Slashdot articles.

I did this too; it worked great!

[Anonymous Coward]

I tried pair programming here at work [WebObjects/ObjectiveC], and it worked wonderfully!

We came out ahead of when we thought we could get through, had very few errors along the way, and had a LOT of fun.

I remember that when my partner left, everything suddenly seemed to slow down. Two people working on a problem at once has a synergetic effect. It forces a laser-like attention without distraction, and shows views of things from more than one angle. At least, that's my explanation for what happened.

An absolutely wonderful experience.

Read the articles here: you'll notice a lot of SPECULATIVE "I don't think it will work"s, and you'll notice a lot of "We DID it, and it works great."

Lion Kimbro =^_^= . o O ( Someday I'll get yet another Slashdot account...)

XP was designed for exactly that situation!

[cpeterso]

Extreme Programming (XP) is supposed to be an ongoing dialog between "Business" (the "Client") and Development. If the client wants a new feature, the developers need to introduce the Client to the "Software Triangle" (Cheap, Fast, or Good: choose two). Clients are paying for the software, so they should get what they are willing to pay for. If a client wants a new feature "quickly", the Developers give the client a revised time and cost estimate for the added feature. Developers are good at deciding how long something will take to implement. Clients are good at deciding whether they want to invest the time and money for that feature, once they know how long the feature will actually take to write.

Re:Today's Buzzword is...

[William Tanksley]

If you don't want to be modded down, guy

Don't threaten, okay? Just ask. You had a good question, but this is just plain rude. An apology would serve you well.

Anyhow...

give us links for that stuff. I'm just a real programmer, not a CS wonk, and I haven't even heard of half of your buzzwords.

No problem.

RMMM doesn't matter (it's a detail of spiral development); look up Spiral Lifecycle Model in any decent SW engineering book. The closest one I have is "Rapid Development", and it does indeed discuss it.

Code Reviews are a formal SW engineering practice documented at Univ. Hawaii's page.

The Surgical Team model is one of the many memorable concepts discussed in Fred Brook's "Mythical Man Month". Read it. It's the #1 most recommended book by the Greats of CS and SW engineering.

Refactoring is part of XP, and is documented best in the book titled "Refactoring". I highly recommend it; serious programming stuff there, no fancy-pants process requirements.

XP is facinating because it tends to reduce the parasitic overhead of process. For example, no formal design documents are required with the exception of the user stories and the code.

-Billy

watch your scope run away

[zzzeek]

my experience with minute-by-minute customer feedback is that the entire notion of a functional specification is lost immediately, as about one out of every five opinions they have on something will in fact be a new feature they just thought of now that they are seeing their product come to life. Having the customer sit with the programmer all the time means there is no project management layer in between, its just the client making demands directly to the programmer who has much more pressing (and non-programmer intangible) issues on his/her mind than all the random little frills that should have been spec'ed out and properly scheduled. All of the customer's thoughts and use case scenarios should be gathered into a proper and complete specification before any programming work is begun - if the programmer needs to ask questions to the customer during development, then the specification was not completed. Continuing down the road of developing a large project without a roadmap will guarantee project failure, major psychological/physical trauma to your programmers, or both.

Re:business programming & Open Source

[Pfhreakaz0id]

I found your comment about Business Programming being "programmed programing" interesting. A fair amount of what I do is repetitive (I would argue that some is not. That some businesses are so specialized in what they do and so convuluted in how they go to market that no AI computer program could ever replace a talented System Analyst and programming team).

This dovetails nicely with something I came across on the web the other day. General Office. It's an Open Source Accounting package. Many different modules. Ability to use a large variety of data sources (Access, SQL server, Oracle), although with the source you could use anything without too much problem. Written in VB6, source included. LIcensing options for resellers. I thought "man, we can start with that, customize for the client, and it'd be like doing a whole custom app for them!" I think this is a GREAT idea.

Re:Recipe for Disaster

[the eric conspiracy]

The benefits of forcing everyone to expand their domain knowledge and have a better overview of the system far outweigh the communication and ramp-up obstacles.

In the real world, a programmer quits on the average of every sixteen months. If that individual "owns" lots of code that others are not
familiar with, the costs are enormous. It happens all the time.

Given that the average tenure is so short, teaching other people the code is still a waste of time because they too will be gone in 16 months.

When everyone is gone, what do you have? Code and the documentation. Except under Extreme Programming the code is built ad-hoc, and the documentation is non-existent.

In addition if you read Brooks, he had a rather different view of the programming process. It is very well known that GOOD programmers are as much as 1000 times more productive than average programmers. Brooks's methods were designed to take maximal advantage of the talent of the top programmer, rather than this pair-programming idea which seems to me designed to achieve least-common denominator code quality.

Rotating responsibilities is just one idea that has immense value outside of general XP practice.

I've worked in rotationg responsibility environments, in technical organizations outside the computer industry. Do you know what happens? Nobody is in a job long enough to become really proficient.

practices that succeed

[Lumpish Scholar]

Caveat: I was one of the technical reviewers of the book. (Yes, Kent misspelled my name.)-:

I see some big problems with the technique:

There are concerns about when and how to apply XP (Kent raises some of them in the book), but your response based on the review is off base.

I haven't "done" XP; I have successfully used some of the practices Kent calls for as part of XP. Here's my experience on how some of those practices work.

no individual code ownership: this seems to suggest that the design would either be imposed from the top, which implies that coders would be treated as unspecialized labor, or that the entire team would be involved in the design of every piece of the code

No. Lack of individual code ownership is one of the extreme practices, but neither of your deductions is correct. The design is fluid (as it is with any working method); no one is forbidden from making changes (there's no code only I can touch), but no one [pair] makes changes without understanding the code as proven by the fact that their changes must pass the existing unit tests and must be supported by additional unit tests created before or at the same time as the changes.

Lack of code ownership is controversial. I work in a project with a couple hundred thousand lines of code, and half a dozen programmers, that makes it work just fine (not using XP). I work alongside a much larger project, with hundreds of programmers, that also works that way.

new team members: the review asserts that "New programmers can also be brought in and up to speed much more quickly." but doesn't specify how.

By pairing new members with more experienced members. (The new member "drives" most of the time, so the information flows through him or her; he/she's not watching, but doing.)

debugging in pairs is a good idea, and should be used by any programming team that has the personnel to spare

No no no no no no no. Programming in pairs is *not* just about debugging, and is NOT about "spare" staff. When it works, two people programming (designing/ coding/ testing/ debugging) in pairs work faster than the two individuals working together ... often much faster.

XP (and refactoring, as described in a book reviewed here a few weeks ago) calls for generating huge numbers of unit tests, completely automated, that can be run at the push of a button, and which are run several times per hour by each pair. That's where your coverage comes from.

design to throw one away ... A methodology that doesn't address the need to redesign ...

XP calls for continous refactoring of the system "to work around mistakes and compromises made when" developing, not disposable prototypes, but earlier versions of the system.

Most programming projects don't have the latitude to set prices for each feature

Each feature will take as long as it takes. XP suggests ways a project can learn how to better estimate these "prices", and spell them out for the customer. Any project, XP or not, where the customer says, "You say this will take six weeks, but I need it in one, so you'll have it in one week," is doomed anyway.

summary of XP

[Lumpish Scholar]

Caveat: I was one of the technical reviewers of the book. (Yes, Kent misspelled my name.)-: Funny coincidence: I got my copy of the book Tuesday, just about exactly when the review was posted at Slashdot.

I can't find fault with anything in particular with the review, but to me it doesn't capture what makes XP so, um, extreme. Here's my take.

XP is a lightweight, high discipline method (to quote Alistair Cockburn). You don't write a lot of documents; no "Victorian novel" requirements boat anchors. What you do, you do all the time; no "cowboy coders" (despite what you might guess from this or other reviews).

What you do in XP:

o Programming in pairs: No one designs, codes, tests, or debugs by themselves. There's an incredible gestalt you can take advantage of; if you've done it, you know what I mean. (If not, read the book.)

o Unit tests: Hundreds, thousands of them, more every time you make a change, run several times an hour, all available at the touch of a button. The idea is to save time, not spend it; if you make a mistake, the unit tests have a good chance to find it fast.

o Refactoring: Don't design or code your system to do what you think it might do; instead, implement the features you're working on currently, but be willing to go back and refactor it (make big changes) "just in time". You're not making the smallest, simplest change each time; that leads to the evolution of unmaintainable dinosaurs. Instead, you're creating the smallest, simplest system that does what you need it to do, even if that's big time different that the smallest, simplest system that does what you needed last week. (How can you be sure you didn't break anything? Hit that "unit test" button, early and often.)

o Low tech project management. Break the project down into "user stories" (use cases); track the estimates and actual time for each; learn what your project's "load factor" is, and how to improve your future estimates. Gantt chart weinies need not apply.

o Short development intervals; incremental releases every few (two to six, often) weeks. Avoid the "90% done" disasters. Yes, this is in the spirit of what Grady Booch calls "spiral development." (Does anyone still think they're following the "waterfall" model???)

o Teamwork. Kent doesn't say so, but XP's not likely to work for programmers who punch a time clock (though he strongly encourages working only eight intense hours a day and then going home!)

XP is a very new method. There's not a lot of track record, sucesses or failures, out there yet. Consider it for a project with three to ten developers and the need for having something minimal working fast, and more features on a regular basis after that. (Read the book first!) Don't bet your next hundred developer project on it ... not yet, anyway.

Re:Recipe for Disaster

[David Kennedy]

"No, you'll have programmers who understand the whole system"

You will? I've been in my current job for 14 months, and I still don't even understand my own subsystem (40+K of C++ involved with network management systems and multiplexers).

The whole system here is just under a MLOC of complex code. (Okay, so it's legacy, but aren't
most systems? Especially those where you need to turn things on their heads to meet new pressures)

We rotate staff frequently, in 14 months we've had 13 people leave (mostly internal transfers). That's bad for a team which has about 7 developers (plus 3 integration/test people).

Swapping people doesn't work. Learning time soaks up any and all advantage in bringing in even highly experienced people.

Aside from that much of the XP rules are okay, but not shockingly novel. I've been pair programming and constantly peer reviewing for months, it helps, but the single most important thing on a big project isn't mentioned in the XP book (skimmed it only so far) - source control.