Slashdot Mirror
Extreme Programming Refactored
The Previous Extreme - What XP Set out to Fix It had previously been accepted practice to spend months (years, even, on large-scale projects) gathering requirements, then another year or two on design, before a single line of production code had been written. The infamous "big bang delivery" occurred when this gigantic monolith of a software system was finally delivered to the customer, only for the customer to retort that this was nothing like what he wanted.
It was also accepted practice to divide the system into separate subsystems, and attempt to integrate them after several months. By this time, each subsystem would have taken on a life of its own, and integrating these disparate monoliths together gave a whole new meaning to "plug and pray."
How XP "Fixes" It XP takes the development process to the other extreme, by shortening the "waterfall" lifecycle to weeks, days, even minutes. In fact, Kent Beck describes XP as a "waterfall run through a blender." Iterations typically last for a week or two; there is a high emphasis on code quality via unit testing; and code is integrated "constantly" so that it never becomes out of synch with the rest of the project. Beck is often quoted for saying that the XP practices "turn the dial all the way up to 10" -- that is, if something is good (testing, integrating, pair programming etc), well then, let's do it all the time.
There's a lot to be gained from learning about XP, and agile practices in general. However, many feel that XP has taken things too far. By taking things to the opposite extreme, we're just introducing a fresh set of problems. The optimum solution, then, must lie somewhere between these two extremes. That is fundamentally what Extreme Programming Refactored (XPR) is about.
Optimizing the Process There's been a lot of controversy surrounding this book. It grew out of an equally controversial article that appeared on the author's website. XP advocates were arguing on Yahoo! Groups over XPR's good and bad points, miraculously, months before the book was even available. XP zealots were even posting messages telling others not to buy the book, before they'd even had a chance to read it for themselves and find out what it's all about.It's important to note that XPR isn't the anti-XP slam piece that some people had been expecting. It does rip into the XP practices in plenty of detail, but importantly it also describes alternatives, and talks about the good aspects of XP.
The authors make the argument that "turning the dial up to 10" mostly isn't such a good thing, and that to achieve our "holy grail" development process, we just need to turn the dial down a little bit -- let the milk simmer rather than boil over. We can do this by adding in some additional practices that take the burden off the overloaded, heavily interdependent XP practices.
For example, not everyone likes to pair program (with two people sitting at one computer). It just isn't for everyone. However, XP relies on everybody in the team pairing all the time. So if you don't like to "pair up," what choice do you have but to leave the project? XPR adjusts the other practices, placing a bigger emphasis on up-front design and documentation, so that pairing up doesn't have to be mandatory.
XPR also argues that it is possible to achieve a decent design before writing the code. The authors don't want a return to "BDUF" (Big Design Up Front), but instead to achieve an ideal middle-ground. The result is more akin to the monthly Sprints found in Scrum (www.controlchaos.com).
Similarly, XPR argues that the customer (and users) usually do have a pretty good idea what they want from a new system, and that they don't have to see a live system first before realizing that they wanted something entirely different. The authors argue in favour of interaction design as a way of achieving this goal.
XPR achieves all of this with more than a mild dose of satire. It's important to realize this -- the book is essentially "taking the piss" out of the more extreme XP practices, and the quasi-religious Extremo culture that has quickly grown up around XP. It has lots of serious things to say, but has a slight danger of that being lost "in the chuckles." There again, the danger is less to do with the book, and more to do with the reader.
XP sealots will never be swayed by such a book, naturally, but they are not the audience. It is for those undecided, or the cowed XP skeptics who know something is very wrong at the heart of the beast, but haven't have the words to say it. Even for zealots, I'd hope they'd put the hatred for long enough to at least temper their XP actions, to turn the dial down a little, to read the contents with the possibility in their minds that XP isn't the final perfect expression of all programming methodologies. Just for a while...
If you are scared of the contents, a favorite XP accusation, then of course you'll point out the 'needless humour,' blah blah, anything rther than address what the book says. Form will be far more addressable than content. It's the old "ignore this man, he wears a colourful tie" excuse, pick on some small detail that you feel is a weakness and totally ignore all the embarrassing questions you'd rather not address. If you like the contents, then the humour will be seen as a playful, court-jester like addition to what is a seriously analytical book
In conclusion, this book is well written, thought provoking, and above all entertaining (an aspect which seems to be proving almost heretical among some XP advocates). I found this to be a fun read, unlike some books, it was never a chore. It's extremely conversational, like having a cynical, wise-cracking guide. It's a pity more computer books aren't this fun. A spoonful of sugar and all that...
In fact this book is pretty damn wonderful. I know, it may sound a bit gushing, but before you review my review, give the book a read yourself. It's a thing of beauty, a rare mix of positive and negative, sweet and sour, opinion, and XP's favorite emotion, courage, courage enough to say "the emperor has no clothes." I can't see how you could read this book from beginning to end and not see XP in a different light.
In fact, XP programmers would do well to read this book, as it presents the negative path, something other than sunny-day scenarios. Using these warnings and guidelines, they'd have much more successful projects, as this book points out the dangers of lack of XP discipline, fragility and so on.
The truth is that I couldn't do justice to this book in such a short review. There is just so much evidence, so many contradictions pointed out, endless damning words from XPer's own mouths. It was supposed to be a small book to start of with, 150 pages or so, but due to the sheer body of evidence and submitted real life stories from those in the trenches, it bloomed to 400+ pages.
As Doug Rosenberg says "I don't want to be nearby when somebody decides to deploy an air traffic control system or some missile-targeting software that has been developed with no written requirements, and where the programmers made the design up as they went along." At least don't say you weren't warned!
You can purchase Extreme Programming Refactored: The Case Against XP from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
I am delighted to see that XP has now been revisited and is being shown to not be the end-all of development styles. Perhaps with the slightly less "extreme" version, more managers will be willing to accept the changes. I have never had a boss accept XP, simply because they were scared of it (IMHO).
stuff |
Well, have you?
I'm about to start a project using pair programming (not exactly Extreme, but something like that) and I don't like it one bit
Pair programming is uncomfortable on our reduced space. And it's noisy.
Are the inconveniences worth it?
Kilroy was here!
The XP guys _have_ made a mark in the world. Now, how that is going to work out longer term, remains to be seen. XP _is_ influecing stuff like IBM's Rational Unified Process.
I tend to like XP. I haven't worked in an XP shop, but I have used it in class projects and some project of my own.
Right now, lots shops have processes that are non-existant or chaotic--that is what XP really needs to be compared against. XP isn't "the emperor" more like an upstart prince edging in on the territory of being eyed(but not governed) by Emperor RUP.
My gut is that was is motivating books like this:
is the fact that XP is being adopted in places where stuff like RUP just would never get a toe in the door in its present state.
The major stages of the opponents of an invention:
a) it won't work
b) its evil
c) its not really new
d) we invented it
This stuff strikes me as somewhere between stage a-b.
I would have like to have seen less of the pre-emptive rebuttal and more reviews on some of the findings and recommendations.
Drop the dogma and theology, and give us the hard core, nitty-gritty, down and dirty facts.
I'm one of those people who have seen XP tried and failed, and to be honest I never found it a suprise. I was educated to be a software engineer, and that the best way to deliver software effectively is to understand the problem domain. Iterative models like RUP or DSDM are great ways of delivering functionality quickly... XP is not... however there are some ideas in XP that are completely un-original that can work
1) Pair programming - first seen in the "Surgical Team" idea from the mythical man month
2) Unit Tests upfront - first seen in the 60s with the moon landing and space programmes.
3) Iterations - First seen during the 80s with the rise of object oriented systems
The ONE original idea in XP is simple...
You don't need requirements before you start coding. For godsake that is a friggin DILBERT cartoon.
XP assumes the John Wayne school of hacking, just hack hack and your talent will get you through. The reality is that if you are brilliant ANY process would be okay, and if you aren't you need more process to make sure you don't FUBAR things.
I loathe XP, its a strong word but to me it represents the Microsoft view of software in its documented form. Quality doesn't matter, it isn't engineering...
Its just lobbing together some-stuff.
I'm an engineer... what are you ?
An Eye for an Eye will make the whole world blind - Gandhi
I cannot attest to actually having been involved in any project which used XP as defined but I can say the following about what I consider similar:
I used to work at a company that strangely enough fostered both getting things done quickly for some projects while fostering a long drawn out method for other projects.
The company was a consulting firm which always needed an answer yesterday for this or that problem.
They also wrote fairly large simulation packages for the FAA. Some of these projects would go up to three years. Lots of planning etc. was usually done.
Regardless of which type project I worked on I always tended to be the type who would get something working quick and then iterate over it several times if necessary.
Others tended to be going at what I considered to be a snails pace mode. They would plan for days, weeks, months on something that I never considered worth that much planning.
I could usually have a pretty good prototype working with a couple of major revisions made before the "planning types" could even get out a initial prototype.
But, over the years I noticed that I (and those like me) tended to get "officially finished" in about the same amount of time that the annoying guys who planned everything out to the Nth degree got finished.
Was my product any better than their's? No not really.
Which methodology was better? It's hard to say, I know I certainly always liked to utilize new methodologies that worked well while they liked to stick with the tried and true.
About the only thing that I can say is that my internal company clients usually appreciated that they had something to work with earlier rather than later.
They knew it was a rough version, but hey, if you need it now, you need it now. And two years from now may not pay the bills.
So, lots of times I came out looking like a hero.
But, looking back on it now I'd have to say that my own personal version of XP wasn't any better than my counterpart's long drawn out process, at least in terms of a final product.
My $0.02 worth.
Sorry if I was too far off-topic...
Caution: Contents under pressure
The "waterfall" was great for determining exactly what the user wanted but then ended up delivering it five years later after the requirements had changed.
"Use-case" analysis was great for determining how the user would interact with the system but tended to automate the existing processes instead of figuring out how automation could make the process unnecessary.
OO was great for implementing the non-reusable objects determined to be needed by "use-case" analysis (see previous item).
Too many shops implement XP as an excuse for not understanding the user requirements but just bashing out a bunch of high quality code that solves *the wrong problem* in a short amount of time. This isn't a criticism of XP so much as a criticism of the way it is misused (as has every other innovative software development technique).
The basic problem is and always has been that inventing the future is difficult mainly because the future has a habit of changing while we're busy trying to invent it. No technique other than a time machine will ever solve this problem because it is inherent in the invention process.
They that can give up essential liberty to obtain a little temporary safety deserve neither safety nor liberty.
Ben
Extreme Programming or Rapid Application Development is just an excuse to bypass solid process in order to ship faster. Does it do the job better? No.
General problems with the approach:
Design on the fly. You get less features, more hastily implemented. Code is less maintainable.
Without a plan up front, you have QA scrambling to keep up. Testing is full of gaps in coverage. Product is more buggy as a result.
The only real benefit is shipping faster. You make everyone work harder and get an inferior product as a result. I suppose Marketing will be happy with that as long as the customers will pay for it.
Personally, I prefer doing the job WELL, not just doing the job, but I seem to be in the minority in the development industry these days. I know a lot of engineers would prefer to do the job well, but go along for whatever reason.
Having been involved in development for about 10 years, I've seen product quality decline over the years. Products in general have become more complex, but defects have increased substantially. I'd like to see a lower level of defects maintained and value to customer in stability, reliability and functionality/features increased.
Call me a dreamer.
> But "real design" doesn't work, at least, not very well.
Says who? Works pretty well for me. My team and I are generally *more* productive when we design *before* we code. Ever hear the saying, "Measure seven times, cut once"?
> "Real design," as you call it, requires that you keep to
> yourself, code code code in secret and hope what was
> defined in the beginning is still what the customer wants.
*cough*bull*cough*shit*cough*
A development *team* usually consists of a point person through whom the specs are communicated, an architect who puts together the high level system design (including interfaces if possible), coders who take their cue from the architect, and testers who make sure that the code meets the specifications - thus closing the loop. There is a LOT of communication going on there. However, your team is only as strong as its weakest link. I'll give anyone the boot who can't accomplish their job (as is all to often the case from "wanna get rich, but don't know how to program/test/communicate" kids coming out of school these days).
> The lack of future planning in XP is not a flaw, it's like
> that on purpose. I believe that hacking is the best most
> ideal way to code.
Dear Lord. You people are INSANE. When the f**k are managers going to stop hiring people who actually know more than themselves about programming? No self respecting manager should EVER allow juniors like this guy to decide how development should progress.
Javascript + Nintendo DSi = DSiCade
Traditional software development has problems so we must do something different.
XP is something different.
Therefore, we must do XP.
It shouldn't come as a big shock that, even though XP is different than the waterfall development model, it is possible to develop overbudget and over schedule applications using XP.
When I was first asked into the project I was shown a "waterfall" application that another firm had developed for them that was over budget and significantly failed to meet the business need. I was asked to re-estimate the project and determine between saving and scrapping the failed code. Considering the falied app only aligned with about 30% of the specified need, we chose to dump the app.
Of course by now alarm bells are going off inside my brain (how can a vendor land that far off target, etc.) and I start to do some digging. It turns out that the vendor they selected was fairly inexperienced, but more importantly the product had been produced using an estimate of about 1/4 the cost estimated by a far more experienced firm, and much closer to my own expectations. Even further digging turned up that the client didn't even have a firm grip on the details of internal the process they were trying to automate. Process changes and adjustments were ongoing, and asside from a few key elements, virtually everything else was subject to "interpretive need"; meaning the rule was whatever best suited the audience or need at the moment.
Of course the new client was hoping I could find a magic wand and produce their project while meeting the unrealistic proposal of the less expensive vendor in this environment. Eventually I was able to develop a product that met the bulk of their need; but it was done with the understanding that until they clearly defined their needs, development and adjustments would need to be ongoing.
The project has been ongoing for almost 2 years now, and meets about 95% of the business needs. About 35% of the development has been part time, as we have figured out what they really want (and need). The remaining needs have been defined and are being developed for as they define the busniess rules.
The point to all this being that many times the client doesn't understand their own needs well enough to provide a spec that can be developed to. If you can't understand the problem fully, it's tough to produce an estimate for a solution.
I've seen plenty of methodologies that say something similiar to "at point X you can estimate to Y hours (or dollars), +/-100%"; that may be true, but it's a tough sell. In the world of small business, most customers would call that "bunk".
Have you ever actually tried pair programming?
I'm not an 'exceptional programmer' - I'm just a fucking good one.
When pair programming, I benefit from the expertise and experience of another developer. I avoid making silly mistakes, because he spots them for me. I don't write as many bugs, because he spots them. I don't dither about variable names, or whether to refactor a method, or which algorithm to use - because I have someone to help me keep focussed, someone that helps me make swift decisions, that keeps me on track and moving forwards.
Sure, sometimes I want to read a news story on the web. That's why when pair programming we take a break every hour or two. It depends on the progress we're making, how focussed we are, whether we've got the code to compile and pass all its tests.
Pair programming is not punishment. It's an enlightening experience, a joy to participate in, something I want to do more often.
Nobody is such an exceptional programmer that they can't learn from someone who still needs to look up the language syntax. Lose your ego, open your mind, and actually try PP.
~Cederic
The unfortunate thing is that most reviewers have lost all sense of reality. One would think a 5-star review represents a classic for all time--maybe something on par with Shakespeare, or (dare I suggest) Design Patterns. A one-star review represents something absolutely worthless and/or dishonest, like the half-made-up Bellisiles' book "Arming America."
Extreme Programming Refactored is neither a classic, nor is it a complete waste of money. There are some good insights in the book, as it points out some of the pitfalls in the application of XP. It is for the most part well-written, but it also resorts to cheap shots and name calling. There are some serious problems with the book as well, including incorrect information and ignorant testimonials.
Most honest people would agree with this assessment of the book. Unfortunately, the bulk of the reviewers aren't. Honest, that is. XP haters rate it five stars, XP lovers rate it one star. [I fall on the pro-XP side of the fence, in case you cared.]
Even big-process bigot Steve McConnell chirps in with a 5-star review. Gosh, Steve, does that mean you think the book is a software development classic, on par or even better than your book Code Complete, which only gets 4.5 stars on average?
In any case, I did write a review of the book at Amazon, complete with quotes and such, and I didn't give it 5 stars, nor did I give it 1 star. But I guess my review sucked, because McConnell's review got more happy-popularity-votes than mine did.
Extreme Programming =
1. User department has some idea of what they want.
2. Users sit down with you and give feedback while you develop the system for them.
3. Regression tests are created early to ensure that changes don't break anything.
4. Another developer works with you to help eliminate coding errors, etc.
5. Users get exactly what they *really* wanted since they were there driving the development and evolution of it.
Non-Extreme Programming =
1. User department has some idea of what they want.
2. Business Analyst manages to interpret 80% of this and puts it in a Requirement spec.
3. Systems Analyst manages to interpret 80% of the Requirements spec and puts it in a Functional spec.
4. Coders manage to produce 80% of the features they sort of understood in the Functional spec.
5. The users test something not at all resembling what they expected, but they are happy to *finally* get something to look at.
6. Users end up with an incomplete, out of date piece of junk that is nothing like they *really* were hoping for in the first place.
Says who? Works pretty well for me. My team and I are generally *more* productive when we design *before* we code. Ever hear the saying, "Measure seven times, cut once"?
Having done things both ways, I'd say that incremental design improvement is the way to go.
Having done XP for a few years, I spend about 20% of my design time up front, about 30% while coding, and the rest while refactoring.
I like this a lot more than trying to do 100% of the design up front. Why? Because your design is only as good as the information you have at the time you do the design. For non-trival projects, it's impossible to have all the information you need up front; even if users never changed their minds once they saw the result (ha!), any good developer learns things as they go.
It also turns out to be much more productive. If I have to do all my design up front, I'm much more likely to overdesign; much better to have too much than too little. But if I do my design on a just-in-time basis, I can avoid a lot of needless work.
And as a bonus, my designs end up being better. The product may be version 1.0, but because I've been iterating over my architecture along the way, the architecture is version 5.0.
There is an interesting interview between Alan Cooper and Kent Beck at http://www.fawcette.com/interviews/beck_cooper/def ault.asp
Cooper was testing his ideas on goal centered design through his "Design+Fun" workshops. I offered up the opinion to Alan that software design is a bi-directional search: with firmware on one end and wetware on the other end. Alan and Kent start from either side of this search.
From XP, I gather that there is much exploration of the domain of APIs to deploy a solution. Writing code immediately helps document the search in such a domain.
From Goal centered design, there is much exploration of the range of a customer's goals. Writing designs helps document the search in such a range.
Using both: Goals help prune unnecessary features, while XP helps prune unrealistic expectations. While using either method could get the job done, perhaps using both is more efficient.