Extreme Programming for Web Projects

PinglePongle writes with this review of Addison Wesley's Extreme Programming for Web Projects, writing "The authors work for a web shop, building websites for customers, and try to use their experience to make life easier for their readers. Their main point concerns how traditional web projects are structured to leave at least one of the parties taking a big risk on the project: if the project is 'fixed price, fixed scope' the developers take all the risk, if it's 'time & materials' the customer takes a risk -- they can not be sure their money will lead to whatever it is they want." Read on to see whether the authors have successfully outlined a fairer, more successful system in the rest of PinglePongle's review, below. Extreme Programming for Web Projects author Doug Wallace, Isobell Raggett, Joel Aufgang pages 165 publisher Addison Wesley rating Poor reviewer PinglePongle ISBN 0201794276 summary A book about applying the Extreme Programming methodology to web projects.

Can good ideas dominate the buzzwords? This risk -- the authors contend -- is the reason many web development projects fail in one way or another. The client's objective is to obtain maximum value, the developer's to incur the least cost possible without getting sued.

The authors show a way in which this risk can be shared fairly between the client and the developer, by using XP and iterative development cycles, alongside a release plan, to acknowledge the risks inherent in a development project, and manage them rather than try to pretend they don't exist. The project team -- client and developer -- work together to create an iteration plan, and use this shared understanding of the requirements to guide the project.

The book is structured into 4 parts: Part 1: XP and Web Projects explores the problems associated with web development projects. Part 2, Working on Web XP Projects explores some of the practicalities of the authors' process - iterative development cycles, the development environment, team roles, and the graphic design process. Part 3: XML and Web XP is a bit of an oddity in a methodology book -- it focuses on some technology-specific issues which the authors claim can be addressed by using XML. Part 4: Web XP Best Practices discusses planning, design, coding and testing issues.

What's good about this book? Well, there are some insights into the relationship between suppliers and customers in development projects. (I don't believe, though, that they're as specific to web projects as the authors seem to claim).

What's bad about this book? It seems to be a sales brochure for the author's web shop -- "we do things thusly, and it yields fantastic results every time." The text is full of fairly broad, even sweeping statements ("Many programmers put SQL code right on a web page" -- when was the last time you saw a select statement on a web page ?).

The authors do not really seem to be able to identify those aspects which make web development projects different from other types of development. Some of the team roles they recommend are bizarre -- the authors identify the role of "Strategist" who seems to help those poor idiot customers to understand their own business. This may be necessary on some projects, but I find this attitude very condescending -- the days when web development was portrayed as a cross between alchemy and spiritual enlightenment are long gone. Many of the sections are very superficial, but the book is littered with footnotes saying "Chapter X discusses this in detail."

In short, I'd say this book is too lightweight for people who understand XP already and want to learn how it applies to web projects, and novices are likely to get hung up on the largely redundant side tracks (CVS versus MS Sourcesafe -- Huh? How did that get past the editors?) to be able to see the extreme wood for the trees.

You can purchase Extreme Programming for Web Projectsfrom bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

I'd rather see Extreme Grocery Bagging

[Adam Rightmann]

for Laid Off WebMasters Who Dropped Out of College With a Passing Knowledge of Front Page to Work For a Dotcom, but then I've always valued a four year degree.

This books sounds like buzzword fluff.

Re:I'd rather see Extreme Grocery Bagging

[Anonymous Coward]

Ummm...bullshit. I have never seen "upper management" give a shit about the education levels of their ranks. Indeed, apart from young graduates it is something that has absolutely no bearing for virtually all of industry. The idea that upper management is filtering through CVs to find someone with insufficient education is ridiculous, especially in technology fields. If you saw someone fired for not having a degree, realize that it was a red herring to fire someone they wanted to fire, and the degree issue is a non-starter.

Re:I'd rather see Extreme Grocery Bagging

[Anonymous Coward]

This is a rather short-sighted view. In 1997, I was faced with the same choice: Experience, or degree. I chose both. Finished high school at the end of 1997, started doing a Bachelor's degree in Information Technology part time, in 1998, looked for work, had a few odd jobs but nothing major, and in my 2nd year, 1999, started an e-billing company with 2 partners. Finished my degree with honours in 2001, and also had a moderately successful company. (Moderately successful back then, substantially more successful now). Good luck to you with your degree only, though.

Admittedly, it has a lot to do with luck, but I'm pretty sure that most people doing their degrees right now could improve their situation significantly, instead of getting their degree and then wondering if it's going to be all that useful in the ultra-dynamic I.T marketplace.

My hiring policy is simple: if you have a degree, great, if not, also fine. It all boils down to how you do in the technical test that my colleagues and I have devised. It's based on knowledge of the following subjects: Operating systems, Python programming, C programming, Zope usage, and Network knowledge. It doesn't matter how many degrees you have, if you want to be a programmer with us, you need to do well in this test. Most of the degreed applicants' CV's look something like this:

Mr Smith

B.Sc (Computer Science) (or B.Tech (Information Technology), or whatever the case may be)

Visual Basic/ASP/ ColdFusion/Windows NT/etc etc.

Look, degrees are impressive and all, but most of the CV's that have them also contain large amounts Microsoft, and that's not acceptable in firms that use non-Microsoft technology for scalability and stability reasons (our custom e-billing app needs to serve millions of requests per day and send large amounts of mail without crashing, freezing, BSODing, or whatever the case may be).

As for other organizations, from all I've heard and experienced, people with degrees are more likely to get laid off because they tend to cost more than non-degreed people, and in business, it's often the bottom line that counts, unfortunately.

From a skills perspective, people who have been programming since their teens or earlier tend to be the best technically (I was one of these people), and degrees do indeed help to sharpen and augment those skills, but are by no means the end-all and be all of programming, experience is far more important.

Hope this helps, and I hope that you manage to find a job.

Re:I'd rather see Extreme Grocery Bagging

[PhillC]

I don't know about the firing side of things, but I have certainly seen CVs rejected during the hiring process for not having the right education qualifications.

An online auction company I recently worked for has a massive hang up about MBAs. If you want to do anything technology related that's not actually coding, for example Project Manager, then you'd better get yourself an MBA.

Also during recent reading of job advertisements I've seen a lot of ads that say something like "must have good degree from a red brick university."

real xtreme programming...

[Joe the Lesser]

is writing code while skiing 20 yards ahead of an avalance AND compiling on first try.

Re:real xtreme programming...

[DA_MAN_DA_MYTH]

While being hopped up on some Mountain Dew!

reliability

[monadicIO]

Somehow, I would never trust an "extreme programmed" program. I feel (perhaps just a prejudice) that extreme programming is a "do now, think later" kind of approach. I'd be interested in knowing if there have been studies with respect to reliability of extreme programmed projects.

Re:reliability

[kryzx]

This comment shows a lack of understanding of the most basic principal of XP: your requirements are defined by your tests, if it passes the tests it passes the requirements.

If a program passes all the tests, but doesn't work the way you want it to, your tests are not good, i.e. you requirements are wrong or incomplete.

The main argument against XP is that some systems are so complex that you cannot build tests to cover all the functionality. Take for example a Quake engine. The combinations of objects, positions, orientations and actions are virtually infinite, so it is impossible to write tests to cover everything. For a system like this XP is not the right approach, and the creators of XP would tell you that.

Re:reliability

[s88]

So let me guess...

You are one of the many who have never read more than two sentences about XP, and jump to the conclusion that because eXtreme is in the name it must be hacking.

I think your assumptions are the exact opposite of what is actually true; "extreme programmed" programs should be more reliable, because they don't guess for tomorrow. If XP is done right, what you have is what you need...nothing more. This is the essence of KISS. If you can't make it reliable by doing the simplest thing possible that solves the problem, I shudder to think what you will produce if you are guessing for tomorrow and adding things you don't need.

There are plenty of studies... just google for them.

Scott

Re:reliability

[monadicIO]

your requirements are defined by your tests
I'd expect many real-life systems to be complex enough that exhaustive testing is not practical. My point is that there needs to be a formal approach to s/w development. As Dijkstra said, testing reveals the presence of errors but not their absence. In safety critical environments, while testing is necessary, it cannot be and shouldn't be deemed to be sufficient.
Again, I'm not passing a blanket statement on ``XP'', I'm merely asking if there is enough statistical evidence of its reliability.

Re:reliability

[kryzx]

Your points are good. By designing well you prevent errors now and in the future.

I guess my issue is that the XP approach does not dictate anything about design. You can and should design well even when doing XP. In fact, in the original book there is a lot of talk about continually reworking the design to simplify and clarify it, and avoiding the temptation to add complexity that is not necessary at the moment, just in case it's needed in the future.

So, I think of XP as a set of tools, none of which restricts or prevents good design.

It would be interesting to collect stats on projects are compare reliability produced by different design methologies, platforms, languages, tools, etc., looking for correlations, but XP is just one piece, and does not equate to a design methodology.

Re:reliability

[William Tanksley]

I guess my issue is that the XP approach does not dictate anything about design.

Actually, it explicitly does. One of its practices is Testy Driven Design: every element of the programmer's design is expressed in the form of a test, written in automatically executable form before anything is added to the code.

Another rule XP requests is to minimise Big Design Up Front: don't do design on something else else until you've proven the design for what you just did. In other words, don't design something and then leave it unimplemented; let every implementation be completed as close to its design time as possible.

I don't use XP, since my company has its own methodology (CMM level 3); I do use Test Driven Design. It works extremely well, and has the pleasant side effect of producing tests to hand in with my code for no additional effort. Given what I've experienced with TDD, I would take up XP in a heartbeat.

So, I think of XP as a set of tools, none of which restricts or prevents good design.

XP is not a set of tools, although there are one or two practices in XP which make sense outside of XP (for example, pair programming can be imagined in another context, and TDD works very well without XP). Rather, it's a complete methodology, much like the Spiral methodology or the much-feared Waterfall (and no, waterfall isn't a myth; I've seen it used a couple of times, and most of the rest of the time it's a good approximation).

-Billy

Re:reliability

[William Tanksley]

Somehow, I would never trust an "extreme programmed" program.

Would you ever trust any other program in this sense? Just curious.

I feel (perhaps just a prejudice) that

Sounds that way. Unless you have some facts that you haven't shared with us...

extreme programming is a "do now, think later" kind of approach.

No, it's the other way around -- "think now, do now." You're supposed to do things as you think of them -- when you come up with a clever design, don't write it down and walk away; instead, codify it as a test and then implement it. All the practices of XP are supposed to come together to prevent premature or delayed action: pair programming ensures that every idea put into code is understood enough to be explained to your peer; TDD ensures that every change made to code actually _adds_ some functionality; and so on.

I'd be interested in knowing if there have been studies with respect to reliability of extreme programmed projects.

A very interesting question! I think that we'll have answers on that in about five years. XP as a movement is a little new; very few companies have had a chance to adopt it, and only a few formal projects have been run with it.

Until then, any XP shop wishing to perform a high-reliability-required task would be wise to adopt some of the known reliabilty practices -- formal reviews, for example.

-Billy

Re:reliability

[phamlen]

In our current project, we've achieved a reliability improvement of almost 2 orders of magnitude based on our XP methodology, and it's increasing every day.

It's actually difficult to track because the original project didn't monitor error rates particularly well. We do know that they had more than 1000 errors over 14000 "processing cycles" (a defined chunk of work.) Our current system has had 7 errors over 7000 "processing cycles".

The vast majority of these improvements come from Pair Programming and Test-Driven Development.

And - of course - we now have tests for those 7 errors!!!

(For those who care, a "processing cycle" has about 10 feature-points that are executed - so the original error rates are somewhere around 1% (pretty dang high) and ours is around .01%)

Re:reliability

[kryzx]

The XP practices dictate how you capture and prioritize your user requirements and how you test your code, and they encourage simple and immediately neccesary design, but they do not dicatate anything about how you actually code your application.

The practices try to make it easier to produce good code design, but they could be applied to any design methodology. You could do XP Object-Oriented or not, with Java, COBOL, perl, basic, or assembler. There is nothing to make your software design good or bad except your skillz and the developer sitting next to you.

So, I'd say XP doesn't enforce any coding methodology, but tries to create an environment that encourages good habits.

Maybe we are having semantic issues. I was talking about the methods for designing at a fairly low implementation level the actual code that makes an application work - the parent comment here expressed concern about code made under XP being sloppy. That's what I meant by design. If you talk about RUP or Waterfall as a design methodology then obviously they are incompatible with XP.

Buzzword compliant.

[bscanl]

Sounds like it hasn't got much actual XP in it.

Surely not another book with a buzzword in the title that doesn't actually focus on the alledged subject at hand?!

wtf

[Anonymous Coward]

when was the last time you saw a select statement on a web page ?

About 10 seconds ago on my current project. I fail to see why I should make a seperate file and include it for one SQL statement, or even for the 20 or so I use...

Can anyone explain the logic behind this? If they mean "when was the last time you saw SQL directly put in the html", then the answer would be never apart from the mysql manual...but then surely thats obvious...if they mean "when was the last time you saw SQL in a page-generating script", then I don't get the problem.

Re:wtf

[mattc58]

Yeah, you want your SQL well away from you presentation level code. Besides any architectural and performance reasons, it's just good practice for maintenance. Your presentation layer really shouldn't care/know that the data it's putting on a screen is coming from this table on that database on that server. It should be a few layers below this that you get into these kind of fun details.

Re:wtf

[Pov]

I would presume they were referring to using stored procedures instead of straight SQL on the page, but if that's the case, then I see it all the time too. I code almost everything to use stored procedures, but occasionally where a complex SQL statement is being dynamically built, it's easier to just code it on the page than try to put all the same logic into a stored procedure that probably won't receive the pre-compiled benefits anyway because it differs too much from run to run. Then again, I could be completely wrong on what they meant. :)

Re:wtf

[clearcache]

You are precisely right...and this hits upon the reason why so many websites are unmaintainable.

Many web programmers don't know the difference between the presentation layer, business logic, etc...and they just throw it all together. You end up with a site that works great in its current version...but is a bear to add functionality to or redesign visually.

Re:wtf

[aug24]

They are referring to the principle that there should be:

a presentation layer, eg jsp over a presentation package, eg tags
and
a business logic layer, eg java classes over a data abstraction layer, eg bc4j

If you do this, you get re-usable, maintainable code. If not, and you're building anything bigger than a pretty noddy app, you're not going to be able to do anything with it six months after the initial build.

Hell, I've even built noddy apps badly and had a hard time maintaining them six months later.

Incidentally, wtf is extreme programming? Somebody?

Re:wtf

[onenil]

While I do know the difference between the presentation layer, business logic, and data storage - what if the customer doesn't actually know the full requirements of the site (or indeed a specific section of the site) prior to them wanting it to go live?

I personally love working on projects where the presentation logic is separated from business logic, but unfortunately the majority of my customers need it in production yesterday, and have no time themselves to actually go through the requirements, nor are willing to commit to requirements that will no doubt change in 3 months time.

And for them, I believe, its just a matter of being in the business they are in - their core business has practically nothing to do with I.T. (short of perhaps the checkout scanners at the front of each of their stores). Its only that they are a nation-wide company, and therefore have an image to uphold.

Re:wtf

[clearcache]

I feel your pain - I really do. We deal with those types of customers every day and it's enough to drive me to drink...it usually does. But I think you'll find that you'll be saving time in the long run by adopting some of these principles in every project.

The Jr. members of my team don't understand how I can make major changes to some of my programs in hours when it takes them days to make simple changes. Well - it's all about the rewrite. I don't find myself rewriting my code as much as I used to...it was like the day I learned that by writing object oriented code, I didn't need to completely reinvent processes to make modifications...just add new methods or - in cases where requirements change - new methods+properties.

And - what I always tell my coworkers - if they needed it yesterday, it's already late. Take 2 hours and come up with a good plan...

The last time I saw a select statement ...

[jlanng]

When was the last time you saw a select statement on a web page

Almost every ASP project I've seen has embedded SQL in the presentation pages (.asp files). Yuck.

.Net goes some way towards alleviating this as the code is usually placed in a paired class (codebehind).

Re:The last time I saw a select statement ...

[Petronius]

When is the last time you worked on a production system? Every ASP, JSP or PHP app I know has them. Eventhough lots of people talk about MVC and Model 2 stuff, very few people actually implement solutions that way: it takes more planning, the code/build/test cycle is longer, etc. People cut corners, it's a fact of life.

OK this one isn't great, but is there a great one?

[japes]

Is there a good book out there on this exact topic? I enjoyed 'Clouds to Code' and 'Design Patterns' and was hoping that there was a another book that delved into designing singular web projects ( with availability later on to pick off the shelf and change simply for new customer ) as well as integrating XML and SQL services.

Does anyone know of any good suggestions?

Re:Jesus Saves!

[TheCrackRat]

Allah Invests!

SQL in web pages

[gmuslera]

"Many programmers put SQL code right on a web page" -- when was the last time you saw a select statement on a web page ?

I think the author mean in php/asp/other embedded languages pages, where finding plain sql code (instead of calling a library that hides for portability the sql code) is very common yet.

Of course, that don't display in the visitor browser, at least, not unless is intended (.phps) o "accidental" (the old ::$DATA that showed the .asp code), or is part of the debugging process (visible or in the html generated code)

XP is a manager's dream

[HarmlessScenery]

How else can you do a major cost cutting exercise (only buying half the PC's you actually need) and at the same time con your geeks into believing that you are following the leading edge of software dev ?

Re:XP is a manager's dream

[dismayed]

You need more computers overall, because you have development machines, then usually your own "personal area" where you can handle email and manage your personal life... unless I misread Planning XP. :(

Re:XP is a manager's dream

[Simon Lyngshede]

Actually managers hate XP. Time planning in XP is much harder than with traditional software development. In XP you can't say "We're done implementing in 4 months" Managers like development that follow the waterfall model, simply because time managing it more clear. Except form that fact that you only need half the PC's, there are no real cost cutting, not in larger projects at least.

Blindly following XP for every project you do is a bad idea. XP bad choise for many projects. Read "Extreme Programming Explained" and you will see that Kent Beck agrees. Also never comment on the glory of XP without knowing other software development technics.

Next week's review: X for Y

[Mirk]

Anyone else notice the incredible range of books with titles of the form X for Y recently? Extreme Programming for Web Projects. Lifecycle Management for Java. Python for Information Engineers. Buzzword Awareness for Techies. Cluefulness for Suits. There seems to be no end to this trend ...

Ignorant Review

[TheTomcat]

"("Many programmers put SQL code right on a web page" -- when was the last time you saw a select statement on a web page ?)"

I think the author was refering to something like this:

<html>blah blah blah
<?php
$sql = "select .... ";
db_query($sql);
?>
blah blah
</html>

and not actually displaying the SQL in the output of the page.. both are bad, but showing the user a query is worse..

Too bad the reviewer doesn't seem to know enough about the subject to actually catch on to this.

S

How much XP is there in the real world?

[kryzx]

I've had this in the back of my mind to submit as an "Ask Slashdot", but this is as good a place as any for it.

XP was all the rage for a little while there. There was talk of it everywhere, especially here. I read about it some and became very interested in it. I think many of the core ideas are valid, and it seems like it would be a fun way to develop.

Now that the hype has died down, my question is this: How many people out there are really doing XP? How much has this really caught on? Is it just a bunch of talk?

If you are actually doing XP, tell me a little about:
* the industry you are in
* the kind of project
* how it was done before
* what prompted you to make the switch to XP
* how that switch work and how long it took
* and how things have been since moving to XP
* do you know others doing XP, if so how many

Thanks for sharing your experience.

Re:How much XP is there in the real world?

[kryzx]

Thanks for the lead.

With the help of Google I found the article here:

http://www.informationweek.com/story/IWK20020111S0 046

Re:How much XP is there in the real world?

[orenmnero]

About two years ago I bagan using an XP style approach, or at least adopted many of the practices. Most notably Unit Tests with a Test First approach. Coded Functional Tests and Continuous Integration

To answer your questions one by one:

* the industry you are in

Financial Industry ( focusing on real-time trading application in the options, futures, and equity industries )

* the kind of project

Mostly applications that either provide infrastructure for doing electronic trading, or applications that implement particular trading strategies.

* how it was done before

The sorts of applications I would work on were often quick prototypes that were meant to exploit an opportunity in the market. This would often mean lots of shortcuts were taken because chances are the code would be thrown away anyway. Problem is that once in a while an application will become become succesful and grow out of one of these prototypes. Since little time was available for proper design, such applications generally became a maintenance nightmare as they kept growing. XP solved this problem by provideing a framework that allowed me to only design what I need when I need it, but enabled me to radically alter the design if ever necessary.

* what prompted you to make the switch to XP

I was exposed to it by some people who were working on a project at my company. I found many of the practices allowed me to create "prototypes" just as quickly as before, but with a higher degree of reliability and a framework that allowed them to grow as requirements changed. I was really sold on it when I saw how simply I was able to implement pieces of functionality that had previously just seemed more complicated.

Pair Programming gets a lot of flak. But really I fully credit it for getting me interested in these techniques. Because I generally paired with someone who knew these practices better than me, I was able to actively see the value of these practices while writing production code. It was far more of an eye opener than reading any book or article.

A lot has been said about Pair Programming that is just not true. It does NOT mean that there is one computer for every two people. Everybody has a machine. Just because you try to write all production code in pairs doesn't mean that everything is done in pairs. Also, you do not get paired up with someone like a ball an chain. Pairs switch, sometimes several times a day. Pairs are NOT assigned. If I need to write an ordermatching algorithm and I've never done it before, I'm going to try to pair with someone who has. By the time we are done with the task, we now have two people who know how to write an ordermatching algorithm. It's also untrue that one person is the designated typer while the other one just sits back and criticizes. Both people type, switching sometimes several times a minute while designing a class or method. Code is typed as a way to communicate concepts. If I want to convey the direction I think something should go, I write a little code. If the pair has another idea, he'll start tapping his ideas in the keyboard. It's a very actively collaborative process.

* how that switch work and how long it took

I haven't yet seen a project switch to an XP style process without problems, and from what I understand is that it is possible but slow. Part of the problem is that people who switch often do so from a less test-centric approach. This makes it difficult as XP is so reliant on aggressive testing to be succesful. It is also a new concept for many developers to take responsibility of some of the testing that has traditionally been soley in the domain of QA.

* and how things have been since moving to XP

Like I said, I have not been involved in a switch, but can speak about the difference I've seen between similar projects. One problem that I've solved in both a non-XP approach and an XP approach is in implementing the FIX protocol.

This protocol is relatively simple, but it is rife with various possibilities for boundry errors, off by one errors, and complicated resend logic. The big difference I found is that I never broke old logic when modifying the code. Generally when doing development, you are only testing the piece of functionality you are working on, so you get that working, but you break something else. This might not be discovered for quite some time. With an XP approach, while developing, you test everything you've ever tested. You find out immediately if your changes break old code.

Not only this, but despite the overhead of writing tests, I found functionality was implemented quicker, more reliably, and far simpler. Some of this was because I learned from previous mistakes, but primarily it was due to a different way to approach a problem.

* do you know others doing XP, if so how many

Yes. But they all do it a little differently. I have not seen two XP projects that look exactly the same. The core value I always see is the emphasis on repeatable tests.

Authors' Site

[Chocolate Teapot]

I just visited the authors' web site at Agile.net. I think it tells us everything we need to know about this book. The home page looks as if it has been through a shredder. Fortunately I have a better back button.

Not a bad book

[null etc.]

I've read this book recently, and must disagree with the reviewer's assessments.

Although I'm not a fan of extreme programming (it seems counter-intuitive to my highly structured mind), the aspects mentioned by this book have accurately reflected the last five years of experience I've had as lead architect and developer at a custom development firm.

Let me give an example. The reviewer condescends the book for assuming a "Strategist" role is necessary in a successful project, since the customer undoubtedly knows his or her own business.

In my experience, which may not be the gospel truth, but is valuable nonetheless, the customers often do not know their own business. The individuals of an organization sometimes know nothing more than the rote daily routine with which they've been guided over the years. Ask them an insightful question about why or how a process came to exist, and they might give contradictory or vague answers. It is the role of the strategist to exhume the truths and necessities of an organization, which are not always superficial or easily understood.

The reviewer also disbelieves that SQL code is ever embedded within web pages. Many quick and dirty (or under-engineered) web sites do use some form of embedded SQL, however. I'm often called in to clean up such sites, and make them more secure and modular.

The book is admittedly light on related topics, and perhaps a more academic treatment of extreme programming would have been more useful. And I do agree with the reviewer in that many statements within the book seem like advertisements for the author's own company.

Nonetheless, Extreme Programming is a practice understood by few (comparitively speaking), and this book serves as a good bridge between Extreme Programming and more structured development methodologies.

Re:Not a bad book

[GreyyGuy]

I agree with your assesment of needing a "Strategist" role. That is the role I take whenever someone comes to me with a new project. They may know their own business, but rarely understand automating it. They have a "process" hat mainly involves doing whatever seems right for that case- something very tricky to code. Or they want to collect data and generate reports that they don't have any real plan to use.

Or even worse, they expect that just putting their processes online will magically solve all their problems. So a "Strategist" is definately needed to help set goals and expectations.

Re:Not a bad book

[Zoop]

The reviewer condescends the book for assuming a "Strategist" role is necessary in a successful project, since the customer undoubtedly knows his or her own business.


Not only do customers often not know their own business (or at least haven't thought about it in any systematic way), they generally have no idea of a) what's possible, and b) what's cheap, and c) what's good on the web. They get really caught up in features.

I groan when I'm in a requirements meeting and I hear a client start, "Now, can there be a 'button' that..." Generally they get this image of a button and various form elements in their head, and don't really have the training to think it through, or even better, step back and think about what they're trying to do and what kinds of services can fit them--let the details work themselves out later.

Clients who have this kind of blinder focus on details tend to have more unrealistic expectations and greater disappointment when the magical mind-reading system doesn't appear. They are very frustrated with the process of fixing this system in development.

This isn't their fault. If this were easy, everybody would do...oh wait, everybody would do it well. They don't know that the system requires mind reading--they just assume several steps in between.

Your average developer, head deep in code, doesn't have the business process experience to chain the processes together and see the problem until they get into the code and realize somebody forgot Phase Two: ... (I suffer from this occasionally). Then they may not have the best people skills to break this to the client easily, and things get testy. YMMV, but I've found this to be true slightly more often than not.

Those who are really good, and they are rare, at Internet strategy tend to have experience in organizations (not necessarily businesses) and technology. Too little technology and they plunge off the cliff with the client (we have one of those in our company). This is why domain experts with tech experience are as valuable on Web projects as they are on traditional client-server projects.

Reliability is the point.

[Joseph Vigneau]

Somehow, I would never trust an "extreme programmed" program. I feel (perhaps just a prejudice) that extreme programming is a "do now, think later" kind of approach.

Then you don't understand the whole approach. XP evolved by taking good programming practices (automated testing, peer code review, "integration builds", client communication, etc.), and kind of going overboard with them. As a result, it typically generates higher quality deliverables. For example, in XP, before you write code, you write the test first. Instead of weekly code review meetings, you code with someone else. Instead of everyone writing their code in isolation, then slamming it together to see if it all works, you see if your code works together all the time. Instead of waiting for one or two business days to get a requirement clarification from the customer, you get that information from them much sooner.

There are more practices such as these that make up XP. I've worked on many projects, particularly with the big consultancies, that use waterfall-type methodologies that fail. A 600 page design document is useless when the requirements change during the project, which they always do, due to market demands, time constraints, etc. XP is no silver bullet, but it makes a lot of sense after you've been through lots of projects that go over budget, without all the desired features, even after working 80 hour weeks.

Re:Reliability is the point.

[pmz]

I've worked on many projects, particularly with the big consultancies, that use waterfall-type methodologies that fail.

There is generally no such thing as a true waterfall in a real project. When one occurs is usually an indication of design by committee and bureaucracy, where the project is doomed regardless of methodology.

I really think latching onto a branded methodology, such as XP or RUP or Waterfall or whatever, for a project is the wrong approach, because it is only good management that makes XP or RUP or Waterfall or whatever work in the first place. The methods themselves are for informational purposes only. They can influence a managerial approach, but anyone exclaiming, "We follow XP methodologies," just looks silly and will invoke resistence from the more cynical team members.

Good managers avoid thinking and speaking in terms of such buzzword methodologies, but they also freely admit that they learned something from those methodologies and applied what they learned to their own approach. Good arguments beat buzzwords when trying to convice the skeptics and cynics among us.

Risks

[joshsnow]

A traditional web project? Has the web been around that long!?
This book seems to assume that customers
a) Know what they want
b) Are capable of helping to formulate a realistic iterative dev plan
In my experience, customers tend to want fixed price projects because they know how much they'll pay up front, but then they also want changes made at a whim if they don't like something, they often expect the developers to be able to read their minds regarding ill defined requirements and they expect it all to be defined, developed and delivered yesterday.
The key to successful fixed price development is to make sure that the client understands their own requirements and understands that anything outside of that understanding is a Change Request for which they will pay over and above the originally decided project cost.
Regarding Extreme Programming - one cornerstone of XP seems to be two developers working at one machine developing one Unit together. You'd have a hard time trying to convice someone managing a fixed price project to sign up to this as instantly, your costs for each unit double (managers can't see time savings).
I think that if a customer can be persuaded to go Time and Materials and a realistic agreement can be made re: milestones between the developer and the customer, then you have the best of all worlds.

Optional Scope is a non-starter

[billtom]

Oh goody, another XP flame war on /. I might as well jump in the fire.

Well, I worked at a web shop for a few years (though, that was during the bubble, maybe things have changed now) and I looked into using XP because many aspects of it seemed to fit our needs. But one aspect always hung me up and that was that XP projects are basically optional scope contracts.

Basically, the clients always wanted to see the whole site (at least mockups) before they would sign off on the work. Even if we all knew that there was likely to be significant changes along the way.

Saying something like "let's get broad agreement on the general nature of the site and then work in iterations to refine the details. Now please sign this contract for three months of work for four developers" just didn't work.

Now, XP proponents will claim that this can be overcome by educating the client. I'll just say that that wasn't my experience. Optional scope contracts just wouldn't fly. Other XP proponents might say to hide the process from the client and make an XP project look, to the outside, like a waterfall with very flexible change management, but I wasn't happy with that sort of methodological dishonesty.

I think that this problem with optional scope is a problem not just with web sites but with any project where contractor and client are different companies. Most of the XP success stories I've heard are on projects where the client is an internal division of the same company, so things are a little less confrontational and more flexible.

Lazy developers?

[Tim Macinta]

This risk -- the authors contend -- is the reason many web development projects fail in one way or another. The client's objective is to obtain maximum value, the developer's to incur the least cost possible without getting sued.

That's a very pessimistic view of things. I know my objective is never to just squeek by with enough output to not get sued. My objective is usually to do a superior job and make the client happy so that I get repeat business and/or referrals. If the client has unrealistic expectations (which would lead to unfairly high time costs on my part), then I'm content to do work which at least I know is high quality and I don't lower the bar to to doing just a subsistence level of work. I really doubt that I'm alone on this. I would think most contract developers like the idea of repeat business and are hopefully clueful enough to realize that sub-par work does not encourage repeat business.

XP is the only way to program for the Web...

[seschmi]

...because it's the only way to finish the Website before the .com-Startup goes bancrupt.

This is not a joke.

But there is an important question here...

[sbrylow]

...and the reviewer captured it - I'd like some discussion on this point:

--- at least one of the parties taking a big risk on the project: if the project is 'fixed price, fixed scope' the developers take all the risk, if it's 'time & materials' the customer takes a risk---

I've seen plenty of this and I believe it's nearly always true. And I think the reviewer is correct in stating that this is not unique to web projects.

So, with that in mind, I'll assert that it would be overall _more efficient_ (less waste of money and resources) if both parties were able to manage uncertainty and risk in projects in a less adversarial manner.

Call me on that assertion if you want, but risk management is an important part of managing the software development process for just that reason. So, why _not_ come up with a better way to manage risk across organizations??

I don't think contracts are bad things (just the opposite) and I don't have The Answer...but I'm imagining a better contractural toolkit and a better set of development and project methodologies that allow some uncertainty and flexibility and assigns risk at a more granular level than 100%-0% or vice versa...

For an analogy that's pretty far afield, I saw a report recently that said something like 50% of mergers and acquisitions fail to add value, but if the M&A was contested or if there were multiple bidders, it goes up to 70-80%. Demonstrating, I think, that while people enter into contracts freely, those entities or contracts that are more adversarial are more wasteful of resources ;^)

OK, let's all get out there and fix the consulting business so it's more fun to work on projects for clients...:^) Comments??

RUBBISH !!!!!

[Anonymous Coward]

"Extreme" Programming must be one of the most idiotic buzz words to ever come out of the software industry. And considering that the industry has a pretty strong track record on idiotic buzz words, the competition is pretty tough in this respect.

Basically, "Extreme" (God - it makes me cringe just to type it!) programming means "doing it right" ! ie - writing a bit of code that (a) works, (b) ...err ...works ? and (c) ...oh dear ...works !

ie - it's meaningless bollox.

Test-First is too simplistic

[SuperKendall]

I am involved in a multi-year cleanup of a system that originated with an XP approach, and test-first design.

To start with, let me say that the XP approach utterly failed in this case, but it was probably in good part to the people we were working with still learning Java at the time and also having terrible design skills (or rather I should possibly say, design experience in other languages but not in Java which led to ill-fitting design). So, they probably would have generated drek no matter what approach they used.

However, as I was around from the start of this project I was able to make some interesting observations. The first is that you are correct, if you build code that passes all tests but still does not do what you want, then your requirements used to build the test are wrong. However, I've always been confused by this aspect as I thougt a requirement-poor environment was supposed to be where XP was helpful...

The larger issue I have noticed through my own experience as well as this project was that test-first is too simplistic a strategy. Instead I would break testing into two sorts - "scaffolding" and "regression".

The original project did not break up tests in this manner. As a result, the core of the project became encrusted in tests. After a while more work was being done on tests than on the actual project... at which point the buisness owner for the project raised hell, and testing was dropped altogether. Obviously that was a bit too far, but it did get the project moving, and ended in a state where it worked (though I still have a terrible mess to clean up and a very fragile system).

Back to the breaking out of tests... I think they would have been much better served by "scaffolding" tests that are created during construction of a system, but ultimatley are thrown away when the system works well so you do not have the overhead of maintaining tests while adding to your system. Then of course there would be true "regression" tests that managed to cover most of the application testing, and ensure large portions still worked after changess... but they would be few enough in number that enhancements of fixes would not mean more work fixing tests than code.

I've used a scaffolding approach in my own work and designs, and find it woorks really well. It gives you enough tests to get much of the touted test-first benefits, but does not leave you with a system that cannot be changed for fear of cascade fixes on multiple, ancient, tests.

I would be interested to hear what criteria other people use to abandon a test when doing test-first programming.

An aside

[JSkills]

Regardless of anyone's predisposition to XP as a development methodology or not (I actually love some of its philosophies, but would never follow some of its others), it's really not the main factor that has to do with the success of a given project.

Successful software development has always been about people - not development philosophies, and not about which language you use. If you're working with people who are winners, you're going to find a way to succeed.

That said, I'm sure there are many who feel very strongly about operating systems, development environments, programming languages, etc. (myself included), but none of it matters if you don't have the right talent in place.

Just my master-of-the-obvious aside ...

Extreme Programming brings teamwork to the cubicle

[rpiquepa]

As I wrote in this column, extreme programming is not really new. "Extreme Programming (XP)" is just another way of saying "Team -- or pair --programming". Programming in pairs is the most difficult aspect for many to accept (believe me). Even for XP die-hards like Edward Hiett, who works for San Francisco-based Evant, programming with someone looking over your shoulder remains disconcerting. ``Programming is a very creative process and requires a lot of concentration. It's natural to want to go away and do it by yourself,'' says Hiett, , where all programmers work in pairs. ``With pairing, you have to give up control.''

Tests

[SimonK]

I've found test driven development very useful on my most recent project, and intend to practise it in future too.

On this occasion, I did what you suggest: I built unit tests during development and threw them away when they were costing more time to maintain than they were saving in bugs that would not have been found another way. Once it was possible, I developed tests from the user's API level (its a middleware product), and these will be maintained for the life of the project. These API level tests include regression tests for specific bugs.

The XP folks seem to suggest maitaining unit tests at a level I consider excessive. I think they suggest one test per non-trivial method on all classes. This just seems too much, and very hard to achieve, since even in the best designed projects, individual methods are hard to test without also testing a bunch of other stuff.

It sounds like your project was suffering mainly from a lack of design skills. Spending a lot of time maintaining tests implies too much coupling between components: the only way that can come about is if changing one component affects many others, so there tests need updating too.

Re:Tests

[superwombat]

Excessive unit tests are not the problem. Code duplication was probably the problem. Very critical for XP are the ideas of OOAO (Once and Only Once) and Refactoring (www.refactoring.com is a good site on this). With out these XP will create huge messes.