Slashdot Mirror
Agile Software Development with Scrum
Extreme Programming (AKA XP) has been an interest of mine for some time, as I struggle to find ways to make it easier to say "yes" to in my domain (embedded systems) and it is in the study of this other development process that I first heard of Scrum. Both XP and Scrum are development processes under the "umbrella" of the Agile Alliance.
First and foremost, let's cover what Scrum is, as presented in this book. I will compare Scrum with XP as I go, for reasons that will become obvious later on.
You have the Scrum Master, who is more or less half technical lead and half project manager. He is defined as being responsible for the success of Scrum. I think the closest thing that XP defines to this is the Coach, but it's a poor fit at best.
You have the Product Backlog, where all the stuff that people want in the product is written (called Work) along with assigned priorities for each item and an estimate of some form indicating how long it will take to do. The assumption is that items with high priority will have more accurate estimates and more precise specifications and the low priority stuff will be more or less SWAGs. The Product Backlog also contains Issues, which are more or less problems that gate one or more Work items. Issues are turned into Work by the Product Owner (see below) at his discretion. The analogous concepts in XP is the Story and the Tasks.
You have the Product Owner. He owns the Product Backlog document and the prioritizations within. While he may need to consult others in the effort to do his job, it is his responsibility to maintain the list of work and issues, to decide the prioritizations and to decide what work will be done in the next Sprint (see below). The Product Owner also owns the estimates in the Product Backlog. It is expected that he consults with development to derive these numbers. These estimates are said not to be binding on the Scrum Team (see below). The XP role akin to this is the Customer, with the exception that, in XP, estimation comes solely from development.
The Scrum Team is just the set of technical people working on the product. So testers, documentation people, developers etc. Should aim to be around seven people (i.e. the "optimal" group size).
The Sprint is the development iteration. It should typically aim to be 30 days. It has a Sprint Goal, which defines the overall objective of the Sprint (think of it as a mission statement just for the iteration), and a set of items from the Product Backlog to develop during the iteration. These items are chosen during the Sprint Planning Meetings mainly by the Product Owner. A secondary meeting is held with just the Scrum Team and Scrum Master to decide who will do what and in what order within the Sprint. Also done in this meeting is the breakdown of Work to Tasks, which are smaller units no longer than 4 to 16 working hours in duration. The XP Sprint equivalent is simply the iteration. To my knowledge, there is no equivalent concept to the Sprint Goal. XP's Planning Game does the work of both the Sprint meetings noted earlier.
Because the duration of the Sprint is respected first and foremost, the Sprint Goal is used to determine what content to remove from the Sprint in the event it is discovered that the deadline is at risk. So, we move work from the Sprint to the Product Backlog while attempting to honor the Sprint Goal in order to satisfy our (e.g.) 30 day schedule. If that is not possible (and in some other situations, like irresolvable organizational impediments) the Sprint is canceled and redone from the Planning Meeting.
You have the Sprint Signature, which tracks the expected Work done against the actuals and provides a day-to-day description of what happened in the Scrum Team (similar to, nut much smaller than, the project log spoken of by Steve McConnell).
Once the Sprint is on, no one outside the Scrum team has anything to do with the Scrum team. XP, again AFAIK, does not state this, but the fact that iterations are, if anything, even shorter than Sprints means that in all but the worst cases, the stakeholders will likely be willing to wait until the next iteration to redirect the effort. In any event, because both processes return so frequently to the customer for guidance and because both processes allow the customer to introduce change throughout the lifecycle,, there is less risk that the customer will feel their wishes are not being respected. Lastly, XP's customer and Scrum's Product Owner both funnel all requirements to development. All the project stakeholders then direct their requests to this person. In so doing, the stakeholders never get told "no" per se, but rather are told that at worst their request won't be evaluated until the next iteration/Sprint planning meeting.
At the daily Scrum Meeting, each Scrum Team member answers the following questions:
- What did you do since the last Scrum Meeting?
- What will you do between now and the next Scrum Meeting?
- What, if anything, impeded your progress?
So, you say how you did yesterday against your commitment from yesterday. You say what you plan to do today. All this is recorded in the Signature which is kept by the Scrum Master. Lastly, you note any obstacles that got in your way. It is the Scrum Master's job to note these and remove them immediately. The Scrum Master is expected to report on his progress in this area at each meeting as well. If the obstacle is a failure to make a decision, the Scrum Master is responsible for making that decision immediately - failing that, to ensure it gets made that day. I really like that, BTW, as experience has made clear to me that, on balance, the risk of making the wrong decision has a much lower negative effect on a team than does leaving it in limbo frequently. In the Scrum meeting, no one is allowed to speak save for the Scrum Team and Master. Others may attend but may not speak. This is intended to keep to meeting focused and short. I like this part but suspect that to do it successfully, you'd be wise not to invite anyone that's not intended to speak.
The Sprint Review is held at the Sprint's end and is intended to be a half-day forum where the Scrum Team presents to the stakeholders what it accomplished during the Sprint. The loose equivalent in XP is the final (for that iteration) successful execution of the Customer Tests (nee Acceptance Tests). I say "loose" because the XP concept is stronger - the iteration's deliverable is not only presented but accepted.
To get the Scrum process to scale, a Scrum of Scrums is used where Scrum Masters from multiple Scrum Teams attend a Scrum Meeting run by a Scrum uber Master.
So, AFAICT, that's pretty much it.In reality, I don't see a whole book's worth of content coming out of this.
So what else do they discuss? There's a fair bit on work environments, the ill effects of heavy weight processes and so on - basically restating stuff in less detail that most everyone who will read this book would be likely to have read elsewhere.
There's a lot on the characterization of Scrum in terms of Process Control Theory and why it would be expected to be a better fit for software development than would other processes. This left me cold, frankly, because I tend to read these books looking for what to do, how to do it and what benefit I should expect to see. The underlying science of it is of some interest but not nearly so important to me as the answers to those three questions. Also I already implicitly favor these iterative approaches. So do many, many others - after all, the evolutionary lifecycle model and the "mini-milestones" proposed by McConnell in "Rapid Development" echo both XP and Scrums concepts.
We also get stuff like this:
"Overlapping development phases: In an environment where some of the requirements are discovered while simultaneously something is created with the information at hand, it is imperative that the phases of discovery, invention, and testing overlap to drive the creation of a new product to completion through self-consistency. Most problems in new product development arise when the phases of the project are separated. Empirically, this overlap in phases enhances shared responsibility and cooperation, stimulates involvement and commitment, sharpens a problem-solving focus, encourages initiative taking, develops diversified skills, and heightens sensitivity toward market conditions."Other than "don't use the waterfall lifecycle," what exactly does all that mean? The end of it is completely unsubstantiated. It drives me nuts when development books do things like this. That paragraph doesn't say much of anything AFAICT but nonetheless manages to set expectations way too high.
Here's another (about the psychological effects of Scrum on the Scrum team):
"They become deeply involved in their work. Scrum drives individuals to focus, commit and excel.I don't know what any of that means and I'd be scared to find out.
They focus on the work and lose concern for themselves.
They experience an altered sense of time.
They consistently produce at high levels of accomplishment.
Scrum allows developers to concentrate most of their time in developing software, and by doing so developers enter 'flow' state."
Out of nowhere, you get statements like this:
"Scrum requires a balance of individuals with at least 50% of them to be experts ..."
Well, I guess that's that then. I think I could define a process which would have demonstrably improved efficiency over the lifecycle with just that one statement - worse comes to worst just sack the other half and away you go :) Anyway, this requirement appears for the first time on page 121 in a book with all of 154 pages. It's hard to say if it's intended to be taken seriously - if it were, you'd think it would have been a lot more front and center.
But let's go back to what Scrum is, as shown earlier. IMHO, it is not a lot more than XP with at least the following removed (this from memory, please don't be too harsh):
- Continuous integration
- Refactoring
- Unit testing
- Paired programming
- Collective ownership
- Customer-defined acceptance tests
- The "yesterday's weather" estimation practice
- Sustainable pace (nee "the 40 hour work-week")
The book is careful to point out that any and all of these "missing" practices (refactoring, unit testing et al) may be used but that Scrum does not prescribe them. And that's fine, but I'm evaluating it on what it actually does prescribe - if Scrum can take credit for what it does not prescribe, they it can lay claim to infinite credit after all. This is maybe a small point but it's important to what follows. Because Scrum does not tell us what to do in these areas, I'm assuming that they are free to vary.
So, if you accept my characterization of Scrum as XP with all the hard technical bits removed. what might you see in a Scrum project?
If you follow Scrum's iterative path with no refactoring to offset the inevitable software entropy, the software would tend to lose architectural unity quixkly. It would be easy to break and hard to fix. - and so on and so on, for all the ills which refactoring is intended to address. If Scrum does not prescribe refactoring explicitly, then we must assume that not doing it is an acceptable Scrum path. If Scrum requires refactoring as a key practice, it should say so. XP and Scrum turn every project into a maintenance project but XP bolsters that position with all the technical practices that ensure the team is *really* good at maintenance. Scrum does none of this. Combine the lack of refactoring with no prescribed regression (unit) testing and I think it becomes clear that Scrum projects that do neither would risk devolving into code and fix affairs.
Because we have no review process prescribed in Scrum, which XP provides via paired programming, and no stated ownership model, we would expect to see Scrum projects which explicitly prescribe neither ending up with code like little fiefdoms where the divisions between one guy's code and another's are painfully obvious. We also would expect to see de facto individual code owners with all the gone-to-greener-pastures risk that entails.
Because we have no estimation practice prescribed we would expect to see poor estimates. Because the Product Owner owns these estimates, there's no reason to expect them to be meaningful at all.
And in return for all this, what do we get? We get the Sprint Goal, which is a fine idea, IMHO. It defines, a priori, a way to sensibly cut work within an iteration. We get the fast decision-making from the Scrum Master. This is also fine by me. We get the intense project progress tracking of the daily Scrum Meeting and the Sprint Signatures. Hmmm ....
And so, at long last, and very much in my long-winded way, I'm finally ready to state my real conclusion about Scrum ...
But first here's another quote from the book :) It's an example Sprint Signature description:
Day 16-17 The team is discouraged by all the work remaining and didn't work during the weekend. No changes in estimated time remaining.
Day 18 The team works more and its remaining work declined. The team then met with the Product Owner and the Scrum Master to determine what tasks could be reduced or removed while still meeting the goals of the Sprint. Some Sprint backlog was dropped; other estimates were lowered because not as much functionality had to be supported. Overall estimated work remaining reduced to 1400 hours. If all this work is completed, the team will still meet the Sprint Goal, although with functionality implemented less completely.
Day 19 The team continues work toward the Sprint Goal using the new Sprint Backlog. Estimated work remaining declines.
Day 20-30 Team is motivated because it can still meet the Sprint Goal if it works hard. The team works regularly including during the weekends. Estimated work remaining declines to zero as the team meets its Sprint Goal by the 31st day."
Here's one thing I've learned in the working world - death marches don't just happen on their own. It's not easy keeping a real death march going. You need to brutally cut and slash the work items in the finest detail to ensure that what you're committing to is actually what's absolutely necessary. To do this, you need to get stakeholders interested enough to grovel around in these nasty details. To really run a death march (and sustain one), you need to track everything everyone's doing down to the granularity of hours so you can tell immediately if someone has returned to a normal level of effort (people will tend to do this long before their actual *hours* spent at work decline). You need to make this tracking of effort very public so people feel intense peer pressure to keep their effort as high as their colleagues. You need to keep the goal always seemingly in reach and make sure this perception is held by everyone on the team because once the expectations are obviously unrealistic, most people will slack off. As long as the goal looks possible with significant effort, people will still buy into it.
And, in the end, that's what I think of Scrum: if you take away all those XP practices and don't replace them with anything, I think you end up with a really good way to run a code and fix death march. And if you put back all those practices, you're pretty much doing XP. I believe parts of Scrum can be lifted and applied in other processes (particularly XP) but that you'd be insane to adopt Scrum as defined in this book without addressing the risks detailed above."
You can purchase Agile Software Development with Scrum from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
I've always worked on projects with SCUM--so that's what the problem was!