Slashdot Mirror

← Back to Stories (view on slashdot.org)

Agile Software Development with Scrum

Posted by timothy on from the moistened-bint-lobbing-scimitars dept.

bucketman writes "Saw this book at Chapter's and bought it and read it straight away. I've been kind of sitting on it for a month or two because my reaction to this book was so strong that I kind of wanted to see if time would mellow it. Well, it hasn't and I'm ready to post now. Note that this book has already been reviewed on Slashdot once, but I've decided to send in my review anyway, as it presents more detail as well as an alternate point of view and might be of interest." Read on below for bucketman's take on the book (and the methodology). Agile Software Development with Scrum author Ken Schwaber and Mike Beedle pages 158 publisher Prentice Hall rating 1 reviewer bucketman ISBN 0130676349 summary Presents the Scrum software development process.

Extreme Programming (AKA XP) has been an interest of mine for some time, as I struggle to find ways to make it easier to say "yes" to in my domain (embedded systems) and it is in the study of this other development process that I first heard of Scrum. Both XP and Scrum are development processes under the "umbrella" of the Agile Alliance.

First and foremost, let's cover what Scrum is, as presented in this book. I will compare Scrum with XP as I go, for reasons that will become obvious later on.

You have the Scrum Master, who is more or less half technical lead and half project manager. He is defined as being responsible for the success of Scrum. I think the closest thing that XP defines to this is the Coach, but it's a poor fit at best.

You have the Product Backlog, where all the stuff that people want in the product is written (called Work) along with assigned priorities for each item and an estimate of some form indicating how long it will take to do. The assumption is that items with high priority will have more accurate estimates and more precise specifications and the low priority stuff will be more or less SWAGs. The Product Backlog also contains Issues, which are more or less problems that gate one or more Work items. Issues are turned into Work by the Product Owner (see below) at his discretion. The analogous concepts in XP is the Story and the Tasks.

You have the Product Owner. He owns the Product Backlog document and the prioritizations within. While he may need to consult others in the effort to do his job, it is his responsibility to maintain the list of work and issues, to decide the prioritizations and to decide what work will be done in the next Sprint (see below). The Product Owner also owns the estimates in the Product Backlog. It is expected that he consults with development to derive these numbers. These estimates are said not to be binding on the Scrum Team (see below). The XP role akin to this is the Customer, with the exception that, in XP, estimation comes solely from development.

The Scrum Team is just the set of technical people working on the product. So testers, documentation people, developers etc. Should aim to be around seven people (i.e. the "optimal" group size).

The Sprint is the development iteration. It should typically aim to be 30 days. It has a Sprint Goal, which defines the overall objective of the Sprint (think of it as a mission statement just for the iteration), and a set of items from the Product Backlog to develop during the iteration. These items are chosen during the Sprint Planning Meetings mainly by the Product Owner. A secondary meeting is held with just the Scrum Team and Scrum Master to decide who will do what and in what order within the Sprint. Also done in this meeting is the breakdown of Work to Tasks, which are smaller units no longer than 4 to 16 working hours in duration. The XP Sprint equivalent is simply the iteration. To my knowledge, there is no equivalent concept to the Sprint Goal. XP's Planning Game does the work of both the Sprint meetings noted earlier.

Because the duration of the Sprint is respected first and foremost, the Sprint Goal is used to determine what content to remove from the Sprint in the event it is discovered that the deadline is at risk. So, we move work from the Sprint to the Product Backlog while attempting to honor the Sprint Goal in order to satisfy our (e.g.) 30 day schedule. If that is not possible (and in some other situations, like irresolvable organizational impediments) the Sprint is canceled and redone from the Planning Meeting.

You have the Sprint Signature, which tracks the expected Work done against the actuals and provides a day-to-day description of what happened in the Scrum Team (similar to, nut much smaller than, the project log spoken of by Steve McConnell).

Once the Sprint is on, no one outside the Scrum team has anything to do with the Scrum team. XP, again AFAIK, does not state this, but the fact that iterations are, if anything, even shorter than Sprints means that in all but the worst cases, the stakeholders will likely be willing to wait until the next iteration to redirect the effort. In any event, because both processes return so frequently to the customer for guidance and because both processes allow the customer to introduce change throughout the lifecycle,, there is less risk that the customer will feel their wishes are not being respected. Lastly, XP's customer and Scrum's Product Owner both funnel all requirements to development. All the project stakeholders then direct their requests to this person. In so doing, the stakeholders never get told "no" per se, but rather are told that at worst their request won't be evaluated until the next iteration/Sprint planning meeting.

At the daily Scrum Meeting, each Scrum Team member answers the following questions:

  • What did you do since the last Scrum Meeting?
  • What will you do between now and the next Scrum Meeting?
  • What, if anything, impeded your progress?

So, you say how you did yesterday against your commitment from yesterday. You say what you plan to do today. All this is recorded in the Signature which is kept by the Scrum Master. Lastly, you note any obstacles that got in your way. It is the Scrum Master's job to note these and remove them immediately. The Scrum Master is expected to report on his progress in this area at each meeting as well. If the obstacle is a failure to make a decision, the Scrum Master is responsible for making that decision immediately - failing that, to ensure it gets made that day. I really like that, BTW, as experience has made clear to me that, on balance, the risk of making the wrong decision has a much lower negative effect on a team than does leaving it in limbo frequently. In the Scrum meeting, no one is allowed to speak save for the Scrum Team and Master. Others may attend but may not speak. This is intended to keep to meeting focused and short. I like this part but suspect that to do it successfully, you'd be wise not to invite anyone that's not intended to speak.

The Sprint Review is held at the Sprint's end and is intended to be a half-day forum where the Scrum Team presents to the stakeholders what it accomplished during the Sprint. The loose equivalent in XP is the final (for that iteration) successful execution of the Customer Tests (nee Acceptance Tests). I say "loose" because the XP concept is stronger - the iteration's deliverable is not only presented but accepted.

To get the Scrum process to scale, a Scrum of Scrums is used where Scrum Masters from multiple Scrum Teams attend a Scrum Meeting run by a Scrum uber Master.

So, AFAICT, that's pretty much it.In reality, I don't see a whole book's worth of content coming out of this.

So what else do they discuss? There's a fair bit on work environments, the ill effects of heavy weight processes and so on - basically restating stuff in less detail that most everyone who will read this book would be likely to have read elsewhere.

There's a lot on the characterization of Scrum in terms of Process Control Theory and why it would be expected to be a better fit for software development than would other processes. This left me cold, frankly, because I tend to read these books looking for what to do, how to do it and what benefit I should expect to see. The underlying science of it is of some interest but not nearly so important to me as the answers to those three questions. Also I already implicitly favor these iterative approaches. So do many, many others - after all, the evolutionary lifecycle model and the "mini-milestones" proposed by McConnell in "Rapid Development" echo both XP and Scrums concepts.

We also get stuff like this:

"Overlapping development phases: In an environment where some of the requirements are discovered while simultaneously something is created with the information at hand, it is imperative that the phases of discovery, invention, and testing overlap to drive the creation of a new product to completion through self-consistency. Most problems in new product development arise when the phases of the project are separated. Empirically, this overlap in phases enhances shared responsibility and cooperation, stimulates involvement and commitment, sharpens a problem-solving focus, encourages initiative taking, develops diversified skills, and heightens sensitivity toward market conditions."
Other than "don't use the waterfall lifecycle," what exactly does all that mean? The end of it is completely unsubstantiated. It drives me nuts when development books do things like this. That paragraph doesn't say much of anything AFAICT but nonetheless manages to set expectations way too high.

Here's another (about the psychological effects of Scrum on the Scrum team):

"They become deeply involved in their work. Scrum drives individuals to focus, commit and excel.
They focus on the work and lose concern for themselves.
They experience an altered sense of time.
They consistently produce at high levels of accomplishment.
Scrum allows developers to concentrate most of their time in developing software, and by doing so developers enter 'flow' state."
I don't know what any of that means and I'd be scared to find out.

Out of nowhere, you get statements like this:

"Scrum requires a balance of individuals with at least 50% of them to be experts ..."

Well, I guess that's that then. I think I could define a process which would have demonstrably improved efficiency over the lifecycle with just that one statement - worse comes to worst just sack the other half and away you go :) Anyway, this requirement appears for the first time on page 121 in a book with all of 154 pages. It's hard to say if it's intended to be taken seriously - if it were, you'd think it would have been a lot more front and center.

But let's go back to what Scrum is, as shown earlier. IMHO, it is not a lot more than XP with at least the following removed (this from memory, please don't be too harsh):
  • Continuous integration
  • Refactoring
  • Unit testing
  • Paired programming
  • Collective ownership
  • Customer-defined acceptance tests
  • The "yesterday's weather" estimation practice
  • Sustainable pace (nee "the 40 hour work-week")
So, all the hard/useful technical bits from XP are removed. In return, we get the Daily Sprint Meeting, the Sprint Goal, the Sprint Signature, and the emphasis on decision-making by the Scrum Master. The remainder of the Scrum practices have analogues in XP, as I've shown above. It's for this reason I found myself smiling when the book notes success in industry in wrapping XP in Scrum - from what I can see, XP wrapped in Scrum *is* XP :)

The book is careful to point out that any and all of these "missing" practices (refactoring, unit testing et al) may be used but that Scrum does not prescribe them. And that's fine, but I'm evaluating it on what it actually does prescribe - if Scrum can take credit for what it does not prescribe, they it can lay claim to infinite credit after all. This is maybe a small point but it's important to what follows. Because Scrum does not tell us what to do in these areas, I'm assuming that they are free to vary.

So, if you accept my characterization of Scrum as XP with all the hard technical bits removed. what might you see in a Scrum project?

If you follow Scrum's iterative path with no refactoring to offset the inevitable software entropy, the software would tend to lose architectural unity quixkly. It would be easy to break and hard to fix. - and so on and so on, for all the ills which refactoring is intended to address. If Scrum does not prescribe refactoring explicitly, then we must assume that not doing it is an acceptable Scrum path. If Scrum requires refactoring as a key practice, it should say so. XP and Scrum turn every project into a maintenance project but XP bolsters that position with all the technical practices that ensure the team is *really* good at maintenance. Scrum does none of this. Combine the lack of refactoring with no prescribed regression (unit) testing and I think it becomes clear that Scrum projects that do neither would risk devolving into code and fix affairs.

Because we have no review process prescribed in Scrum, which XP provides via paired programming, and no stated ownership model, we would expect to see Scrum projects which explicitly prescribe neither ending up with code like little fiefdoms where the divisions between one guy's code and another's are painfully obvious. We also would expect to see de facto individual code owners with all the gone-to-greener-pastures risk that entails.

Because we have no estimation practice prescribed we would expect to see poor estimates. Because the Product Owner owns these estimates, there's no reason to expect them to be meaningful at all.

And in return for all this, what do we get? We get the Sprint Goal, which is a fine idea, IMHO. It defines, a priori, a way to sensibly cut work within an iteration. We get the fast decision-making from the Scrum Master. This is also fine by me. We get the intense project progress tracking of the daily Scrum Meeting and the Sprint Signatures. Hmmm ....

And so, at long last, and very much in my long-winded way, I'm finally ready to state my real conclusion about Scrum ...

But first here's another quote from the book :) It's an example Sprint Signature description:

Day 16-17 The team is discouraged by all the work remaining and didn't work during the weekend. No changes in estimated time remaining.

Day 18 The team works more and its remaining work declined. The team then met with the Product Owner and the Scrum Master to determine what tasks could be reduced or removed while still meeting the goals of the Sprint. Some Sprint backlog was dropped; other estimates were lowered because not as much functionality had to be supported. Overall estimated work remaining reduced to 1400 hours. If all this work is completed, the team will still meet the Sprint Goal, although with functionality implemented less completely.

Day 19 The team continues work toward the Sprint Goal using the new Sprint Backlog. Estimated work remaining declines.

Day 20-30 Team is motivated because it can still meet the Sprint Goal if it works hard. The team works regularly including during the weekends. Estimated work remaining declines to zero as the team meets its Sprint Goal by the 31st day."

Here's one thing I've learned in the working world - death marches don't just happen on their own. It's not easy keeping a real death march going. You need to brutally cut and slash the work items in the finest detail to ensure that what you're committing to is actually what's absolutely necessary. To do this, you need to get stakeholders interested enough to grovel around in these nasty details. To really run a death march (and sustain one), you need to track everything everyone's doing down to the granularity of hours so you can tell immediately if someone has returned to a normal level of effort (people will tend to do this long before their actual *hours* spent at work decline). You need to make this tracking of effort very public so people feel intense peer pressure to keep their effort as high as their colleagues. You need to keep the goal always seemingly in reach and make sure this perception is held by everyone on the team because once the expectations are obviously unrealistic, most people will slack off. As long as the goal looks possible with significant effort, people will still buy into it.

And, in the end, that's what I think of Scrum: if you take away all those XP practices and don't replace them with anything, I think you end up with a really good way to run a code and fix death march. And if you put back all those practices, you're pretty much doing XP. I believe parts of Scrum can be lifted and applied in other processes (particularly XP) but that you'd be insane to adopt Scrum as defined in this book without addressing the risks detailed above."

You can purchase Agile Software Development with Scrum from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

8 of 166 comments (clear)

  1. first by Anonymous Coward · · Score: -1, Offtopic

    postus

  2. SCROTUM CUM! YEAH! by Anonymous Coward · · Score: -1, Offtopic

    Word.

  3. LATEST FROM MACWORLD CONFERENCE by Anonymous Coward · · Score: -1, Offtopic

    - A big request is the go through photos right when imported. rate them or discard or adjust them while importing.
    - New album - "last 12 months". Can change it to 9 months or whatever... customizable. And you can have last two rolls or last three rolls etc... Smart Album: Example - all holiday photos, done by "date range between 12/20 and 1/1". Ratings also. 1-5 stars. Smart album - like top rated photos.
    - Over 25,000 photos. Showing how he can scroll through them quickly. Resize them. (fast). Really Really nice. Very responsive. New effects: edit mode. Sepia finallly. Another cool thing: time based organization. In the photo library, you can now open the library and automatically gives u photos by calender year.
    - Demo of iPhoto
    - iPhoto: Time based organization, Smart albums (like smart playlists), Rate photos, enhanced slideshows, and rendezvous photo sharing.
    - Includes the latest and greatest of iTunes. Everything works together. iTunes works seemlessly with the other apps. iPhoto - we've all got a few thousand photos now. We wanted to get ahead of that. so the #1 feature is that it supports up to 25,000 photos with blazing performance
    - Next - iLife. We launched iLife a year ago. A lot of the apps we lanched before that but we pulled them all together into a suite. Today, we are launching iLife '04. "for the rest of your life"
    - So that is iTunes.
    - Pepsi and Apple are going to give away 100 million songs legally starting on Feb 1. Pepsi bottles (300mil) with specially marked caps. There will be a special pepsi button on iTunes and get a free credit from the specially marked caps. Promo runs for 60 days.
    - We have now got 500,000 songs available today on iTunes. That makes iTunes the largest online music stores in the world
    - Over 120,000 classical tracks today added.
    - New features: Billboard Charts (hot 100 charts) from 1946 to present. More billboard charts to be added for various genres. Available today.
    - iTunes Essentials - collections of essential songs
    - AOL Integration.
    - Gift certificates. Over 100,000 gift certificates sold
    - AudioBooks. Pleased to announce we sold over 50,000 audio books in the first quarter.
    - Who was the top customer? The top spender spent $29,500.
    - #1 in songs and revnue on every platform in the world
    - Marketshare data from SoundScan. iTunes = 70% marketshare.
    - The Gap is getting bigger and bigger (compared to competitors)
    - Now, iTunes. Pleased to announce we've crossed our 30millionth song.
    - $3/gigabyte. Far cheaper than other options
    - Xserve RAID certified on two versions of linux and Windows XP Pro and Windows 2003 Server.
    - We've qualified a bunch of Fibrechannel switches.
    - Xserve RAID: 3U. Added new. 3 1/2 Terrabytes. SFP connectors. RAID Set slicing.
    - We thought we ought to also update Xserve RAID.
    - $2999 - single 1GHz. $3999 Dual 2.0GHz.
    - Panther Server 10.3 - unlimited client license
    - Now What they probably wished they could have used were Xserves... but xserves only had G4s in them until today. G5 Xserve. two air vents in the front. 1U. Single and Dual 2.0GHz Processors. ECC Memory. DDR 400 up to 8GB. Up to 750GB of storage. 3 hot pluggable drive modules. Full complement of comm ports.
    - Jobs: They looked at everything intel etc... and they fell in love with the G5 and Mac OS X. We expect a few more popping up.
    - Assembled in 3 weeks. #3 computer in the world. first academic machine to break 10 Teraflop range ceiling
    - Showing Video from Virginia Tech talking about the PowerMac G5
    - Can't talk about the G5 without talking about Virginia Tech and their Supercomputer.
    - Steve Jobs. So, these are just a few of the apps that have taken us to 10,000 native OS X apps. Next up... the G5
    - Microsoft Rep leaves stage
    - MS Technology Guarantee Program. Of you buy a copy of Office 10, you get a free upgrade to Office 2004 when available in the spring
    - Demo of Office complete.
    - Can share projects with others. On a server/idisk.
    - New feature that's integrated

  4. 2nD p0s7 by Anonymous Coward · · Score: -1, Offtopic

    omg hi2u

  5. Aha! by Anonymous Coward · · Score: 0, Offtopic

    I've always worked on projects with SCUM--so that's what the problem was!

  6. I HAVE A GREASED UP YODA DOLL SHOVED UP MY ASS!!!! by Anonymous Coward · · Score: -1, Offtopic

    GO SCO!

  7. LATEST FROM MACWORD CONFERENCE by Anonymous Coward · · Score: -1, Offtopic

    - Demoing iDVD
    - A big request is to create and archive on non-Superdrive machine. Can now do that.
    - We couldn't update iMovie without updating iDVD. 20 new high quality themes, plus old ones. enhanced slide shows, improved navigation app. pro encoding from Final Cut Pro. Dramatically better quality - up to 2 hours encoding now!
    - Sharing. We want to put that on our website. Even easier. Select clip, just go to share and you pick homepage and choose selected clip only. Will auto-convert to something appropriate for the web and upload it
    - Directly imports iSight video
    - Can now trim clips in place. non desctructive. can go back. Audio scrub
    - Next Up: iMovie. iMovie 4 - can trim clips directly in the timeline. this is going to save alot of work. alignment guides added. new and enhanced titles. can import video directly from an isight camera. can share movies easier. now demoing
    - Photo Books available in Europe in March. Japan later this month
    - That is the new iPhoto 04
    - Rendezvous photosharing. demos photo sharing... just like iTunes rendezvous sharing
    - Improved slideshows. cool new effects. can pick music/playlists from itunes.
    - A big request is the go through photos right when imported. rate them or discard or adjust them while importing.
    - New album - "last 12 months". Can change it to 9 months or whatever... customizable. And you can have last two rolls or last three rolls etc... Smart Album: Example - all holiday photos, done by "date range between 12/20 and 1/1". Ratings also. 1-5 stars. Smart album - like top rated photos.
    - Over 25,000 photos. Showing how he can scroll through them quickly. Resize them. (fast). Really Really nice. Very responsive. New effects: edit mode. Sepia finallly. Another cool thing: time based organization. In the photo library, you can now open the library and automatically gives u photos by calender year.
    - Demo of iPhoto
    - iPhoto: Time based organization, Smart albums (like smart playlists), Rate photos, enhanced slideshows, and rendezvous photo sharing.
    - Includes the latest and greatest of iTunes. Everything works together. iTunes works seemlessly with the other apps. iPhoto - we've all got a few thousand photos now. We wanted to get ahead of that. so the #1 feature is that it supports up to 25,000 photos with blazing performance
    - Next - iLife. We launched iLife a year ago. A lot of the apps we lanched before that but we pulled them all together into a suite. Today, we are launching iLife '04. "for the rest of your life"
    - So that is iTunes
    - Pepsi and Apple are going to give away 100 million songs legally starting on Feb 1. Pepsi bottles (300mil) with specially marked caps. There will be a special pepsi button on iTunes and get a free credit from the specially marked caps. Promo runs for 60 days.
    - We have now got 500,000 songs available today on iTunes. That makes iTunes the largest online music stores in the world
    - Over 120,000 classical tracks today added.
    - New features: Billboard Charts (hot 100 charts) from 1946 to present. More billboard charts to be added for various genres. Today.
    - iTunes Essentials - collections of essential songs
    - AOL Integration.
    - Gift certificates. Over 100,000 gift certificates sold
    - AudioBooks. Pleased to announce we sold over 50,000 audio books in the first quarter.
    - Who was the top customer? The top spender spent $29,500.
    - Marketshare data from SoundScan. iTunes = 70% marketshare.
    - The Gap is getting bigger and bigger (compared to competitors)
    - Now, iTunes. Pleased to announce we've crossed our 30millionth song.
    - $3/gigabyte. Far cheaper than other options
    - Xserve RAID certified on two versions of linux and Windows XP Pro and Windows 2003 Server.
    - We've qualified a bunch of Fibrechannel switches.
    - Xserve RAID: 3U. Added new. 3 1/2 Terrabytes. SFP connectors. RAID Set slicing.
    - We thought we ought to also update Xserve RAID.
    - $2999 - single 1GHz. $3999 Dual 2.0GHz.
    - Ships with Panther Server 10.3 - unlimited client lice

  8. Re:Scrum? by Anonymous Coward · · Score: -1, Offtopic

    "Scrum" as in rugby; "rugby" as in "better when you're drunk".