Slashdot Mirror
Microsoft Lauds Scrum
under_score writes "According to eWeek.com Microsoft is adopting the agile methodology called Scrum to get software built faster. Is it working? They seem to be claiming that Scrum and Extreme Programming have helped them get recent releases such as SQLServer out the door faster with better quality. Many other large organizations are also adopting agile methods including Yahoo, and Google. Are agile methods the next big thing in software development?"
- I work for a company that adopts new trends at least 5 years late - that's about when they're being phased out elsewhere. For example, we're now deeply committed (or should I say sentenced?) to J2EE, RUP and outsourcing;
- Our company works on a contract basis, with complete and "firm" specifications (BDUF, anybody?) going in and deliverables coming out, with no wiggle room, no negotiation (at least from our side), no onsite customer.
Given such an environment, which I'm sure will sound familiar to others, do we stand a chance of ever being able to work this way too?When one person suffers from a delusion, it is called insanity. When many people suffer from a delusion it is called Rel
With Microsoft's virtual monopoly, they have a guranteed market for a program and would have to cock it up REALLY, REALLY bad for that market to shrink to a noteworthy degree. So they need to get programs out there to rake in the cash. Quality isn't really an issue for them.
However hopefully if they continue to use this flawed business plan, they'll continue to slowly lose customers. Well, I can dream, can't I?
So how do you write tests for ..
- applications that are a constantly moving target "this would be cool to have"
- applications where the moving-targetness lies in the presentation, while at the same time some customers bitch about any change in presentation
- applications with changing data sets - you can run your tests fine on the standardized data set, but then when it hits the real-world data, all you can say is "Sorry my application is perfect, it just doesn't work with with that data.".
I'm still trying to figure out what people mean by 'social skills' here.
Take a look at one of the Agile Poster Children and his proof that it works.
Quote: "Because of the newness of agile methods there simply hasn't been sufficient time to prove that they work in a wide variety of situations."
Thats a wonderful way to dismiss anyone saying bad things, and it's rubbish, because the burden of proof for any claim is independent of its age.
Quote: "the question "where is the proof" is typically asked by organizations that fit the late majority or even laggard profiles ... Because agile techniques clearly aren't at that stage in their lifecycle yet I believe that this question simply isn't a fair one at this time."
So the act of asking for proof these things work means you're not ready? Ad hominem alert.
Quote: "Are they really interested in finding an effective process or are [they] merely looking for a reason to disparage an approach that they aren't comfortable with? Are they realistic enough to recognize that no software process is perfect, that there is no silver bullet to be found? Are they really interested in proof that something works, or simply an assurance of perceived safety?"
Ad hominem again.
Then you look at the project that started Agile, the Chrysler Comprehensive Compensation (C3) project. It was lauded as the first agile program and a success, however by February 2000 with the system was failing when paying 76,000 of the company's 86,000 employees. It was cancelled. Apparently this failure is now the new success.
Every methodology has rapid followers who will hear not evil said of it, but when looking at these things you have to remember "He's NOT the Messiah ... he's just a very naughty boy."
My guess is that XP is working only because it banished the "good on paper" metodologies and because of the refactoring formalism. All the other points you cite are a description of "good management", and as so, don't change just because the company adopted another metodology. But getting ride of a lot of useless paper (by letting the programers decide what is usefull) is the way to go.
Rethinking email
I'm a developer at a smallish company (~25 developers) which started using scrum in August. I was very skeptical at first but really like it now. The short version:
Previously management just did whatever felt right.
Sometimes this was very good, sometimes it was very bad. Sometimes it was just inconsistent.
Now management has a defined methodology that they follow. There are some rules. The rules need not be particularly great ones (although I don't mean to suggest scrum isn't good), just so long as management is thinking about them and makes a concious decision to be consistent and let develoeprs know what to expect.
Specifically, scrum has helped us overcome the "holy shit, its a big customer bug!" panic that happened occasionally. We still panic, but its not the entire organization jumping onto one bug, its just a single scrum team.
Posting as AC, as my coworkers AND management read slashdot and will recognize me.
I used to work in a scrum-based web development shop.
Meetings went something like this:
Go around the room, and say #1 - what am i going to do today, #2 - whats in my way of getting #1 done. One two people were allowed to talk, the person who's turn it was, and the manager in charge of the meeting. If another person in the meeting was the cause of someone's #2, the manager would turn to them, give them (and only them) them the chance to respond. Lather rinse repeat.
There was no "I did this yesterday" because a) we supposedly heard about that the day before, b) the assumption was that you got it done.
Even with at least three different projects going on, and maybe 15-20 people in the room, we were out of there in 30-45 minutes. Any major issues were taken offline so that the rest of us could get back to work.
We usually had only one meeting a day, sometimes two. I found it worked extremely well with a minimal amount of thrashing. We might have been using a modified version of scrum; can't remember - those were dotcom days, everything's still a blur.
At Alias, we use Agile and Scrum very extensively. The Sketchbook team was among the first at Alias to adopt it at our company -- Jim Highsmith even gave us a little write up in one of his books.
We don't, however, do that much pair programming. And the whole completely open office space works for some, and definitely not for others. For myself, I'm way too easily distracted -- so I need a nice quiet and private cubicle in order to achieve the state of "flow" where I can write code. In my experience, pair programming works for debugging and integrating code -- and not so well for creating it. YMMV.
Sketchbook has come in on time and on budget, and with extremely high quality. Agile and Scrum had something to do with it. I think the fact that we had a clear vision, a small and very experienced team, a really good working relationship with our usability team and research team, great QA, and excellent management had at least as much to do with it.
As a process, its the only one I've seen in 20 years of doing this that actually makes the life of a programmer better, not worse.
Ian Ameline
"Steve Ballmer, chief executive of Microsoft acknowledged during his keynote address at the Microsoft launch event that the company needed to get more agile and to produce software faster, to the tune of delivering technology every 18 to 24 months."
No. No one wants such a short release cycle but you and your shareholders. If I may borrow one of your favorite words, there is not enough "innovation" in most of the technologies you purvey to justify an 18 month release cycle. You managed to pull it off for Windows XP and did nothing but piss off those of us who bought your line about Windows 2000. To us, it was clear that XP was nothing more than the finished version of Windows 2000. We had just spent a fortune on upgrading to the future only to be told 18 months later that we weren't worthy of free utilities, functionality upgrades, or even comprehensive service packs since we weren't on the latest release. As far as I'm concerned, you can keep your interim versions to yourself. Anything shorter than 3-4 years for operating systems and server products is lunacy.
But like any process, scrum won't work unless you have buy-in from every level. I think it took us almost a year before we really got into a groove with scrum and started getting really big benefits out of it.
Developers now work without meddling from management for at least the duration of a sprint (a month). We can focus and get lots of work done.
Transparency has built trust between the developers and the other stakeholders, like testers, usage experts, and management. There's far, far less tension between these groups. And whatever tension that does exist is kept off the shoulders of the developers.
We were a small company, bought out by a very large one, and now our group and our process is starting to be viewed as a model for other groups in the company to emulate, because we're (apparently) far more efficient, and we're getting a lot more work done.
We don't use XP (although we do have a lightweight code review process). The benefits of XP weren't quite as evident to us, so it's not something we mandate - developers can do it sometimes at their discretion.
Accountability on the heads of the powerful.
Power in the hands of the accountable.
But what is wrong with it?
You've given us a rocking-horse analogy, and you've given us a purported example of a failed project that used it. But software projects go massively over budget all the time, and they get cancelled all the time. Given that people often learn the wrong lessons from failures, the negative opinions of the methodologies by GM executives may not be a compelling indictment of the methodologies themselves.
Or, to put it in a more Slashdot-friendly way: Who cares what a bunch of incompetent blowhards in suits think?
Seriously, though. What specific problems have you had using these methodologies?
I've tried XP, though in an academic setting on a small project (about two months), and I thought it was the most fun I'd ever had programming.
You want the truthiness? You can't handle the truthiness!
will MS really change their processes or will it become nothing more than mindless corporate a paper chase? Will they embrace the culture and beliefs of the new methodology, or will it become just another 'flavor of the month'?
From my experience in corporate culture I suspect the latter. What often happens is a group with in a company adopts a methodology which works for them. They then become the poster children for 'organizational change'. The processes are then rammed down the throats of other departments who may be resistant to change, or for whom the processes are inappropriate.
Managers will go to be trained in the new metods and learn the form but not the true spirit of the reform. 1/2 hour scrums will begin to creep up to 2-3 hour daily meetings. In order to take advantage of the accelerated release cycle, products will be scheduled for release sooner, and testing/QA will suffer. Quick fixes will be prefered over true fixes and, inevitably, release will slip. That is my prediction.
Truly changing an organizaqtion takes years. Everyone originally working under the old system essentially has to be fired, quit, transferred, retire or die (one article I read, though I cannot find the reference, said it takes about 20 years for a large company to change).
This might be interesting to watch, in a morbid sort of way. MS is looking for a magic bullet, and we all know how well that works.
putting the 'B' in LGBTQ+
Apple software has plenty of bugs too. The system update that would hose disk partitions if they were labeled with spaces in their names. Airport failures if you have > 1GB of RAM in the current series of PBs. iTunes5 trashing filelists. ibook logic board problems that killed their video.
almost all projects that use it are colossal failures by every reasonable metric including the famed Chrysler project that pioneered the whole process. XP projects are often late and full of bugs and defects. And then you have the maintenance problems from the lack of documentation created as well as unstructured and unmanaged architecture that emerges. There are a lot good individual practices in XP and Scrums, but the whole is much weaker than the sum of its parts.
I think the lack of meaningful design and analysis is the real killer of XP. XP relies on a methodology called continuous integration to overcome this deficiency. Continuous integration is simply a way of saying continuously change the design of the architecture to meet the needs of the day. And so you end up continuously rewriting the same code to meet new usages. XP has another concept called test driving development which dictates unit tests are written before the actual functionality. So when you continuously integrate, not only do you have to continuously update objects that use your code that you're changing all the time, but you also have change and maintain the unit tests already created. This whole series of chasing your tail can be avoided with a little forethought (aka design and analysis) to what you're doing.
XP can be a useful methodology in environments where the requirements are changing daily, there are a lot of unknowns and risks in the project such a dependency that doesn't exist yet or that is unstable and so changing a lot or projects whose future from a business perspective is questionable. But overall, XP is as big of a joke on the development community as scientology is to religion. Both are pushed by zealots detached from reality and neither has any evidence to back its dogma.
Since I bashed XP above, I'll enumerate some of the good things about it.
1. Continuous integration if done in a more sane manner like coming up with a design before hand, but not being afraid of changing it when needed and having tools to manage changes (like a continuous integration server)
2. quicker release cycles
3. continuous input from the project champion and business users.
4. having unit tests
5. daily meetings on the issues of the day
6. making a business case for refactoring code instead of doing so because it's cool or technically correct.
7. building only what you need and nothing more
For us the weakest part of the scrum process ended up being the time tracking (which is really cool in theory and draws pretty pictures for sr mgmt on progress). This isn't due to a fault in the concept, but the nature of our workload. Many of the groups were still heavily into the 'R' side of R&D at this stage, and its very hard to predict what you'll turn up and how much that will cost you when you're still in research and design work. From a mgmt point of view this looked like us slipping daily on the charts, which caused some bad feelings.
Once things moved to implementation and testing in a given group the scrum stuff worked brilliantly. As one of the team leads I generally dislike excessive meeting time (preferring instead more informal 1:1 or 1:n in the hallway or on IRC), but these got short enough and had enough value they were worthwhile.
It really does help to force everyone to think about what they've accomplished in the past day and 'promise' what they expect to accomplish in the next day to their team. We didn't have any real slackers, but just spending the couple of minutes planning out your day enough to tell everyone else what you'd be up to was very beneficial.
Generally the 'what help do I need' part of the meeting was the least useful, as most people would IRC or email around directly (perhaps at the cost of some NMI style distraction) and not really ever come to a meeting needing anything. It was still IMO worthwhile.
Scrum only worked when we could break down implementation into bite sized chunks (no more than 2 days I think is the guideline in the book); at the risk of repeating myself it really didn't work well going into a big problem and trying to work out a plan and design.
First, the scrum book, by its co-creator ken schwaber http://www.amazon.com/exec/obidos/tg/detail/-/0735 61993X/ref=pd_sim_b_4/103-6526029-2864653?_encodin g=UTF8&v=glance
interestingly enough, its published by MS Press
Second: what people seem to be missing in this thread is that scrum is essentially a anarchist/communist/utopian project management technique. there are no bosses telling you what to do. teams are self-organizing and autonomous. this is a _radically_ different project management technique -- no folks, its not about the daily meetings. it's about being bossless.
A buddy of mine went through scrum training with schwaber, and he had them do an interesting exercise. teams of two were instructed to walk exactly 300 paces in exactly 2 minutes. each team was composed of a boss and a walker. the walker was told by the boss to take a step, slow down, or go faster.
_none_ of the teams successfully walked the 300 paces in the time period. so they were broken up such that each person had no boss, and simply had to walk the 300 paces. _every_ person completed the task.
lesson: micro-managing bosses just slow you down.
third: a very interesting practical example of this sort of project management technique is the GE jet engine plant in durham, NC.
http://www.fastcompany.com/online/28/ge.html
This shop is organized around a bossless culture. They are the most successful and productive jet engine manufacturer in the world. A great example of how this sort of technique isn't just for pot smoking hippies.
Finally: the essential thing that binds all of these agile/xp/scrum-ish like techniques together is TESTING. No code can be written without unit tests. All requirements have a direct mapping to a suite of acceptance tests (written, say, using fitnesse -- www.fitnesse.org).
There are three types of developers in this world: those that test first (the ones I want to work with), those that have never tested at all, and don't want to (stay away from these ones), and those that have read/heard about test first and want to try it out (these can be saved).
--sozin
Also look at Apple's software before Steve came. MacOS was absolute crap. Copland, which was supposed to make MacOS a modern operating system, was a late, dismal failure that was eventually killed because Apple was too incompetent.
Apple is not a good example of what to do. Not every company can be a personality cult.
Sometimes it's best to just let stupid people be stupid.
Waterfall make work in theory.
In practice I've yet to work for a business willing to wait 2 years for their product. Where the business environment, aims, markets don't change for that period of time.
Pragmatically Waterfall doesn't work. It often almost works, and most developments muddle through. But my customers want new functionality next week, not next year. They'll wait until next month, but any longer and they start complaining.
Since I know of ways of successfully delivering next month, I'd be failing them to force them to wait until next year while I kick off a full waterfall process. Especially since they wouldn't give it the support it needs to work properly.
I can deliver every 2 months. I can deliver approximately bug-free code in that timeframe. I can deliver genuine business advantage, meeting actual business need and keeping customers happy. And I can repeat it.
Other engineers don't write software the way they build bridges, design power stations, construct roads. Oddly enough some of them have tried - and if they'd succeeded they'd be winning ALL the business. Different environment, different constraints, and therefore different approaches needed.
Incidentally, "every step must be done in order" is very wrong. Test before you build - how else do you know you built the right thing. (And test again after.)
You also forgot various steps, but that's a whole other discussion..
Considering you got fired after your XP experience you quitting the proejct, it's not suprising that you have a negative opinion.
I'd also have to say that you (self-admittedly) sabotaged the unit testing process automatically diqualifies you from providing an objective opinion on unit testing.