When Agile Projects Go Bad

blackbearnh writes "CIO Magazine has an article up looking at some of the ways that Agile projects can fail, or Agile can be misapplied in organizations. Some of the issues raised may not be new, but folks might want to pay special attention to these, since the people throwing the stones are two of the original Agile Manifesto signatories, Alistair Cockburn and Kent Brock. From the article: 'Once individuals become familiar with Agile, either through training or practice, they can become inflexible and intolerant of people new to the process. Cockburn has seen this in action. "I'm one of the authors of the manifesto, so if I say something 'weird,' they can't tell me I don't understand Agile. But if someone else — and it doesn't matter how many years of experience they have — says something funny, they get told they don't understand Agile."'" Here's another recent article by the same author on the perils now besetting Agile.

Comments

No way!

[Anonymous Coward]

You mean when a good idea is elevated to an inflexible ideology, it can become a bad idea? This has never, ever happened before. :)

SHOCKA

[kevination]

Bad management leads to bad results, no matter the methodology.

Re:SHOCKA

[MadKeithV]

Sure it wasn't a failure because the engineers involved wanted it to fail? Because the number one thing that should be added to a team is responsibility - those engineers should have been screaming at management that it was failing.
Of course, they probably did, and management probably ignored it because the Agile Consultant had a nice suit and a cool car...

Re:SHOCKA

[Nelson]

I'll agree with that. Bad management, a top notch team and agile can not only lead to worse results but effectively dismantle the team as well. From the rest of my career experience, assembling that "right" team is a very very hard thing to do. I'd trash almost any product if it meant keeping a high quality, high functioning team.

Agile sort of masks management mistakes for a while and creates the illusion that it's not as important, it places emphasis on the team, encourages team responsibility and the breaking point is when the team is so overloaded that they start to fail. A lot of really good teams will absorb that load for a while though, you'll hear some rumblings but the management mistakes will continues to build. When the load starts to break the team, it's way way too late to fix, the team will fracture and break apart.

I don't like the "2.0" moniker, so I won't call it "web 2.0" or whatever but the current generation of top notch software, lead by Apple and the very few similar companies (come to think of it, I can't name one that is truly similar so much as wants to be and pretends to be, except maybe google) is kind of different. You really have one shot to win or lose the customer, the expectations are exceedingly high, and every product has to pretty much be "lights out" from the start. The old MS "fix it in 2.0" doesn't cut it, there can't be much that needs fixing and in 2.0 you need to further raise the bar, not continues to get in to the game. Web services are a bit different in that you have a little bit more room for mistakes, but not much, the web crowd won't use crap while you figure out the "right way." To put together an itunes or an iphone you need excellence throughout the organization, not a razor sharp dev team and a bunch of morons surrounding them playing iterative games to mask their own incompetence. I think as the rest of the industry wakes up to it and realizes what they have to do they'll realize that it is much harder than it was before. You need a very cohesive management team with vision and high standards that is all on the same page and you can't just pick that up at some seminars. If you have that then maybe Agile might work for you.

Extremist Programming

[composer777]

We've had quite a few people around my workplace promoting extreme programming and (fr)agile software development. It's not going to replace lack of talent, lack of planning, or not knowing what the true costs of things are. I think many people in management have taken in only the parts that they want to hear, and ignored the rest. The results of this kind of misapplication are fairly obvious.

My biggest objection is the lack of credible proof that it actually works any better than the tried an true software methodologies.

Re:Extremist Programming

[Anonymous Coward]

I think many people in management have taken in only the parts that they want to hear, and ignored the rest. The results of this kind of misapplication are fairly obvious.

My biggest objection is the lack of credible proof that it actually works any better than the tried an true software methodologies.

as a part of a company that only does things the agile way, we have a saying "you cannot do agile, you can only be agile" in short it is completely dependent on the kind of people you have on your team.

Re:Extremist Programming

[wrook]

My biggest objection is the lack of credible proof that it actually works any better than the tried an true software methodologies.

But can you provide proof that the "tried and true software methodologies" work?

Our biggest problem in process improvement is that we can't easily measure productivity. If we could, then it would be a walk in the park to determine if one way of doing things was better than another. Instead we can say that project A was successful at meeting its targets while project B was not. Even then management always games the system to claim that their projects were successful. How else would they get promoted?

Lately I've been hearing from a lot of people who claim to agree with agile principles and therefore feel that they must be agile. Whether or not they actually achieve results seems to be secondary. A good example would be, does your team *actually* welcome changing requirements? If not, then I would argue that you aren't agile.

If you read through the agile principles and think to yourself, what would you do to actually achieve this, I hope that people start to realize that becoming agile isn't trivial. In fact, I think many people would argue that it's impossible. Having worked on a couple of teams that were actually agile in this respect, I disagree.

I think scepticism is healthy. But when we're comparing development methodologies it's hard to find ways of comparison. Even though I'm retired from professional programming, if I were to take it up again I wouldn't work in a more traditional setting. Especially one of the XP teams I worked on, while not being particularly exceptional in terms of talent, was one of the most amazing experiences of my career. I highly recommend others to try to get the same feeling. But it is by no means easy.

Uhm...

[NaCh0]

Alistair who ??

Re:Uhm...

[jeroenb]

Actually, it's pronounced Coburn because he's from Scotland and apparantly that's how they pronounce it there. He starts nearly every talk by explaining how to say it correctly :)

Kent Brock - s/ro/e - Kent Beck

[Z-MaxX]

Hello, McFly! It's Kent Beck, not Brock.

Cockburn?

[BountyX]

Cockburn has seen this in action.

I have personally experienced cockburn and assure you will become "become inflexible and intolerant".

Buzzword Boredom

[Anonymous Coward]

In my thirty years of design and programming on projects big and small, high level to low, I've seen this crap come and go. It's always more or less the same, and it's always instigated/pushed/proposed by those who cannot code, or those who are looking for something to do besides code.

Re:Buzzword Boredom

[Shinobi]

Agreed. Despite some successes, generally speaking mostly small or low mid-scale projects, "Agile" is to software engineering what Red Bull Flugtag contraptions are to aviation engineering.

Re:Project steps

[famebait]

list the steps from requirements gathering right through to production deployment.

That would not be agile :-)

Re:pft

[famebait]

no, I've done both and they are very different.

If your impression is that agile methods are directionless then you have met poor practitioners or read negatively selective descriptions. It actually quite rigorous, but on other axes than traditional methods. They focus on process and feedback and controlling uncertainty as you go rather than pretending you can plan away uncertainty.

Of course like any buzzword it attracts unwarranted 'namedropping' use, and like any subculture it attracts overzealous idiots who just want to be 'part of it' and look down on others. But that doesn't really say a lot about what it can offer or not.

Re:pft

[SharpFang]

Agreed; Agile is a specific methodology that is quite orderly and efficient.

Too often, though, sloppy managers let the project run wild, zero specs, zero plans, do what you feel like doing how you feel like doing it, the deadline is yesterday, so there's no time to plan anything, the customer is our alpha-tester - and they call it "agile development" because "total brothel development" doesn't have the right ring in the name. And people who see such projects really believe this is what Agile is all about.

You guys were using Scrum the wrong way

[Qbertino]

Just reading your post shows me that you guys where using scrum the wrong way. Most people don't understand the true purpose of Scrum. We do Scrums every day at 11:30 and they take about a minute ot two per team-member (just did todays and I'm checking on slashdot while adding my new tickets to mantis).

Scrums are stand-up meetings. Everybody stands and keeps his turn short. You say what you where doing since yesterday, what you achieved and what you are going to do today. It's mostly about self-awareness and being able to judge your capacities - it's a mental exercise, not a classic meeting (we have those too, but only like once every 8 weeks).

Agile came out of the need to deal with controll-freaks who didn't do their homework and tend to weigh down things that should be lighweight with to much bureaucracy. Where human interfacing is a key point of your software, Agile is a very good method to handle things. If the pipeline has skilled people at each section, that is. Our company is technology driven with managers that actually wrote code at some point in their career. I'd call our process somewhat agile - we get changes every odd week - and everybody deals with it without a single hitch. ... Coming to think of it, maybe that's why we dominate our market.

Bottom line:
You guys got Scrum and Agile all wrong. Maybe you should've steared clear from the get go.

Re:You guys were using Scrum the wrong way

[winomonkey]

I love it when a person uses their own non-standard definition of a word to tell everyone else that they are wrong and have quite simply missed the point. While qbertino is quite right to say that there is a stand-up meeting portion of scrum (frequently called a "daily scrum"), it is not the entirety of the scrum process. A quick look to wikipedia shows that perhaps there is a better definition out there than qbert believes.

Our team uses some of the artifacts of scrum - the daily meetings, burndowns, backlogs, but we fall into the "scrum in name and not in spirit" category of many others. The team has a tendency to push back against any definition and any documentation. Cowboyisms overpower agile engineering practices a little too often, which is a dangerous thing.

I am also curious what company you work for ... I know a lot of "technology driven companies" that let the tech of the day drive their agenda, which is not always where your customers want to be. It is always good to find a company out there that can live an agile life while playing with the latest and greatest technology AND be top of their market segment.

Bottom line;
What has worked for you and your team does not always work for all other teams, nor does it mean that you get to redefine the definition of scrum for all other folks out there. That is the big problem - people making grand and dogmatic statements about how things work or do not work. Finding something that works for you is more important than finding something that works for somebody else.

yes, but here it's funnier

[Moraelin]

There isn't one of them that doesn't have individuals that follow the processes blindly.

Yes, but here it's funnier, since you were supposed to trust it without any proof, and in fact in spite of proof to the contrary, from the start.

Since the topic is agile projects gone bad, you only need to look at the original Chrysler C3 project that became the poster child for XP: from Chrysler's point of view, the project was an abject failure. By the time it was _years_ overdue, it still hadn't delivered a tenth of the promises. At least one of the managers involved as client-representative in that project got stress-related health problems during it. Chrysler not only cancelled the project, but forbade XP. It's something you'd normally call "EPIC FAIL!" as projects go.

But in true CCCP fashion, a failure got redefined into a smashing success, and presented as such in the books. In fact, turned into the poster child for successful XP.

So there we have our answer already: when XP projects go bad, failure gets redefined as a smashing success.

Later studies into, well, how much better _is_ pair programming, actually showed that such a pair is somewhat better than a single programmer for inexperienced freshman year programmers, but barely marginally better than a single programmer when done with experienced programmers. Both cases delivered actually less than the two programmers working separately on different parts of the problem.

Again, it won't stop anyone from still pretending that it's the opposite.

XP also redefined what "bug" is, by pretending that only stuff caught by the client's automated tests are bugs. Hence, if your tests for my "int add(int x, int y)" method only test for x = 2 and y = 3, and the whole method is just a "return 5;", it's not buggy. If you wan it changed, that's a change request, mate.

Hence making possible the claim that XP delivers 100% bug-free code. Now, if you want to redefine what "bug" means, you could do the same for any other methodology, but XP zealots like to ignore that. (I mean, seriously, if "not caught by the acceptance tests" doesn't mean "bug", then Windows 3.0 was 100% bug free too. Dumbly enough it was only tested internally on identical Compaq computers, but it ran flawlessly on those and with the particular software mix used in the test. If it doesn't work on your computer, though, XP would call it a change request.)

So, heh, it's mildly funny to hear the same guys who did that reality redefinition complain about others doing the same.

Seriously, it reminds me of a joke by Vince Ebert, translated loosely from memory: If you think there's a beer in the fridge and go look, that's science. If you think there's a beer in the fridge but don't look, that's religion. And if you look, see there's no beer, and still think there's a beer in the fridge, that's esotheric.

That's what XP is: something that started as a religion, and ended up an esotheric cult.

This is actually where XP/Agile have a major advantage over other more formal processes - the Agile processes try not to promote rigid thinking and if applied correctly should be self-correcting or applied in more than just name only (but you would know that if you had read TFA).

I'll be the first to say that flexible thinking is great, and that XP even has some good ideas. The point where it fails is that "turning all the knobs to 10", to quote the authors.

Essentially what they do is discovering something like "hey, some salt in a soup tastes nice, and a bit of vinegar makes it even better"... and from there formalizing an ideology in which all soup is done only with salt and vinegar. In fact, by boiling a kilo of salt in 5 litres of vinegar, since that's the turning all knobs to the max point.

Lots of ragging on Agile here

[Qbertino]

I'm reading lot's of ragging on Agile here. Maybe you guys should actually f*cking read the agile manifesto. It's only a few sentences after all. You'll notice that it's actually quite a good thing that tries to rid the software devprocess of bloat and get close to the people for whom software is written.

Whenever I've used agile with the right people, it was a breeze getting the job done. It's basically common-sence systematically applied to software developement in my book.

Re:Lots of ragging on Agile here

[RobinH]

Whenever I've used agile with the right people, it was a breeze getting the job done.

Well, whenever I've done anything with the "right people", it's always been a breeze. The problem is that those projects are few and far between. The methodology that will eventually work the best is one that takes the wrong people and makes them productive. (Like assembly lines making cars - if you had a bunch of skilled mechanics, they could make a good car, but if all you have is a bunch of high school drop outs, and you want to build good cars, you need an extremely rigid process to make them useful.)

Re:Lots of ragging on Agile here

[MadKeithV]

The problem is that those projects are few and far between.

Rather, it's "right people" who are rare. Not every project can hire only programmers that are in the top 10%. There just aren't that many to go around. Maybe as low as 1 in 10, by some estimates...

That can't be right - over 75% of the developers that I've asked rate themselves in the top 10%!

Re:Measuring productivity

[h4rm0ny]

Furthermore, you can't consider a group comprised of people in the top 10% (however you determine that they are) to necessarily perform X amount better than a group that comprises people in the bottom 10%. Supposes your top 10% group contains two genius prima donnas who grow to hate each other? Suppose your bottom set contains a group of jobsworths who don't show much initiative and consequently follow the design plans of the power-hungry one and consequently follow a cohesive and well-delegated plan? How complex is the task? If it's simple code-assembling, an office full of plodders might have little significant disadvantage compared to the creative geniuses. Or if it's highly complex, they might botch the whole job. You can't easily prove the effectiveness of different methodologies and I don't think it's possible at all without significant resource.

But supporting the argument that comparing methodologies can be difficult is not me personally arguing for the effectiveness of "Agile" which I view as some common sense, some ideas that would be bad if applied in the wrong circumstance and a lot of vague principles that should be read to stimulate insights and then forgotten about.

Re:Measuring productivity

[MadKeithV]

I think the most effective test would be to keep throwing terrible programmers at a problem and see which methodology fails the least badly.
Because in real life you rarely find enough of the top 10%-ers to do all the work that needs to be done.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Agile as a defense mechanism

[Jahf]

As someone who had the "fun" of becoming a scrum master at a company that was trying (and eventually failed) move off of waterfall I got to be the main Agile advocate with our management.

This company failed from both ends. Developers began to use Agile as a shell to protect them from the problems that were in the old system. That's understandable given how badly they were overworked during deadlines. However what they didn't see was how this poisoned upper management to the entire concept. Management was willing to give a little to get a LOT (they wanted projects to miraculously start to "just work" as one put it) but wanted to have the process be open. Development wanted ANYTHING to shelter them from the hell they had been going through.

Both parties were going into Agile for the wrong reasons. They both thought they were the right reasons.

By the time I got fed up and left we had everyone calling our process "Agile" because we did scrum meetings, but all the processes behind the scenes had reverted back to waterfall at the core. And when I would tell management -or- development that they were doing things for the wrong reason. I tried to avoid saying "you don't understand Agile" so that I wouldn't get defense mechanisms raised and funny part was the more I avoided using that phrase the more they would toss it back in my face (both sides). We had multiple meetings, even bringing in outside consultants, and the meetings always ended with them agreeing to the points brought up but then a week or a month later caving back to their old ways (this was a shop that was VERY entrenched in old ways, having done web services for financial companies for well over a decade).

The point here being ... it isn't always just management that doesn't "get" Agile. Sometimes the devs don't "get" it either (and very often the middle group of PMs has plenty of "wtf" left as well). Agile isn't a defense mechanism that allows you to block people from giving you work or to let you work on only the things you find interesting. And if -neither- side wants to be Agile for the right reasons it will fail hard and fast.

PS. Me? I went back to being a Sales Engineer / Field PM for companies in other timezones ... however the company I went to DOES do Agile and does it WELL. My scrum master time left me with a bad taste for ever doing it again and I will -never- try to lead a transition to Agile, but it was nice to see my opinions on processes and reasons work. I had convinced myself when I left the scrum master gig that -I- didn't "get" Agile ... but I did ... the company just did not -want- the changes required. Only the benefits.