Slashdot Mirror
Is Agile Development a Failing Concept?
Nerval's Lobster writes: Many development teams have embraced Agile as the ideal method for software development, relying on cross-functional teams and adaptive planning to see their product through to the finish line. Agile has its roots in the Agile Manifesto, the product of 17 software developers coming together in 2001 to talk over development methods. And now one of those developers, Andy Hunt, has taken to his blog to argue that Agile has some serious issues. Specifically, Hunt thinks a lot of developers out there simply aren't adaptable and curious enough to enact Agile in its ideal form. 'Agile methods ask practitioners to think, and frankly, that's a hard sell,' Hunt wrote. 'It is far more comfortable to simply follow what rules are given and claim you're 'doing it by the book.'' The blog posting offers a way to power out of the rut, however, and it centers on a method that Hunt refers to as GROWS, or Growing Real-World Oriented Working Systems. In broad strokes, GROWS sounds a lot like Agile in its most fundamental form; presumably Hunt's future postings, which promise to go into more detail, will show how it differs. If Hunt wants the new model to catch on, he may face something of an uphill battle, given Agile's popularity.
DNRTFA, but I think this will be a fun thread.
Regardless of what Agile really is (a true scotsman?) the abuses perpetrated in the name of agile are appaling. I think a lot of people think agile means something like:
make bad developers good by not bothering to organise things properly
Which is really amazingly appealing if completely bogus.
SJW n. One who posts facts.
Like so many other things, it's very difficult to take an ideal theory and put it into practice in the real world. If your team really understands the ideas behind Agile and you have a good process in place to make it happen, you can have a great deal of success.
Unfortunately, like so many other things in life, most teams don't get it right and they end up failing to some degree or another.
It is hard to take someone seriously when the argument seems to be 'my idea is brilliant! but most people are not good enough to implement it!' rather than 'hrm, maybe my idea is not the universal solution and one needs to fit the methodology to the requirements and resources involved?'
Yes.
SJW n. One who posts facts.
No, it's just no longer selling as many books and consulting hours as it once did--so it's time to invent a new scam.
There is no way to optimize the development process for software. Inserting terms such as "Agile" or "Waterfall" really just create bloat and waste time. The best software development process is to have no process and work in the way that fits you best. For instance when I write large software projects, I just start coding, I pick a place to start at and go. I wait until I have large testable blocks completed and then debug and integrate them. I don't follow and form of standard development process and yet have never been held up via a deadline of meeting request.
When developers try and add those stupid terms, they're basically saying that they can't self manage and instead of taking responsibility, they're going to throw silly management methods around in an attempt to streamline a task which is unique to each individual developer and situation.
The only things you need to write good code are the right language, the right platform and proper requirements. Once you have those, you can just start and work to completion.
Agile started failing pretty much as soon as it met the real world. I disagree with Andy Hunt's explanation of why, though. It's not because "thinking is hard", it's mostly because of two things: management not allowing agile to be done correctly, and (from the developer's point of view) it drains software engineering of the things that make it a satisfying and enjoyable activity, turning it into the software equivalent of grunt factory work.
That's not even close to enough time for a major cultural change to take place. The Agile Manifesto describes a culture of work that is so fundamentally different from how work was (and still is) performed, that I expect it will take another 15 to 30 years for organizations to really "get it". This is the same thing that happened with Lean manufacturing. Toyota developed it, other manufacturers adopted it as a fad over the course of about 15 years, and then it declined in popularity... but it never died out because it was "correct" and "good". Now, 40 years later, most manufacturers are still learning to be lean, but lean has fundamentally changed the culture of manufacturing. I have clients that will probably be working to adopt Agile methods over a 10 to 20 year period. Agile hasn't failed... Andy Hunt's patience has failed.
Helping with organizational effectiveness is our job.
I'll dub thee DevOps
It only works when everyone is on board and willing to keep working on it. Right now I work on a large project that started Agile but the only thing Agile about is the velocity they want out of us while throwing away the rest.
No.
But it'll be yes again by the time of the next scrum.
Agile is not failing because there is nothing to replace it. Are we going to go back to Waterfall? Software development is inherently hard. No methodology is going to make it easy. But Agile works better than any other methodology that I have used, even when implemented piecemeal, by mediocre programmers, rather than in its "ideal form".
Ideas should be judged against their real world alternatives, not some perfect ideal. For Agile, there are no better alternatives. GROWS (whatever that is) may be an alternative, but it is untested, and looks like just an incoherent pile of buzz words.
Are we going to go back to Waterfall?
In my experience, that would be greatly preferable to Agile.
But Agile works better than any other methodology that I have used, even when implemented piecemeal, by mediocre programmers, rather than in its "ideal form".
My experience is just the opposite: agile is very nearly the worst of the methodologies I've used. The one great thing about it is something that can be done in any methodology: increased communications. We can throw out the bathwater and keep that baby.
There are more choices than just Agile and Waterfall.
I've got to agree with JohnFen. As a Program Manager, while Waterfall techniques could frequently end up with late or over budget, or both, projects, at the end of every project (I oversaw 5 multimillion dollar projects using Waterfall methods) we at least had a working application that met the original specifications.
Now, after two similarly sized Agile projects, all I can say is it seems to be an excuse for developers to skip QA/QC procedures "because we're already into the next scrum" and end up with a mess that doesn't come close to matching the original specification at the end blaming changing requirements and "developmental issues" during the scrum process. I just turned down a contract that explicitly required Agile coding because I don't have any confidence that the end user will be satisfied with the results.
"Unheard of means only it's undreamed of yet,
Impossible means not yet done." ~~ Julia Ecklar
I also have to agree but for different reasons. What I've experienced is Agile as excuse for micromanagement. Projects take much longer than they used to because we now agonize in meeting after meeting over details that used to be left to the developer to decide. Agile is a recipe for managing programmers fresh out of college perhaps, but most I work with aren't those, and they work better when you trust them with more of the detais and have management worry about the bigger pictures instead...
Skip QA/QC procedures? Then you're not Done. That means you've completed 0 stories and get to finish them on your next Sprint.
it's management. When you get a good project manager its like a breath of fresh air. The best PM I ever worked for was a guy that used to be a developer and just didn't understand object based programming, after an honest assessment, so he decided to go into project management. He shielded us from all the corporate BS and just let us code.
Most of the other PM's I have worked for have no background in programming. Some of them claimed to and didn't, which is much worse than someone that just tells you they don't. They would insist on idiotic exchanges like the following:
PM: How long will it take to code this?
Me: I'm not sure until I get all of the requirements
PM: Can you give it a guess?
Me: Sure but what's the point? It won't be a very good guess.
PM: That's OK I just need something to put on the project plan
Me: *Bullshit radar is now on full alert* So you just want me to pull something out of my ass so that you can finish up your project plan? Is that it?
PM: Umm, well, no...it's not like that
Me: OK, fine. I'll give you numbers but they are going to be grossly inflated to account for the unknowns. It covers my ass. Kind of like what you are doing, no?
PM: *Grunts and walks away*
Most of these people look at project management as if we were building widgets on an assembly line. As if we know exactly how long each task is going to take. Well, software development is not not like that. Not in the least. The ones that understand that - the ones that are truly "Agile" as it were - are the successful ones. The successful ones understand that any number of things can go wrong and plan accordingly.
DING!
If by saying QA/QC isn't completed you mean that unit and functional testing is missing, then the developer is not done. If you have problems with the developer not writing these tests, then be sure that "Definition of Done" includes some acceptable target level of unit/functional testing.
If on the other hand you get to the end of a story and accept it only to find out that it doesn't meet your QA standards, then you as a product manager haven't done your job in properly validating the story prior to acceptance; add to your own procedures the time required to properly validate the story for acceptance. Maybe you need a testing resource to do this if you are overworked as a product manager - an assistant product manager, even.
Agile is as good as you make it.
Let us live so that when we come to die, even the undertaker will be sorry -- Mark Twain
None of those are failings of waterfall at all. There is nothing about waterfall that requires you to make ironclad decisions at the very start, and there is nothing that prevents you from adapting the course of development as the project proceeds.
In other words, you aren't describing waterfall in your comment. Yes, I'm invoking the "No True Scotsman" fallacy, since that is usually what is invoked when Agile is criticized.
The real truth is that all methodologies can be done well or poorly, including waterfall and agile. The difference that I've seen in practice is that it's incredibly hard to implement Agile correctly (such that I've never seen it done), but implementing waterfall correctly is not a huge chore.
At its inception, the Agile manifesto was simple. Four priority/value statements and then a list of simple principles. The goal? Merely to say that delivering value to the customer, collaborating with customers, frequent delivery and feedback, and team empowerment are the way to deliver software. Focus on delivering value. Don't focus on delivering things that aren't valuable. Very simple.
Once Agile values started to become embraced and a couple of new development processes came along (SCRUM, etc), you all of a sudden had a bunch of consulting companies and community meetups appear that all but destroyed the perception of Agile. For these companies and community groups, it's all about the process. They will teach you how to "do agile". They will provide you with bodies/contractors who can "do agile". They will sell you certifications which show you "do agile". They will sell you seminars and training on how to "do agile". They will sell you software which "does agile". Agile has went from a basic set of values to becoming a fundamentalist religion.
So my statement to "Agile Process Improvement" firms is this: You are all just scammers and profiteers. You are software development Pharisees. It is amazing that you focus on profiting from creating processes, enforcing processes, teaching processes, and writing process software... for a methodology where the first value statement is "Individuals and interactions over processes and tools". Why don't you guys teach REAL agile? Why don't you teach companies how to better define value and deliver it to customers instead of selling new processes, fundamentalism, and bodies?
For the rest of you, if you want to be "Agile". Read the Agile manifesto. Create your own process that suits your team and your business. Work continuously with your customers to understand what is valuable to them. Deliver good value to them often and get their feedback. Allow your team to learn and grow and understand the needs of the customer. THAT is Agile. THAT is all you need to do.
Agreed. Too many companies use the Agile "be flexible" rule as a carte blanche, get-out-of-jail free card.
They pick and choose what parts they're going to do, which parts they're going to ignore, and which parts to which they're just going to pay lip service, and call the end result "agile". They don't understand how each part reinforces the other, and as such pick and choose among practices and ceremonies, and tell one another that they're being "flexible".
Failure is usually with implementation, not with Agile.
Any sect, cult, or religion will legislate its creed into law if it acquires the political power to do so.
Agile is failing in some areas, surviving in others. One snag is that Agile is a cult, pure and simple. If you vehemently disagree, then perhaps consider that you may be a cultist.
And the Agile cultists also believe there are only two world views: Agile versus Waterfall. Which is ridiculous, because the vast majority of developers use neither and instead have a hybrid approach from many different modesl including some ad-hoc methods. I know people who work as contractors who've said every single company they've been at use a different method, even those who claim to use Agile all use it in extremely different ways. When you are told essentially "do you really want to go back to Waterfall" then you know you're talking to a cultist.
Agile is pushed by its promoters as ideal for all types of projects. It does work well in some sorts of projects, where everything can be divided into tiny one or two week chunks, but it fails horribly where you need months of work for some features and lots of upfront design with large teams that need to coordinate. For example, taking an existing web design and maintaining it is easy with Agile, but designing an entire product from scratch, including hardware, software, and services will find Agile to be a frustration. What I have seen happen a lot is that there's a lot of old style design behind the scenes by the designers but then Agile is used up-front by the coders; each sprint picks out portions of the grand plan to work on, features are split into tiny two week slivers and so forth.
Don't forget the Agile industry here too. People whose high paying jobs are to teach Agile, to facilitate Agile, to be paid scrumm masters rather than developers, and so forth. I've seen no other development style that involves so many paid outsiders, most of whom are evangelists.
Of course there are good things with Agile but failing to acknowledge the bad things is not being objective.
Right now I'm on sprint number 10 for my feature. I get pressure to finish up the feature, but then at the same time there is pressure to stop working on it and instead deal with the unexpected emergencies. At the start I thought it was simpler than it really was and my design (which took longer than a sprint to think up) turned out to have flaws. It's embedded so the whole continual testing thing is extremely difficult, you can't put in unit tests when you don't even have enough memory to fit in the basics. And ideally, that two week sprint really should be 3 days of implementation only so that you have time for all the documentation, testing, handoff, training, integration, figuring out the obscure parts that come with no documentation, etc.
The ultimate thing that Agile is doing for me is making me work longer hours than I ever have in my life. That's the goal I think, it's why managers love it. Ie, I have to give a two week estimate of what I can get done. Now I feel personally responsible to get things done. The deadline is no longer an external deadline by people unfamiliar with what needs to be done but instead it is a self-imposed deadline. And self-imposed means I want to get it done so that I don't look foolish. Other people are waiting for it to be done so that they can do their part. If I do ask for more time I get glared at. And what happens now is that there is a deadline EVERY TWO WEEKS. It is ALWAYS crunch time! And there is still behind the scenes the high level deadline from the executives that can not slip.
Waterfall just cannot work.
You might want to talk to NASA and tell them all their missions have failed.
1. All the descions are taken at the time you know the least about the outcome, ie at the begining.
You know what that's a sign of? Failure to specify your requirements. (Something Agile people know so little of these days it's unsurprising they should be banned from programming)
2. By the time you get to the end your requirements are out of date, you get the product you wanted at the start of the project not the one you need now.
I need what I specified. If I need something else now, I should have added on more requirements. In fact, at the start of the project I should have added in an allowance that just maybe I will need to change something, or a dependency in the project may shift in time or not meet its capabilities. BTW, for you Agile people, this is where the Project Manager usually does his work. If they're managing which requirement you're coding and how, he's not doing his job.
If you try to support requirement change all you end up doing is replanning and pushing back delivery.,
If I was originally scheduled to make a simple sort in an adequate time period and then need to add in a complex multi-layered sort, um, yeah? What makes you think changed requirements wind up being deliverable in the same time frame? What vacuum do you live in? Are there pink ponies there too?
3. It was a failed methodology from the start, the original paper that started waterfall was an example of how not to write software
https://pragtob.wordpress.com/...
Again, I guess you better let all gov agencies know that all their projects involving software prior to 2005 or so failed. Guess that moon landing really was on a stage somewhere.
Having been exposed to a number of Agile projects that all ran over budget, schedule and/or failed, I can truthfully say Agile itself was the cause. I've detailed some of those elsewhere, and the response from the believers is always "Oh, you're doing it wrong" which is hilarious, because in multiple cases they were following exactly what little is laid out by the Agile camp. The real truth is that there are developers of different skills, and there are some that excel in specific areas while the rest plod along, and they are not interchangeable like Agile states they are unless you're going LCD. In fact, if you estimate your "velocity" by your worst performing programmer on an Agile team, you may, just may, have a realistic estimate of when you'll supposedly get to the finish line as long as no one throws any new requirements your way. As soon as they do, everything you think you know just went away. Sorry, my next SCRUM is pulling me out of this one...
The cesspool just got a check and balance.
That is exactly what happens. And, some of the leaders in the field have realized that a lot of what is called Agile (Poppendiek, Larman, Vodde, Leffingwell) are essentially implementations of Lean Product Development from Toyota to Software Development. What often happens is that, like Lean in manufacturing, organizations take up some of the forms of Lean/Agile while completely missing the fundamentals and the culture. That is key. If you don't realize that Agile or Lean are an organizational culture change not a set of practices that are a silver bullet what you end up doing is slapping Lean or Agile onto a dysfunctional culture. And, one of the things that many Agile practices will expose very quickly is all of the dysfunctions and waste built into an organization.
Consider the poster who complained about QA/QC being ignored. What was happening in waterfall was the exact same thing, but the long QA/QC cycle was hiding the fact that the developers were producing crap. Actually, calling it QA/QC was a misnomer, it was actually finish all the stuff we half-assed during the development phase. Oh and what management thought they would get from Agile was that they could knock out QA/QC and get the same work done in the same time that would have been allocated for the development phase in waterfall. So, the same schedule pressures that forced developers to take short cuts in waterfall never went away, and the result is that the same shortcuts happened without the QA/QC phase to fix them. And, was there a retrospective or root cause analysis or empowerment of individuals to pull the stop cord on the train to correct these issues that were exposed. Of course not because if you stop you will miss the schedule and that cannot happen, etc.. etc... Slapping Agile practices onto a dysfunctional organization does not fix the dysfunction. Culture changes take years, especially in larger organizations when there is not the top to bottom commitment to changing the entire organization.
There is also the problem of "flexible" translating to "without discipline". My experience is that Agile and Lean requires a level of discipline far beyond typical waterfall. The waterfall processes often require the facade of discipline, but rarely actual discipline especially at the individual level. And, when discipline is attempted on individuals it usually has more to do with imposing additional process and reporting that does not contribute to actually delivering results. Which is exactly the same thing another poster complains about where Agile gets used to micro-manage daily tasks. Which is another example of missing the point, daily tasks and a daily meeting is to free PMs and Managers from micro-managing holding each person accountable for accomplishing something each day. I call it peer pressure accountability because other than sociopaths most people don't want to be the guy who tells their team mates that they did not do what they said they would do yesterday, or did some ridiculously trivial task while some one else did something significant.
In some cases Agile shouldn't be used, but those are generally caused by external forces like being in an inflexible contract situation. In that case, I advise the philosophy of "Don't lie to yourself." Instead understand your constraints and realize a lot of the practices used by Agile and Lean organizations are almost universally beneficial. Automated testing and continuous integration are a good idea regardless. Splitting work into small chunks and working on a cadence (takt time) can be beneficial locally even though globally you are still producing tons of WIP. And, so on and so forth...