Slashdot Mirror
Is Agile Development a Failing Concept?
Nerval's Lobster writes: Many development teams have embraced Agile as the ideal method for software development, relying on cross-functional teams and adaptive planning to see their product through to the finish line. Agile has its roots in the Agile Manifesto, the product of 17 software developers coming together in 2001 to talk over development methods. And now one of those developers, Andy Hunt, has taken to his blog to argue that Agile has some serious issues. Specifically, Hunt thinks a lot of developers out there simply aren't adaptable and curious enough to enact Agile in its ideal form. 'Agile methods ask practitioners to think, and frankly, that's a hard sell,' Hunt wrote. 'It is far more comfortable to simply follow what rules are given and claim you're 'doing it by the book.'' The blog posting offers a way to power out of the rut, however, and it centers on a method that Hunt refers to as GROWS, or Growing Real-World Oriented Working Systems. In broad strokes, GROWS sounds a lot like Agile in its most fundamental form; presumably Hunt's future postings, which promise to go into more detail, will show how it differs. If Hunt wants the new model to catch on, he may face something of an uphill battle, given Agile's popularity.
DNRTFA, but I think this will be a fun thread.
Regardless of what Agile really is (a true scotsman?) the abuses perpetrated in the name of agile are appaling. I think a lot of people think agile means something like:
make bad developers good by not bothering to organise things properly
Which is really amazingly appealing if completely bogus.
SJW n. One who posts facts.
Like so many other things, it's very difficult to take an ideal theory and put it into practice in the real world. If your team really understands the ideas behind Agile and you have a good process in place to make it happen, you can have a great deal of success.
Unfortunately, like so many other things in life, most teams don't get it right and they end up failing to some degree or another.
It is hard to take someone seriously when the argument seems to be 'my idea is brilliant! but most people are not good enough to implement it!' rather than 'hrm, maybe my idea is not the universal solution and one needs to fit the methodology to the requirements and resources involved?'
No, it's just no longer selling as many books and consulting hours as it once did--so it's time to invent a new scam.
Agile, done right, has been a game changer here. Yes, we had to let some of our developers go - the ones who couldn't actually think. We're better off for it.
There is no way to optimize the development process for software. Inserting terms such as "Agile" or "Waterfall" really just create bloat and waste time. The best software development process is to have no process and work in the way that fits you best. For instance when I write large software projects, I just start coding, I pick a place to start at and go. I wait until I have large testable blocks completed and then debug and integrate them. I don't follow and form of standard development process and yet have never been held up via a deadline of meeting request.
When developers try and add those stupid terms, they're basically saying that they can't self manage and instead of taking responsibility, they're going to throw silly management methods around in an attempt to streamline a task which is unique to each individual developer and situation.
The only things you need to write good code are the right language, the right platform and proper requirements. Once you have those, you can just start and work to completion.
Our development and project management groups (~ 50 people) moved from waterfall to scrum/agile 2 years ago. Since then, we have seen a significant increase in quality and velocity. It is also more comfortable for the devs on the teams because they know exactly what the PM process will be and what is expected. I don't know that Scrum/Agile is the best practice out there but in our case, it has brought increased productivity, higher product quality, less worry and overall comfort with all our processes.
Error reading device 'Signature'. (A)bort, (R)etry, (F)ail?
Agile started failing pretty much as soon as it met the real world. I disagree with Andy Hunt's explanation of why, though. It's not because "thinking is hard", it's mostly because of two things: management not allowing agile to be done correctly, and (from the developer's point of view) it drains software engineering of the things that make it a satisfying and enjoyable activity, turning it into the software equivalent of grunt factory work.
That's not even close to enough time for a major cultural change to take place. The Agile Manifesto describes a culture of work that is so fundamentally different from how work was (and still is) performed, that I expect it will take another 15 to 30 years for organizations to really "get it". This is the same thing that happened with Lean manufacturing. Toyota developed it, other manufacturers adopted it as a fad over the course of about 15 years, and then it declined in popularity... but it never died out because it was "correct" and "good". Now, 40 years later, most manufacturers are still learning to be lean, but lean has fundamentally changed the culture of manufacturing. I have clients that will probably be working to adopt Agile methods over a 10 to 20 year period. Agile hasn't failed... Andy Hunt's patience has failed.
Helping with organizational effectiveness is our job.
I'll dub thee DevOps
Agile is not failing because there is nothing to replace it. Are we going to go back to Waterfall? Software development is inherently hard. No methodology is going to make it easy. But Agile works better than any other methodology that I have used, even when implemented piecemeal, by mediocre programmers, rather than in its "ideal form".
Ideas should be judged against their real world alternatives, not some perfect ideal. For Agile, there are no better alternatives. GROWS (whatever that is) may be an alternative, but it is untested, and looks like just an incoherent pile of buzz words.
As a system admin, I admire agile for the rapid proto-typing. Because as we all know, business users seldom know what they really want, but they all know what they don't like. However, I hate agile for being the universal excuse for turning project management into an exercise for "let's make it up as we go along", because then everyone expects me to work like that too. They don't want to acknowledge, let alone understand, that being a good system admin is about being organized and informed and having a more than 5 minute attention span.
Iterative, incremental development - the core of agile - has been around at least since Barry Boehm described the "spiral model" of development in 1986, and has been successfully used for nigh upon 30 years since under various monickers (and I'll bet there were practitioners before Boehm's paper).
"Agile" has matured and developed a lot of inter-related supporting practices and tools that have made it increasingly powerful and easy to implement. Continuous integration, test-driven development, use of "stories" as a tool for requirements definition, you cannot tell me these techniques and tools are not successful.
I have personally seen the development practices of a company dramatically transformed for the better by having an agile trainer brought in and training the entire staff, including managers, and instituting formal agile practices. It is great when a junior developer can tell the VP of marketing to take a hike because his request has not been submitted through the grooming and priority assignment process.
This one experience gives me complete confidence that people mocking agile simply do not know what they are talking about.
One problem agile does have is with zealots who don't understand that this is, and has to be, a collection of related practices that must be tailored to the needs of the environment, not a one-size-fits-all, all-or-nothing thing. Another problem is thinking that "form" is what is important, not "what is happening".
For example: holding stand-ups is not agile. It is a common, useful tool to use in an agile environment. If your team is coordinating with informal sessions as needed, Skype, chat tools, and an updated Wiki in real time, and the managers are keeping in the loop using these tools, then maybe a stand-up is a waste of time for you. I think most teams benefit, but design and planning is not part of a stand-up, other meetings are needed for these.
There can be long-term planning that does not follow the agile model, and can be described as water-fall, and this has its place too. But I think the only really successful development practices are variants of an "agile" type process.
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
Skip QA/QC procedures? Then you're not Done. That means you've completed 0 stories and get to finish them on your next Sprint.
it's management. When you get a good project manager its like a breath of fresh air. The best PM I ever worked for was a guy that used to be a developer and just didn't understand object based programming, after an honest assessment, so he decided to go into project management. He shielded us from all the corporate BS and just let us code.
Most of the other PM's I have worked for have no background in programming. Some of them claimed to and didn't, which is much worse than someone that just tells you they don't. They would insist on idiotic exchanges like the following:
PM: How long will it take to code this?
Me: I'm not sure until I get all of the requirements
PM: Can you give it a guess?
Me: Sure but what's the point? It won't be a very good guess.
PM: That's OK I just need something to put on the project plan
Me: *Bullshit radar is now on full alert* So you just want me to pull something out of my ass so that you can finish up your project plan? Is that it?
PM: Umm, well, no...it's not like that
Me: OK, fine. I'll give you numbers but they are going to be grossly inflated to account for the unknowns. It covers my ass. Kind of like what you are doing, no?
PM: *Grunts and walks away*
Most of these people look at project management as if we were building widgets on an assembly line. As if we know exactly how long each task is going to take. Well, software development is not not like that. Not in the least. The ones that understand that - the ones that are truly "Agile" as it were - are the successful ones. The successful ones understand that any number of things can go wrong and plan accordingly.
I do systems engineering work for a professional services/software company. Development is fully Agile with a capital A, whether or not it makes sense for a particular project. On the systems side of the house, we have another particular religion called ITIL which lots of companies have jumped into with both feet. The problem with both of these concepts is that they are adhered to, almost to a comical level, even if it's painfully obvious that parts of it don't fit.
Adhering to all of ITIL, for example, is a really good way to ensure your production systems almost never change. The number of people and sheer volume of paperwork, tickets and meetings to get anything even scheduled for a change in a "true ITIL" system is beyond insane. The same goes for incident management -- we have so many single-task focused "resolver groups" that I have no idea how anyone knows how any of our systems operate end-to-end. ITIL is great for mainframe systems, safety sensitive stuff, and networks which never change.
"True Agile" and "True Waterfall" are opposite ends of the spectrum. Agile gets you very fast development, at the expense of pinning down any sort of architecture in the beginning. Waterfall often results in software you have to throw away because the requirements change out from under you. However, there are some things that require at least some discipline, both in systems and development. No systems guy would ever advocate just logging in and making random changes on a production system to see what happens. No smart developer/architect charged with writing something that underpins tons and tons of other things would advocate swapping out the core components without at least some backward compatibility thrown in. The prpblem is that "gurus" make their money selling management on these methods. In the case of both Agile and ITIL, it's a manager's dream -- everyone becomes a replaceable unit and business requests can get promoted to production in one Sprint.
Why would anyone take something seriously that was created and peddled by consulting outfits at $700-per-hour bill rates? I had the misfortune of being incarcerated at Pivotal Labs for a month by a misguided boss who thought their bizarre religion was the answer to all of life's ills. Boy, was that eye-opening.
As many people rightly point out, that doesn't mean we can't pick up some new ideas from it - my company now does short daily meetings and have a chart with everyone's daily tasking on it, and those have proven very effective. But the other 99% of what the Snowbird 17 vomited forth upon our industry is empty zealotry and jingoism. It was like Scientology right in our codebases, and worked about as well. And no, for god's sake, it wasn't because we just weren't doing it well enough.
We do have shockingly dramatic quality issues in the software industry, but they will never be addressed by the next dumb-ass management fad. We need to sit down and re-think the ways that we learn to code, get serious about "Software Engineering" degrees in our colleges, and let go of fetishized code patterns as the primary unit of engineering ability. In my own experience, we know plenty enough design patterns, but almost no one understands how to exercise coding judgment in the context of a team or long-term project.
It can't fail cause all of the critics are doing it wrong.
Agreed. Too many companies use the Agile "be flexible" rule as a carte blanche, get-out-of-jail free card.
They pick and choose what parts they're going to do, which parts they're going to ignore, and which parts to which they're just going to pay lip service, and call the end result "agile". They don't understand how each part reinforces the other, and as such pick and choose among practices and ceremonies, and tell one another that they're being "flexible".
Failure is usually with implementation, not with Agile.
Any sect, cult, or religion will legislate its creed into law if it acquires the political power to do so.
it drains software engineering of the things that make it a satisfying and enjoyable activity, turning it into the software equivalent of grunt factory work
I'm confused. The satisfaction and joy of software engineering is turning a problem into a working solution.
The agile methods I followed let me realise that joy multiple times a day - checking in working code, and see it pass the automated test bed on the build server.
What is it that you perceive to be satisfying and enjoyable, and how are you losing that?
If you want to do it semi-scientifically, then test a given methodology for at least 7 years on actual projects for multiple project and organization types.
Why is it methodologies start spreading before being properly field tested? It appears to be hypsters jumping the gun in order to get rich selling hype (books, training, & consulting), but I'm open to other explanations.
Table-ized A.I.
Agile is failing in some areas, surviving in others. One snag is that Agile is a cult, pure and simple. If you vehemently disagree, then perhaps consider that you may be a cultist.
And the Agile cultists also believe there are only two world views: Agile versus Waterfall. Which is ridiculous, because the vast majority of developers use neither and instead have a hybrid approach from many different modesl including some ad-hoc methods. I know people who work as contractors who've said every single company they've been at use a different method, even those who claim to use Agile all use it in extremely different ways. When you are told essentially "do you really want to go back to Waterfall" then you know you're talking to a cultist.
Agile is pushed by its promoters as ideal for all types of projects. It does work well in some sorts of projects, where everything can be divided into tiny one or two week chunks, but it fails horribly where you need months of work for some features and lots of upfront design with large teams that need to coordinate. For example, taking an existing web design and maintaining it is easy with Agile, but designing an entire product from scratch, including hardware, software, and services will find Agile to be a frustration. What I have seen happen a lot is that there's a lot of old style design behind the scenes by the designers but then Agile is used up-front by the coders; each sprint picks out portions of the grand plan to work on, features are split into tiny two week slivers and so forth.
Don't forget the Agile industry here too. People whose high paying jobs are to teach Agile, to facilitate Agile, to be paid scrumm masters rather than developers, and so forth. I've seen no other development style that involves so many paid outsiders, most of whom are evangelists.
Of course there are good things with Agile but failing to acknowledge the bad things is not being objective.
Right now I'm on sprint number 10 for my feature. I get pressure to finish up the feature, but then at the same time there is pressure to stop working on it and instead deal with the unexpected emergencies. At the start I thought it was simpler than it really was and my design (which took longer than a sprint to think up) turned out to have flaws. It's embedded so the whole continual testing thing is extremely difficult, you can't put in unit tests when you don't even have enough memory to fit in the basics. And ideally, that two week sprint really should be 3 days of implementation only so that you have time for all the documentation, testing, handoff, training, integration, figuring out the obscure parts that come with no documentation, etc.
The ultimate thing that Agile is doing for me is making me work longer hours than I ever have in my life. That's the goal I think, it's why managers love it. Ie, I have to give a two week estimate of what I can get done. Now I feel personally responsible to get things done. The deadline is no longer an external deadline by people unfamiliar with what needs to be done but instead it is a self-imposed deadline. And self-imposed means I want to get it done so that I don't look foolish. Other people are waiting for it to be done so that they can do their part. If I do ask for more time I get glared at. And what happens now is that there is a deadline EVERY TWO WEEKS. It is ALWAYS crunch time! And there is still behind the scenes the high level deadline from the executives that can not slip.
Waterfall just cannot work.
You might want to talk to NASA and tell them all their missions have failed.
1. All the descions are taken at the time you know the least about the outcome, ie at the begining.
You know what that's a sign of? Failure to specify your requirements. (Something Agile people know so little of these days it's unsurprising they should be banned from programming)
2. By the time you get to the end your requirements are out of date, you get the product you wanted at the start of the project not the one you need now.
I need what I specified. If I need something else now, I should have added on more requirements. In fact, at the start of the project I should have added in an allowance that just maybe I will need to change something, or a dependency in the project may shift in time or not meet its capabilities. BTW, for you Agile people, this is where the Project Manager usually does his work. If they're managing which requirement you're coding and how, he's not doing his job.
If you try to support requirement change all you end up doing is replanning and pushing back delivery.,
If I was originally scheduled to make a simple sort in an adequate time period and then need to add in a complex multi-layered sort, um, yeah? What makes you think changed requirements wind up being deliverable in the same time frame? What vacuum do you live in? Are there pink ponies there too?
3. It was a failed methodology from the start, the original paper that started waterfall was an example of how not to write software
https://pragtob.wordpress.com/...
Again, I guess you better let all gov agencies know that all their projects involving software prior to 2005 or so failed. Guess that moon landing really was on a stage somewhere.
Having been exposed to a number of Agile projects that all ran over budget, schedule and/or failed, I can truthfully say Agile itself was the cause. I've detailed some of those elsewhere, and the response from the believers is always "Oh, you're doing it wrong" which is hilarious, because in multiple cases they were following exactly what little is laid out by the Agile camp. The real truth is that there are developers of different skills, and there are some that excel in specific areas while the rest plod along, and they are not interchangeable like Agile states they are unless you're going LCD. In fact, if you estimate your "velocity" by your worst performing programmer on an Agile team, you may, just may, have a realistic estimate of when you'll supposedly get to the finish line as long as no one throws any new requirements your way. As soon as they do, everything you think you know just went away. Sorry, my next SCRUM is pulling me out of this one...
The cesspool just got a check and balance.
Why "waterfall"? Why is every Agile evangelist so uptight about "waterfall", do they honestly think that there are only two possible methodologies in the world?
That is exactly what happens. And, some of the leaders in the field have realized that a lot of what is called Agile (Poppendiek, Larman, Vodde, Leffingwell) are essentially implementations of Lean Product Development from Toyota to Software Development. What often happens is that, like Lean in manufacturing, organizations take up some of the forms of Lean/Agile while completely missing the fundamentals and the culture. That is key. If you don't realize that Agile or Lean are an organizational culture change not a set of practices that are a silver bullet what you end up doing is slapping Lean or Agile onto a dysfunctional culture. And, one of the things that many Agile practices will expose very quickly is all of the dysfunctions and waste built into an organization.
Consider the poster who complained about QA/QC being ignored. What was happening in waterfall was the exact same thing, but the long QA/QC cycle was hiding the fact that the developers were producing crap. Actually, calling it QA/QC was a misnomer, it was actually finish all the stuff we half-assed during the development phase. Oh and what management thought they would get from Agile was that they could knock out QA/QC and get the same work done in the same time that would have been allocated for the development phase in waterfall. So, the same schedule pressures that forced developers to take short cuts in waterfall never went away, and the result is that the same shortcuts happened without the QA/QC phase to fix them. And, was there a retrospective or root cause analysis or empowerment of individuals to pull the stop cord on the train to correct these issues that were exposed. Of course not because if you stop you will miss the schedule and that cannot happen, etc.. etc... Slapping Agile practices onto a dysfunctional organization does not fix the dysfunction. Culture changes take years, especially in larger organizations when there is not the top to bottom commitment to changing the entire organization.
There is also the problem of "flexible" translating to "without discipline". My experience is that Agile and Lean requires a level of discipline far beyond typical waterfall. The waterfall processes often require the facade of discipline, but rarely actual discipline especially at the individual level. And, when discipline is attempted on individuals it usually has more to do with imposing additional process and reporting that does not contribute to actually delivering results. Which is exactly the same thing another poster complains about where Agile gets used to micro-manage daily tasks. Which is another example of missing the point, daily tasks and a daily meeting is to free PMs and Managers from micro-managing holding each person accountable for accomplishing something each day. I call it peer pressure accountability because other than sociopaths most people don't want to be the guy who tells their team mates that they did not do what they said they would do yesterday, or did some ridiculously trivial task while some one else did something significant.
In some cases Agile shouldn't be used, but those are generally caused by external forces like being in an inflexible contract situation. In that case, I advise the philosophy of "Don't lie to yourself." Instead understand your constraints and realize a lot of the practices used by Agile and Lean organizations are almost universally beneficial. Automated testing and continuous integration are a good idea regardless. Splitting work into small chunks and working on a cadence (takt time) can be beneficial locally even though globally you are still producing tons of WIP. And, so on and so forth...
Ridiculous. Maybe in stupid-ass punch-the-monkey dev shops writing web-based crap no one will care about in 6 months. There are plenty of industries where this is nonsense.
In international finance, some country decides on a new law (usually a new regulation). The software therefore must change.
Is the law going to change again this year? No.
Is the development cycle less than a year? Yes.
Is the development cycle longer than a fucking sprint? Yes.
Waterfall will work just FINE.
I had a small project which despite meet the deadline has took more than double of expected work hours.
Did I know I have to evaluate and change underlying platform 3 times so it could work with other dependencies flawlessly? No.
Did I know I have to write part of DB driver because some thing that has been in DB for years isn't supported in its official driver? No.
Did I know I have to dig into source code of dependent libraries and fix their bugs and even change part of their architecture to meet performance requirement? No.
See? None of these involves requirement changing. How you can do waterfall in large projects worth millions is beyond my imagination...