Slashdot Mirror
Is Agile Development a Failing Concept?
Nerval's Lobster writes: Many development teams have embraced Agile as the ideal method for software development, relying on cross-functional teams and adaptive planning to see their product through to the finish line. Agile has its roots in the Agile Manifesto, the product of 17 software developers coming together in 2001 to talk over development methods. And now one of those developers, Andy Hunt, has taken to his blog to argue that Agile has some serious issues. Specifically, Hunt thinks a lot of developers out there simply aren't adaptable and curious enough to enact Agile in its ideal form. 'Agile methods ask practitioners to think, and frankly, that's a hard sell,' Hunt wrote. 'It is far more comfortable to simply follow what rules are given and claim you're 'doing it by the book.'' The blog posting offers a way to power out of the rut, however, and it centers on a method that Hunt refers to as GROWS, or Growing Real-World Oriented Working Systems. In broad strokes, GROWS sounds a lot like Agile in its most fundamental form; presumably Hunt's future postings, which promise to go into more detail, will show how it differs. If Hunt wants the new model to catch on, he may face something of an uphill battle, given Agile's popularity.
DNRTFA, but I think this will be a fun thread.
Regardless of what Agile really is (a true scotsman?) the abuses perpetrated in the name of agile are appaling. I think a lot of people think agile means something like:
make bad developers good by not bothering to organise things properly
Which is really amazingly appealing if completely bogus.
SJW n. One who posts facts.
Like so many other things, it's very difficult to take an ideal theory and put it into practice in the real world. If your team really understands the ideas behind Agile and you have a good process in place to make it happen, you can have a great deal of success.
Unfortunately, like so many other things in life, most teams don't get it right and they end up failing to some degree or another.
It is hard to take someone seriously when the argument seems to be 'my idea is brilliant! but most people are not good enough to implement it!' rather than 'hrm, maybe my idea is not the universal solution and one needs to fit the methodology to the requirements and resources involved?'
Yes.
SJW n. One who posts facts.
No, it's just no longer selling as many books and consulting hours as it once did--so it's time to invent a new scam.
Sales / Management are on strict Waterfall and dev is on Agile
Until those concepts align in a company they will always be at odds with each other
There is no way to optimize the development process for software. Inserting terms such as "Agile" or "Waterfall" really just create bloat and waste time. The best software development process is to have no process and work in the way that fits you best. For instance when I write large software projects, I just start coding, I pick a place to start at and go. I wait until I have large testable blocks completed and then debug and integrate them. I don't follow and form of standard development process and yet have never been held up via a deadline of meeting request.
When developers try and add those stupid terms, they're basically saying that they can't self manage and instead of taking responsibility, they're going to throw silly management methods around in an attempt to streamline a task which is unique to each individual developer and situation.
The only things you need to write good code are the right language, the right platform and proper requirements. Once you have those, you can just start and work to completion.
Our development and project management groups (~ 50 people) moved from waterfall to scrum/agile 2 years ago. Since then, we have seen a significant increase in quality and velocity. It is also more comfortable for the devs on the teams because they know exactly what the PM process will be and what is expected. I don't know that Scrum/Agile is the best practice out there but in our case, it has brought increased productivity, higher product quality, less worry and overall comfort with all our processes.
Error reading device 'Signature'. (A)bort, (R)etry, (F)ail?
Agile started failing pretty much as soon as it met the real world. I disagree with Andy Hunt's explanation of why, though. It's not because "thinking is hard", it's mostly because of two things: management not allowing agile to be done correctly, and (from the developer's point of view) it drains software engineering of the things that make it a satisfying and enjoyable activity, turning it into the software equivalent of grunt factory work.
Agile is great in theory and if you can get everyone involved to understand it then it's great in practice, too. The reason I say agile is a failing concept is because those of us in the industry understand it but are remarkably terrible at selling it to the non-technical people we need to have involved in order for our projects to be successful. What good Agile methodology really drives at is an effective amount of involvement from all parties -- customer, analyst, technical team, testers, operations, etc. Most businesses want to fire things at their IT department and then go off and do other things while the project is underway. This is clearly more conducive to the "waterfall" model (which we all know is terrible) and it prevents them from ever seeing and effective agile development implementation.
So, yes, Agile is a failing concept. Not because the idea bad, but because it's so incredibly difficult to implement fully, and it's not very valuable when partially implemented (you basically just turn it into mini-waterfalls).
I foresee DevOps ending up with a similar problem.
That's not even close to enough time for a major cultural change to take place. The Agile Manifesto describes a culture of work that is so fundamentally different from how work was (and still is) performed, that I expect it will take another 15 to 30 years for organizations to really "get it". This is the same thing that happened with Lean manufacturing. Toyota developed it, other manufacturers adopted it as a fad over the course of about 15 years, and then it declined in popularity... but it never died out because it was "correct" and "good". Now, 40 years later, most manufacturers are still learning to be lean, but lean has fundamentally changed the culture of manufacturing. I have clients that will probably be working to adopt Agile methods over a 10 to 20 year period. Agile hasn't failed... Andy Hunt's patience has failed.
Helping with organizational effectiveness is our job.
I'll dub thee DevOps
No.
But it'll be yes again by the time of the next scrum.
Agile is not failing because there is nothing to replace it. Are we going to go back to Waterfall? Software development is inherently hard. No methodology is going to make it easy. But Agile works better than any other methodology that I have used, even when implemented piecemeal, by mediocre programmers, rather than in its "ideal form".
Ideas should be judged against their real world alternatives, not some perfect ideal. For Agile, there are no better alternatives. GROWS (whatever that is) may be an alternative, but it is untested, and looks like just an incoherent pile of buzz words.
Are we going to go back to Waterfall?
In my experience, that would be greatly preferable to Agile.
But Agile works better than any other methodology that I have used, even when implemented piecemeal, by mediocre programmers, rather than in its "ideal form".
My experience is just the opposite: agile is very nearly the worst of the methodologies I've used. The one great thing about it is something that can be done in any methodology: increased communications. We can throw out the bathwater and keep that baby.
I second the management thing. None of the managers want to change the way they manage things (I need a schedule for your work for the next 6 months!) in away that prevents any of the advantages of Agile from being fully realized.
Agile is not failing because there is nothing to replace it. Are we going to go back to Waterfall?
Quo usque tandem abutere, Nimbus, patientia nostra?
That's it, exactly. You can use agile to weed out developers who can't think for themselves, who really are of no use in a development team anyway.
The methodology debate kind of misses the point. Agile is no silver bullet. A high-functioning team can be successful with agile, or with various other methodologies for that matter.
A dysfunctional team isn't going to succeed with agile, or with anything else.
There are more choices than just Agile and Waterfall.
Nah - that one won't get as far - it usually requires that people prove their ability as both a sysadmin *and* as a developer (or at least as a member of a dev team). Most applicants choke hard on one or the other in the interview, and I'm not seeing a credible 'certification' yet that can paper over that deficiency.
Quo usque tandem abutere, Nimbus, patientia nostra?
The articles to which TFS links point have more than a few links and references to other articles, blogs and books (yes, "see my article/book") written by Andy and the "gang of 17". Not exactly astroturfing, but certainly rather self-promotional. "Agile" is one methodology, appropriate for some situations, but certainly not the "one ring to rule them all." (if I recall, wearing that ring had some negative effects...) Now he wants us to move on to: "The GROWS (TM) Method."
All this probably benefits someone, not sure it's always us.
It must have been something you assimilated. . . .
I've got to agree with JohnFen. As a Program Manager, while Waterfall techniques could frequently end up with late or over budget, or both, projects, at the end of every project (I oversaw 5 multimillion dollar projects using Waterfall methods) we at least had a working application that met the original specifications.
Now, after two similarly sized Agile projects, all I can say is it seems to be an excuse for developers to skip QA/QC procedures "because we're already into the next scrum" and end up with a mess that doesn't come close to matching the original specification at the end blaming changing requirements and "developmental issues" during the scrum process. I just turned down a contract that explicitly required Agile coding because I don't have any confidence that the end user will be satisfied with the results.
"Unheard of means only it's undreamed of yet,
Impossible means not yet done." ~~ Julia Ecklar
As a system admin, I admire agile for the rapid proto-typing. Because as we all know, business users seldom know what they really want, but they all know what they don't like. However, I hate agile for being the universal excuse for turning project management into an exercise for "let's make it up as we go along", because then everyone expects me to work like that too. They don't want to acknowledge, let alone understand, that being a good system admin is about being organized and informed and having a more than 5 minute attention span.
So true. When we help an organization "go Agile" it is critical that the managers also use Agile and that they stick with it. But this doesn't mean exactly "by the book" since, for example, Scrum might not be the best approach for a management team. Kanban, OpenAgile, Crystal or other Agile methods or techniques might work better for any given team (including a management team). Long term success of Agile methods in an organization requires that management become Agile too.
Helping with organizational effectiveness is our job.
Iterative, incremental development - the core of agile - has been around at least since Barry Boehm described the "spiral model" of development in 1986, and has been successfully used for nigh upon 30 years since under various monickers (and I'll bet there were practitioners before Boehm's paper).
"Agile" has matured and developed a lot of inter-related supporting practices and tools that have made it increasingly powerful and easy to implement. Continuous integration, test-driven development, use of "stories" as a tool for requirements definition, you cannot tell me these techniques and tools are not successful.
I have personally seen the development practices of a company dramatically transformed for the better by having an agile trainer brought in and training the entire staff, including managers, and instituting formal agile practices. It is great when a junior developer can tell the VP of marketing to take a hike because his request has not been submitted through the grooming and priority assignment process.
This one experience gives me complete confidence that people mocking agile simply do not know what they are talking about.
One problem agile does have is with zealots who don't understand that this is, and has to be, a collection of related practices that must be tailored to the needs of the environment, not a one-size-fits-all, all-or-nothing thing. Another problem is thinking that "form" is what is important, not "what is happening".
For example: holding stand-ups is not agile. It is a common, useful tool to use in an agile environment. If your team is coordinating with informal sessions as needed, Skype, chat tools, and an updated Wiki in real time, and the managers are keeping in the loop using these tools, then maybe a stand-up is a waste of time for you. I think most teams benefit, but design and planning is not part of a stand-up, other meetings are needed for these.
There can be long-term planning that does not follow the agile model, and can be described as water-fall, and this has its place too. But I think the only really successful development practices are variants of an "agile" type process.
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
I've been watching the idea since the early 00's. I've been on teams that have adapted the processes to work for the team and have been very successful doing so. I've seen a team get a cadence going and become extremely accurate at estimating new work for a product the same 5 people worked on for 5 years. During that time they also dramatically improved the quality of the code, reducing crashes that required weekend coverage to almost 0. Every once in a while they'd adjust their processes if things weren't working smoothly. Teams can work very effectively in an agile environment, if they're actually allowed to.
If you follow the evolution of agile, you see a lot of key concepts that get repeated over and over. The guys who wrote it understood that code is never perfect and never really correct the first time you write it. It pushes unit testing as a core component of the process. As with other things, making mistakes and correct them teaches you something about the problem, and so the whole process is designed around uncovering those mistakes quickly, throwing code away and rewriting it and constantly improving quality. The philosophy of most companies is that the developers should just crap something out that kind of works and then move on.
What it basically comes down to is just because your team is agile doesn't mean you can hire chimpanzees to write your code. Or manage the team. If you're looking for a silver bullet that will fix what's wrong with your company, agile isn't it. It enforces much more discipline than whatever crappy process you were using before that, but you really have to understand what it's about, and most people don't.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
I also have to agree but for different reasons. What I've experienced is Agile as excuse for micromanagement. Projects take much longer than they used to because we now agonize in meeting after meeting over details that used to be left to the developer to decide. Agile is a recipe for managing programmers fresh out of college perhaps, but most I work with aren't those, and they work better when you trust them with more of the detais and have management worry about the bigger pictures instead...
Skip QA/QC procedures? Then you're not Done. That means you've completed 0 stories and get to finish them on your next Sprint.
Waterfall just cannot work.
1. All the descions are taken at the time you know the least about the outcome, ie at the begining.
2. By the time you get to the end your requirements are out of date, you get the product you wanted at the start of the project not the one you need now. If you try to support requirement change all you end up doing is replanning and pushing back delivery.,
3. It was a failed methodology from the start, the original paper that started waterfall was an example of how not to write software
https://pragtob.wordpress.com/...
it's management. When you get a good project manager its like a breath of fresh air. The best PM I ever worked for was a guy that used to be a developer and just didn't understand object based programming, after an honest assessment, so he decided to go into project management. He shielded us from all the corporate BS and just let us code.
Most of the other PM's I have worked for have no background in programming. Some of them claimed to and didn't, which is much worse than someone that just tells you they don't. They would insist on idiotic exchanges like the following:
PM: How long will it take to code this?
Me: I'm not sure until I get all of the requirements
PM: Can you give it a guess?
Me: Sure but what's the point? It won't be a very good guess.
PM: That's OK I just need something to put on the project plan
Me: *Bullshit radar is now on full alert* So you just want me to pull something out of my ass so that you can finish up your project plan? Is that it?
PM: Umm, well, no...it's not like that
Me: OK, fine. I'll give you numbers but they are going to be grossly inflated to account for the unknowns. It covers my ass. Kind of like what you are doing, no?
PM: *Grunts and walks away*
Most of these people look at project management as if we were building widgets on an assembly line. As if we know exactly how long each task is going to take. Well, software development is not not like that. Not in the least. The ones that understand that - the ones that are truly "Agile" as it were - are the successful ones. The successful ones understand that any number of things can go wrong and plan accordingly.
I do systems engineering work for a professional services/software company. Development is fully Agile with a capital A, whether or not it makes sense for a particular project. On the systems side of the house, we have another particular religion called ITIL which lots of companies have jumped into with both feet. The problem with both of these concepts is that they are adhered to, almost to a comical level, even if it's painfully obvious that parts of it don't fit.
Adhering to all of ITIL, for example, is a really good way to ensure your production systems almost never change. The number of people and sheer volume of paperwork, tickets and meetings to get anything even scheduled for a change in a "true ITIL" system is beyond insane. The same goes for incident management -- we have so many single-task focused "resolver groups" that I have no idea how anyone knows how any of our systems operate end-to-end. ITIL is great for mainframe systems, safety sensitive stuff, and networks which never change.
"True Agile" and "True Waterfall" are opposite ends of the spectrum. Agile gets you very fast development, at the expense of pinning down any sort of architecture in the beginning. Waterfall often results in software you have to throw away because the requirements change out from under you. However, there are some things that require at least some discipline, both in systems and development. No systems guy would ever advocate just logging in and making random changes on a production system to see what happens. No smart developer/architect charged with writing something that underpins tons and tons of other things would advocate swapping out the core components without at least some backward compatibility thrown in. The prpblem is that "gurus" make their money selling management on these methods. In the case of both Agile and ITIL, it's a manager's dream -- everyone becomes a replaceable unit and business requests can get promoted to production in one Sprint.
DING!
If by saying QA/QC isn't completed you mean that unit and functional testing is missing, then the developer is not done. If you have problems with the developer not writing these tests, then be sure that "Definition of Done" includes some acceptable target level of unit/functional testing.
If on the other hand you get to the end of a story and accept it only to find out that it doesn't meet your QA standards, then you as a product manager haven't done your job in properly validating the story prior to acceptance; add to your own procedures the time required to properly validate the story for acceptance. Maybe you need a testing resource to do this if you are overworked as a product manager - an assistant product manager, even.
Agile is as good as you make it.
Let us live so that when we come to die, even the undertaker will be sorry -- Mark Twain
Why would anyone take something seriously that was created and peddled by consulting outfits at $700-per-hour bill rates? I had the misfortune of being incarcerated at Pivotal Labs for a month by a misguided boss who thought their bizarre religion was the answer to all of life's ills. Boy, was that eye-opening.
As many people rightly point out, that doesn't mean we can't pick up some new ideas from it - my company now does short daily meetings and have a chart with everyone's daily tasking on it, and those have proven very effective. But the other 99% of what the Snowbird 17 vomited forth upon our industry is empty zealotry and jingoism. It was like Scientology right in our codebases, and worked about as well. And no, for god's sake, it wasn't because we just weren't doing it well enough.
We do have shockingly dramatic quality issues in the software industry, but they will never be addressed by the next dumb-ass management fad. We need to sit down and re-think the ways that we learn to code, get serious about "Software Engineering" degrees in our colleges, and let go of fetishized code patterns as the primary unit of engineering ability. In my own experience, we know plenty enough design patterns, but almost no one understands how to exercise coding judgment in the context of a team or long-term project.
Which is true. But here we are talking about tools, concepts and processes that are known to produce good results when used intelligently (or that should be known by anyone worth his/her salt in this industry.)
No, we have a process that is full of "no true scotsman" defenders who do no introspection on why the methodology has numerous notable failures beyond blaming everyone but themselves. And I say that as someone who does like many aspects of Agile, but it is far oversold on what it can offer by its true believer priests.
Agile is nothing but an admission by clueless marketing/business development folks that they're terrible at their jobs. They have no idea how to do market research, they have no idea how to interact with actual or potential customers, they have no idea what a company's products actually do and what solutions they provide, they have no idea how to do the analytical side of their job, and have no vision at all for their industry. So instead they shove all the responsibility onto the development team: hack a small iteration together quick, show it to a customer (or more likely, show it to a cross-function representation of development groups within the company, none of which is trained in market analysis), and repeat until someone in upper management says "hey, this was supposed to be released this quarter". Then marketing will swoop in and offer to put together some glossies.
Obviously for situations like with a small start-up, you aren't going to have a well-rounded business development unit, and you'll be forced into having developers and other non-specialists pick up the business development slack. That's when agile makes sense: when you have no other option. But big companies with full business development and analysis teams pushing agile on its developers is nothing but welfare for idiot marketers.
None of those are failings of waterfall at all. There is nothing about waterfall that requires you to make ironclad decisions at the very start, and there is nothing that prevents you from adapting the course of development as the project proceeds.
In other words, you aren't describing waterfall in your comment. Yes, I'm invoking the "No True Scotsman" fallacy, since that is usually what is invoked when Agile is criticized.
The real truth is that all methodologies can be done well or poorly, including waterfall and agile. The difference that I've seen in practice is that it's incredibly hard to implement Agile correctly (such that I've never seen it done), but implementing waterfall correctly is not a huge chore.
It can't fail cause all of the critics are doing it wrong.
I started coining the term "Chunked Waterfall". Take all the inefficencies of waterfall, split it into chunks and add more inefficiencies due to management ;-)
Whenever somebody tells me they are doing agile I ask them how their chunked waterfall methodology will help them once they realize that 50% of their assumptions on a project are wrong and the market has changed while they were busy with scum meetings.? Denial is usually the response....
At its inception, the Agile manifesto was simple. Four priority/value statements and then a list of simple principles. The goal? Merely to say that delivering value to the customer, collaborating with customers, frequent delivery and feedback, and team empowerment are the way to deliver software. Focus on delivering value. Don't focus on delivering things that aren't valuable. Very simple.
Once Agile values started to become embraced and a couple of new development processes came along (SCRUM, etc), you all of a sudden had a bunch of consulting companies and community meetups appear that all but destroyed the perception of Agile. For these companies and community groups, it's all about the process. They will teach you how to "do agile". They will provide you with bodies/contractors who can "do agile". They will sell you certifications which show you "do agile". They will sell you seminars and training on how to "do agile". They will sell you software which "does agile". Agile has went from a basic set of values to becoming a fundamentalist religion.
So my statement to "Agile Process Improvement" firms is this: You are all just scammers and profiteers. You are software development Pharisees. It is amazing that you focus on profiting from creating processes, enforcing processes, teaching processes, and writing process software... for a methodology where the first value statement is "Individuals and interactions over processes and tools". Why don't you guys teach REAL agile? Why don't you teach companies how to better define value and deliver it to customers instead of selling new processes, fundamentalism, and bodies?
For the rest of you, if you want to be "Agile". Read the Agile manifesto. Create your own process that suits your team and your business. Work continuously with your customers to understand what is valuable to them. Deliver good value to them often and get their feedback. Allow your team to learn and grow and understand the needs of the customer. THAT is Agile. THAT is all you need to do.
I always find it funny for something named Agile and aimed at being responsive to needs that people are so "by the book" about it. I find it oxymoronic that there's "one true way" to do something called Agile.
Some employers are so by the book that they have to have a physical whiteboard with postits even though they also have to have jira and keep both those in sync. The purist agile says no remoting- face to face only, which I think is incorrect- I think "some kind of verbal or visual chat" is sufficient, but the key is communication beyond say hipchat and jira and email.
Some employers claim to be really purist about it and yet depart in significant ways. I think a lot of employers also use Agile as a way to squeeze long hours out of devs at the end of every sprint even though "purist" agile says 40 hour work week.
I generally like agile and it took a while for me to understand that MVP doesn't mean do the bare minimum for the sake of doing the bare minimum but with the idea you get it to the customer for feedback sooner and you iterate.
Management push-back is a tough one and understandable. They want to know where the company is going in a quarter, two quarters, next year. That means big plans, and it means estimating the size of things way in advance. That's something that Agile is specifically designed to avoid - unnecessary advance planning. I think this conflict exists (or should exist) even in the best Agile development shops. The alternative is the ultimate in management short-sightedness - no plan for the future, just get through the quarter. On our team the compromise is doing some one quarter rough-grooming and having our engineering team manager (who was a team member before being promoted) flesh out the more distant epic level grooming with our PM.
Let us live so that when we come to die, even the undertaker will be sorry -- Mark Twain
Well, my take on it is that agile is not actually Agile.
ie, all the rubbish people do to pretend they're working in an agile way is just an excuse to do far less work and far more process. Just the opposite of what Agile is all about.
Alistair Cockburn said it in his Shu Ha Ri page - agile is about Put 4-6 people in a room with workstations and whiteboards and access to the users. Have them deliver running, tested software to the users every one or two months, and otherwise leave them alone
It is not about daily meetings, more meetings, more review meetings, postits in place of documentation, more meetings to discuss what postts to put in the meeting you're going to have the next day to confirm the postits you decided would be in the next planning process...
I think I should start a new agile methodology - the bugtracker agile system.
You have a bug tracker (where bug also means task, requirement, change or just plain bug) with as many bugs in it as you can think of to get the project going (should be easy - you know what you want after all). Then you tell your dev team - here's the bug list, get on with it. I'll be back in a month to see how you're getting on, you'd better have something to show me - tech docs at least if not some form of running product. If you have any questions, ask Dave the customer liaison chap (or tech architect fellow, or product owner bloke), he'll clarify any confusion in the requirements.
And that's it. Trouble is, I doubt I'd be able to sell many books or conferences with that. Pity, 'cos it works.
We effectively do without a scrum master. It can be done with an open and communicative team, so long as everyone recognizes the rules of the game and is willing to speak up to guide the process. Product owner buy-in is essential (and a scrum master IMHO an essential backup if the PM is fighting the system); they don't make the system work, but without good backlog management they can make the system break.
Our team succeeds at Agile more than anything else because our developers are respected by management. Our product owners and management have both wanted longer term planning and a more waterfall process because it's easier for them, but we have the ability to push back, and they have the respect to listen.
Let us live so that when we come to die, even the undertaker will be sorry -- Mark Twain
I've got to agree with JohnFen. As a Program Manager, while Waterfall techniques could frequently end up with late or over budget, or both, projects, at the end of every project (I oversaw 5 multimillion dollar projects using Waterfall methods) we at least had a working application that met the original specifications. Now, after two similarly sized Agile projects, all I can say is it seems to be an excuse for developers to skip QA/QC procedures "because we're already into the next scrum" and end up with a mess that doesn't come close to matching the original specification at the end blaming changing requirements and "developmental issues" during the scrum process. I just turned down a contract that explicitly required Agile coding because I don't have any confidence that the end user will be satisfied with the results.
Having participated in both; I can see benefits either way. However, allowing the developers to do what you've said is a fault in the management of the project. QA/QC/QE is part of Agile. If need be, you just add tasks (stories) in Agile to do it and make sure they get done - if not, you're not managing the project correctly.
Agile itself is about being responsive to the needs of the project, and that includes all the QA/QC/QE stuff, as well as Security, Bugs, Changing Requirements, etc.
Now, if you're working in an organization that will enforce that the requirements will not change (do any such organizations exist?!!) then Waterfall is probably better than Agile.
Truth is like the sun. You can shut it out for a time, but it ain't goin' away. - Elvis Presley (source: imdb.com)
There are more choices than just Agile and Waterfall.
I think Iterative and incremental development is a good approach.
Agreed. Too many companies use the Agile "be flexible" rule as a carte blanche, get-out-of-jail free card.
They pick and choose what parts they're going to do, which parts they're going to ignore, and which parts to which they're just going to pay lip service, and call the end result "agile". They don't understand how each part reinforces the other, and as such pick and choose among practices and ceremonies, and tell one another that they're being "flexible".
Failure is usually with implementation, not with Agile.
Any sect, cult, or religion will legislate its creed into law if it acquires the political power to do so.
it drains software engineering of the things that make it a satisfying and enjoyable activity, turning it into the software equivalent of grunt factory work
I'm confused. The satisfaction and joy of software engineering is turning a problem into a working solution.
The agile methods I followed let me realise that joy multiple times a day - checking in working code, and see it pass the automated test bed on the build server.
What is it that you perceive to be satisfying and enjoyable, and how are you losing that?
Agile is failing in some areas, surviving in others. One snag is that Agile is a cult, pure and simple. If you vehemently disagree, then perhaps consider that you may be a cultist.
And the Agile cultists also believe there are only two world views: Agile versus Waterfall. Which is ridiculous, because the vast majority of developers use neither and instead have a hybrid approach from many different modesl including some ad-hoc methods. I know people who work as contractors who've said every single company they've been at use a different method, even those who claim to use Agile all use it in extremely different ways. When you are told essentially "do you really want to go back to Waterfall" then you know you're talking to a cultist.
Agile is pushed by its promoters as ideal for all types of projects. It does work well in some sorts of projects, where everything can be divided into tiny one or two week chunks, but it fails horribly where you need months of work for some features and lots of upfront design with large teams that need to coordinate. For example, taking an existing web design and maintaining it is easy with Agile, but designing an entire product from scratch, including hardware, software, and services will find Agile to be a frustration. What I have seen happen a lot is that there's a lot of old style design behind the scenes by the designers but then Agile is used up-front by the coders; each sprint picks out portions of the grand plan to work on, features are split into tiny two week slivers and so forth.
Don't forget the Agile industry here too. People whose high paying jobs are to teach Agile, to facilitate Agile, to be paid scrumm masters rather than developers, and so forth. I've seen no other development style that involves so many paid outsiders, most of whom are evangelists.
Of course there are good things with Agile but failing to acknowledge the bad things is not being objective.
Right now I'm on sprint number 10 for my feature. I get pressure to finish up the feature, but then at the same time there is pressure to stop working on it and instead deal with the unexpected emergencies. At the start I thought it was simpler than it really was and my design (which took longer than a sprint to think up) turned out to have flaws. It's embedded so the whole continual testing thing is extremely difficult, you can't put in unit tests when you don't even have enough memory to fit in the basics. And ideally, that two week sprint really should be 3 days of implementation only so that you have time for all the documentation, testing, handoff, training, integration, figuring out the obscure parts that come with no documentation, etc.
The ultimate thing that Agile is doing for me is making me work longer hours than I ever have in my life. That's the goal I think, it's why managers love it. Ie, I have to give a two week estimate of what I can get done. Now I feel personally responsible to get things done. The deadline is no longer an external deadline by people unfamiliar with what needs to be done but instead it is a self-imposed deadline. And self-imposed means I want to get it done so that I don't look foolish. Other people are waiting for it to be done so that they can do their part. If I do ask for more time I get glared at. And what happens now is that there is a deadline EVERY TWO WEEKS. It is ALWAYS crunch time! And there is still behind the scenes the high level deadline from the executives that can not slip.
Waterfall just cannot work.
1. All the descions are taken at the time you know the least about the outcome, ie at the begining.
2. By the time you get to the end your requirements are out of date, you get the product you wanted at the start of the project not the one you need now.
Ridiculous. Maybe in stupid-ass punch-the-monkey dev shops writing web-based crap no one will care about in 6 months. There are plenty of industries where this is nonsense.
In international finance, some country decides on a new law (usually a new regulation). The software therefore must change.
Is the law going to change again this year? No.
Is the development cycle less than a year? Yes.
Is the development cycle longer than a fucking sprint? Yes.
Waterfall will work just FINE.
If Agile is failing despite voluminous developer output, that's where the rubber hits the road.
In real world development, you are going to have acceptance criteria written to your PMs level of understanding of the product. Most businesses don't understand product managers, they understand *project managers*, and since project managers are just bookkeepers, organizers, and communication traffic facilitators, meant to be fungible across many types of projects, it just all goes to shit.
The experts who know the system can't be bothered to write the stories, and the project manager has no fucking clue what they are typing up.
Here's the stupidest thing ever: a company that develops with Agile, yet still has Subject Matter Experts communicating through the project managers. Of course you have to have a project manager, because it isn't a single team, or even all Agile teams, and the teams need to coordinate their efforts. And of course those SME's are too busy answering everyone's questions to be bothered to write out the specs properly.
Agile is like most high school physics: it works *perfectly* in low-friction environments.
Now, the pro-Agile people are going to say you aren't doing it right, etc. etc. You people are like communists: it works if you do it right! No, it doesn't, because it doesn't take into account corruption, greed, politics, propaganda, and deliberate mis-control of information for the benefit of a few. And those things are going to exist in any large-enough group of people.
Agile will not fix your organizational problems. It only shines a spotlight on them. Usually, when you point this out, everyone will nod their heads. They will all agree it needs to change. The problem is that no one with the power to actually fix the issue is in the room.
My organization accounts for QA/QC with the definition of "done"; QA is not a second class citizen of the organization, but rather a crucial part of the development team/process - and the story is not DONE until QA says it is. Therefore it rolls over across iteration boundaries as needed, and is only demo'ed when it is done.
The problem we have had with Agile thus far seems to be our inability to produce accurate estimates without doing Big Design Up Front which ultimately means spiking every story before we can get started. Nearly every time we try to shoot from the hip on story estimation for anything moderately complex (or worse), we have missed by several multiples the actual amount of work needed.
This is mainly due to the product being very complex (think enterprise scale SaaS, tens of millions of users, terabytes of data, complex data modeling, and numerous technologies being adapted with a variety of API/interfacing solutions) with many interconnected systems across multiple data centers and cloud services... you just can't stare at a story in the backlog and come up with a meaningful estimate off the top of your head no matter how well defined the acceptance criteria are because no one person knows what the potential impact is to all those systems.
But we're committed to working on improving our processes, cross-training, and reduction of overall system complexity to eventually be able to do just that and are sticking with Agile because it has forced us to take smaller bites which has really been a challenge for our sales/marketing and product owner teams because they want the world and they want it yesterday... and Agile empowers the scrum team to give them a reality check and say no.
I apologize for the run-on sentences... too lazy to edit at the moment.
Waterfall just cannot work.
You might want to talk to NASA and tell them all their missions have failed.
1. All the descions are taken at the time you know the least about the outcome, ie at the begining.
You know what that's a sign of? Failure to specify your requirements. (Something Agile people know so little of these days it's unsurprising they should be banned from programming)
2. By the time you get to the end your requirements are out of date, you get the product you wanted at the start of the project not the one you need now.
I need what I specified. If I need something else now, I should have added on more requirements. In fact, at the start of the project I should have added in an allowance that just maybe I will need to change something, or a dependency in the project may shift in time or not meet its capabilities. BTW, for you Agile people, this is where the Project Manager usually does his work. If they're managing which requirement you're coding and how, he's not doing his job.
If you try to support requirement change all you end up doing is replanning and pushing back delivery.,
If I was originally scheduled to make a simple sort in an adequate time period and then need to add in a complex multi-layered sort, um, yeah? What makes you think changed requirements wind up being deliverable in the same time frame? What vacuum do you live in? Are there pink ponies there too?
3. It was a failed methodology from the start, the original paper that started waterfall was an example of how not to write software
https://pragtob.wordpress.com/...
Again, I guess you better let all gov agencies know that all their projects involving software prior to 2005 or so failed. Guess that moon landing really was on a stage somewhere.
Having been exposed to a number of Agile projects that all ran over budget, schedule and/or failed, I can truthfully say Agile itself was the cause. I've detailed some of those elsewhere, and the response from the believers is always "Oh, you're doing it wrong" which is hilarious, because in multiple cases they were following exactly what little is laid out by the Agile camp. The real truth is that there are developers of different skills, and there are some that excel in specific areas while the rest plod along, and they are not interchangeable like Agile states they are unless you're going LCD. In fact, if you estimate your "velocity" by your worst performing programmer on an Agile team, you may, just may, have a realistic estimate of when you'll supposedly get to the finish line as long as no one throws any new requirements your way. As soon as they do, everything you think you know just went away. Sorry, my next SCRUM is pulling me out of this one...
The cesspool just got a check and balance.
Once Agile values started to become embraced and a couple of new development processes came along (SCRUM, etc)
That's a nice thought, but SCRUM actually predates Agile by half a decade. SCRUM was introduced ~1995, and Agile ~2001.
Though the actual Agile techniques are much, much older. Reading the Mythical Man Month can convince you of that.
"First they came for the slanderers and i said nothing."
Why "waterfall"? Why is every Agile evangelist so uptight about "waterfall", do they honestly think that there are only two possible methodologies in the world?
That is exactly what happens. And, some of the leaders in the field have realized that a lot of what is called Agile (Poppendiek, Larman, Vodde, Leffingwell) are essentially implementations of Lean Product Development from Toyota to Software Development. What often happens is that, like Lean in manufacturing, organizations take up some of the forms of Lean/Agile while completely missing the fundamentals and the culture. That is key. If you don't realize that Agile or Lean are an organizational culture change not a set of practices that are a silver bullet what you end up doing is slapping Lean or Agile onto a dysfunctional culture. And, one of the things that many Agile practices will expose very quickly is all of the dysfunctions and waste built into an organization.
Consider the poster who complained about QA/QC being ignored. What was happening in waterfall was the exact same thing, but the long QA/QC cycle was hiding the fact that the developers were producing crap. Actually, calling it QA/QC was a misnomer, it was actually finish all the stuff we half-assed during the development phase. Oh and what management thought they would get from Agile was that they could knock out QA/QC and get the same work done in the same time that would have been allocated for the development phase in waterfall. So, the same schedule pressures that forced developers to take short cuts in waterfall never went away, and the result is that the same shortcuts happened without the QA/QC phase to fix them. And, was there a retrospective or root cause analysis or empowerment of individuals to pull the stop cord on the train to correct these issues that were exposed. Of course not because if you stop you will miss the schedule and that cannot happen, etc.. etc... Slapping Agile practices onto a dysfunctional organization does not fix the dysfunction. Culture changes take years, especially in larger organizations when there is not the top to bottom commitment to changing the entire organization.
There is also the problem of "flexible" translating to "without discipline". My experience is that Agile and Lean requires a level of discipline far beyond typical waterfall. The waterfall processes often require the facade of discipline, but rarely actual discipline especially at the individual level. And, when discipline is attempted on individuals it usually has more to do with imposing additional process and reporting that does not contribute to actually delivering results. Which is exactly the same thing another poster complains about where Agile gets used to micro-manage daily tasks. Which is another example of missing the point, daily tasks and a daily meeting is to free PMs and Managers from micro-managing holding each person accountable for accomplishing something each day. I call it peer pressure accountability because other than sociopaths most people don't want to be the guy who tells their team mates that they did not do what they said they would do yesterday, or did some ridiculously trivial task while some one else did something significant.
In some cases Agile shouldn't be used, but those are generally caused by external forces like being in an inflexible contract situation. In that case, I advise the philosophy of "Don't lie to yourself." Instead understand your constraints and realize a lot of the practices used by Agile and Lean organizations are almost universally beneficial. Automated testing and continuous integration are a good idea regardless. Splitting work into small chunks and working on a cadence (takt time) can be beneficial locally even though globally you are still producing tons of WIP. And, so on and so forth...
Or, as I put is in a longer post. Agile practice have a tendency to expose all of the dysfunctions and waste. The analogy is that a deep slow moving river hides even the big rocks. Agile lowers the water level and starts exposing all the rocks, and the point of having short iterations and retrospectives is to recognize the rocks and remove them. Summarized as "Go slower to go faster." But, management decides not to invest in removing the rocks because the software has to be done on schedule, but reality is all the bombs that are going to cause the schedule to be missed are left behind.
met the original specifications
The key word here being "original" when discussing waterfall vs iterative development. Agile is not meant to deliver the original specification; it's meant to allow developers to adapt to a changing specification.
Unfortunately, it often requires the client to accept a product that was different from their original specification thanks to the dropping of features along the way.
Which is worse, not allowing a change in specifications because it puts the deadline in jeopardy or dropping some specifications to allow time for new specs that are more important to the customer to be implemented?
To me, meeting all the original specs is still failing if the end product can't be used by the customer.
Because I'm being reponsible to the needs of the project, I end up being unable to do the tasks (stories) assigned during the development period (sprint). If I was irresponsible I would do just my parts in the sprint and tell all the competing needs to bugger off.
So the stories spill from one sprint to another. There's nothing wrong with that. That's part of the methodology of Agile, and also points that the project manager is not properly managing the project - whether not having enough people on the project to handle the work required for the sprint durations, or overloading the sprints in hopes of getting work done faster than they should, or the people involved in the project are not accurately breaking down stories into suitable chunks.
Truth is like the sun. You can shut it out for a time, but it ain't goin' away. - Elvis Presley (source: imdb.com)
Ridiculous. Maybe in stupid-ass punch-the-monkey dev shops writing web-based crap no one will care about in 6 months. There are plenty of industries where this is nonsense.
In international finance, some country decides on a new law (usually a new regulation). The software therefore must change.
Is the law going to change again this year? No.
Is the development cycle less than a year? Yes.
Is the development cycle longer than a fucking sprint? Yes.
Waterfall will work just FINE.
I had a small project which despite meet the deadline has took more than double of expected work hours.
Did I know I have to evaluate and change underlying platform 3 times so it could work with other dependencies flawlessly? No.
Did I know I have to write part of DB driver because some thing that has been in DB for years isn't supported in its official driver? No.
Did I know I have to dig into source code of dependent libraries and fix their bugs and even change part of their architecture to meet performance requirement? No.
See? None of these involves requirement changing. How you can do waterfall in large projects worth millions is beyond my imagination...
I'd argue the opposite. As a consultant, when we're brought in for a 1.0 type project, we're more successful with an Agile approach because requirements are less solidified in detail but the high-level stories are fairly well known. Each sprint, we can take a story, detail it out and implement it. The customers see progress and enjoy the feedback loop.
Projects that are more focused on enhancements (smaller scale, not a "major upgrade") tend to have issues in an Agile fashion because everything needs to be done "right now" because it's "affecting such and such group" and the prioritized backlog turns into a big mess of conflicting priorities.
That being said, my main complaint about Agile is that the typical implementation of it is too short-sighted. Sure, Agile allows for refactoring phases, but if you are only focused on what's in the current sprint, you might make a sub-optimal decision in Sprint 2 for a feature that isn't even considered until Sprint 6......but the decision you made in Sprint 2 locked you in to a bad approach and refactoring will take almost a full Sprint.....had you known about the Sprint 6 feature, you could have implemented the Sprint 2 feature in a way that the Sprint 6 feature could be implemented in a matter of days.
"Because it sounds good?"
No. Because it is about individuals, not unfaced meatballs.
The problem is not with Agile, but with people who believe in magic potions.
Nothing in this world ever solves all problems. Nothing in this world ever fits everyone. Agile is no exception. It might be good or bad, depending on circumstances such as your team, your culture, your project and a dozen more.
The best you can do as a leader (manager, lead dev, CTO, whatever) is to pick and choose and come up with a system that works for your company, your people. It might be Agile, or Agile with something else mixed in or something else with some Agile mixed in, or no Agile at all. It depends.
If you believe you can take something that someone else cooked up without knowing your situation, and just apply it by the book and that's it, then you are not doing your job.
Assorted stuff I do sometimes: Lemuria.org