Slashdot Mirror
Ask Slashdot: Development Requirements Change But Deadlines Do Not?
cyclomedia writes "Over a number of years my company has managed to slowly shift from a free-for-all (pick a developer at random and get them to do what you want) to something resembling Agile development with weekly builds. But we still have to deal with constant incoming feature changes and requests that are expected to be included in this week's package. The upshot is that builds are usually late, not properly tested and developers get the flak when things go wrong. I suspect the answer is political, but how do we make things better? One idea I had was that every time a new request comes in — no matter how small — the build gets pushed back by 24 or even 48 hours. I'd love to hear your ideas or success stories. (Unfortunately, quitting is not an option)"
We have a bastardized combo of waterfall and agile here. I call it the Drunken Sailor approach.
People in cars cause accidents....accidents in cars cause people
Any additions must arrive 3 days before weekly build otherwise they come out the following week. That is a perfectly reasonable approach to keep things moving on time.
The sprint should start over. It encourages stakeholders to not interrupt the current sprint and to wait for the next sprint to shift priorities or introduce requirements.
For any change, have them sign a document and charge them.
New requests should not be integrated into current or even the build, but should be kicked into the long grass and added as time permits. Do not make agreements to have new features integrated into a particular release, just state that they gradually appear in time. Then there is no pressure and the quality of the builds will go up.
Tell them that you do 1-2 week sprints where you have a set amount of tasks. If they want new things added, they have to wait until the next sprint. Give them a login to your tracker site so they can view the progress/status. Have them come to sprint meetings as well so they have some input.
-SaNo
We use agile and determine our teams average velocity per 2 week iteration to be able to gauge how much work we can do during a 2 week period. The team commits to a certain amount of work for two weeks during a planning meeting. If new things need to come in after the planning meeting, something we were going to work on will need to go out to replace it. Otherwise, the new item goes in the backlog for consideration in the next iteration.
I learned the "say no" trick. Basically I present them with what I'm working on right now and say "this can't all happen, what would you like to take out". It doesn't make me super popular, but I've found most people appreciate honesty over disappointment.
But we still have to deal with constant incoming feature changes and requests that are expected to be included in this week's package.
The feature requests can come in at any time, but tell "them" that they will get prioritised and planned once per week and the important ones will get done in that time box. You will not change course between planning sessions.
After three or four weeks "they" will see that progress is quicker over all and the code is more stable.
Push back on your management. As a professional, it's your responsibility to do what you can to ensure the quality and timeliness of the end product. This is part of that responsibility.
Stick Men
For any change, have them sign something and charge extra for it as it was not included in the original design.
When someone requests new features you have two options:
- Tell them they have missed the feature/requirement freeze and will have to wait until next iteration.
- Tell them that, if they insist, it will delay the release.
Do not compromise the quality of the release.
$KaChing$
Quote the number of hours to implement a new feature or change request.
Adding an arbitrary fixed minimum for the changes and features is just going to piss them off and seem stupid, especially for trivial changes.
Quote them a realistic time, show them that quote when they want something rushed through and it deploys with serious issues.
It's a discipline, it's part of project management, it works. You can look it up.
I don't believe this answer will be well received on /. because it is usually practiced by project managers, and /. doesn't believe in project management.
In my experience, this is an indication of weak management, usually (but not exclusively) in the software development group. Rather than adjusting the build schedule you need to stop the feature creep. Once a development cycle starts the feature list should be locked until the next cycle starts.
Since you suggest that this occurs on a semi-regular basis, someone in the management chain either doesn't care about the slips or doesn't recognize what is actually happening because their subordinates are incompetent. No software manager enjoys being castigated for schedule slips, and good ones figure out what is wrong and address it. If you really want things to change, you'll probably need to figure out where the weak link is and work to get it removed.
Have a deadline for new features or bug fixes for that week.
Proper engineering (engineering is about how you build things, not about just crafting the thing itself) typically should look like this:
- Requirement deadline (Specify bug to fix or new feature)
- Software domain deadline (To do the above)
- Integration deadline. (Integrate above into larger project)
- Test deadline (verify bug fix or feature. Without getting fancy, must budget time for repairs)
- Customer delivery deadline.
The key is to work this stuff backwards.
Software moves a lot faster than hardware, but it does not move infinitely fast. Normal hardware works on the order of months for these things, working backwards from Job 1 to factory tooling to a half-dozen proveouts of HW and assembly line.
Software needs to recognize this and stop playing the infinitely fast "It should work." famous last words game.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
1. Before any work is done with respect to a given request it is first assigned to a developer.
2. The developer's first job is to estimate how long it will take to satisfy the request.
3. If the request is too vague for an estimate to be made then the developer conferences with the request's originator to get the information he needs.
4. Once a time estimate has been made, the developer communicates it to a project manager.
5. If the request can be accommodated without delaying the release then the project manager gives the go ahead for the work to begin.
6. If the request cannot be accommodated without delaying the release then the project manager conferences with its originator (and the originators of any other requests currently slated for the current release) to determine which will be dropped.
First, you can -- and probably should -- just accept that the deadlines don't mean anything. They self-evidently don't to anybody else, so why should they to you?
But if you must pretend that they mean something, then you've really only got three options:
1) adjust the deadline based upon how much actual work is involved with the new request;
2) factor into your initial estimate how much you think it'll take to do what you think they're likely to add on later;
3) or make new requests a separate project with their own life cycle.
This, of course, assumes that you're the one estimating time and setting deadlines. If somebody else is doing all that, forget about it. It's not your problem; it's the problem of whomever is setting deadlines. Either they need to be doing a better job at time / project / resource management, or they need to bring on enough additional developers to meet the demands, or they need to fire the incompetent hacks they've got working for them now who can't meet the demands of the job. Whatever the case may be, it's a management problem and nothing for a developer to worry about.
Cheers,
b&
All but God can prove this sentence true.
Force a delay somehow.
Requirements given for a new project? Send it back to analysis with some question that you know will force it to be reviewed. Knowing most BA's that should give you at least 2 days.
I hope no one thinks this is a new problem. We had it back in the early 1960's (yes, really, all of a half-century ago). I once was told to attend a conference which stretched on for three days trying to get agreement on how long after the last change order the users would have to wait for delivery. The closest we could get to agreement was that if the change orders never stop there will never be delivery, and only the developers agreed to that: all the managers would agree to was "It can't be that bad." I didn't go back after the first day; I had constructive things to do.
No amount of sane, productive, alternate solution propositions will get these assholes to treat you like a human being.
I have worked at my present company for 15 years. What you are describing is simply one aspect of what I consider normal development here. We use a modified waterfall. Requirements are gathered and estimates are produced for everything through testing and deployment. That's not requirements then estimates it's estimates then requirements usually and sometimes estimates after some of the requirements. The estimates are always called estimates and everybody seems to understand that they are very ballpark. Somehow, though, they end up in MS Project as deadlines and milestones with a heads may or may not roll consequence associated at the end. They keep paying me and I keep coming back, so I guess you can get used to anything.
Sounds like my company. The only thing people seem to understand is when I started being a complete dick, and when someone says I need this done by this date and it is unreasonable, simply say no, too bad, you are the fucking moron sales guy that promised it on a whim, so no go back and tell your customer No. I get lucky and got support from my CIO and CEO, instead of getting fired. 99 out of 100 developers have an excruciating time saying No to people, so upper management gets used to the answer always being Yes. Not until you get a development leader with some balls that can say no will your situation improve.
What I always did with change requests: The rule was, you must get an estimate and approval for ALL changes. The estimate would include how long it would take -- and I was real good at estimating that. The approval would explicitly be for the additional time required for the change -- meaning how much that change would push back the schedule. Most "urgent changes" became "oh, never mind". Any that survived and got approved automatically adjusted the budget and schedule to accommodate the change - so I remained on schedule and on budget.
If you are using Agile with a combination of Scrum (like we do), then every task is roughly estimated for the size of work required. In each sprint, you can only accomplish so much work. Over time you determine your teams "velocity" (the estimated size of work you can do in a sprint).
Then, you have a person who plays the role of Scrum master. His or her job is to "protect the sprint". Meaning they help keep new issues from entering the queue during the sprint. When an actual emergency or rush item comes up, the Scrum master (or lead, whomever) asks, "what is OK to drop from the sprint if we can't get both done?". Some places take turns being the Scrum master, so it need not be a set role.
The Scrum master has to be willing to be that gentle jerk, and say things like, "not now, but we can work on that in the next sprint".
If you keep missing deadlines and find yourself in overtime crunches, the problem is with the way you estimate costs. Lots of shops assume that you're supposed to start working 60 hour weeks and running around flailing your arms right before every release. If that's your pattern, figure out why things are so delayed and factor that in. If management doesn't accept your estimates and they turn out to be closer to reality, there's not much more you can do.
Any changes and feature additions obviously have to be included in the estimate.
I swear to God...I swear to God! That is NOT how you treat your human!
Ultimately someone needs to be responsible for what gets into a given release. And given a fixed pool of developers, if something new comes in then something else likely needs to get pushed out.
You can't do the impossible, and no techniques will allow you to do infinite work in a given period of time. This can be a permanent push back (never going to do it) or a temporary one (we'll discuss it at the next planning meeting).
If they won't be pushed back, stop caring and dust off the resume. Don't work for people who aren't willing to compromise.
I still have more fans than freaks. WTF is wrong with you people?
The problem here is management not wanting to take responsibility for their changes. The only way I know of to fix this is to force the issue. You need a process in place that requires sign-offs for schedule changes due to new requests. Then every time these requests come in you prepare a time estimate and send out the new request and current work-in-progress with a request for management and business to assign a priority to the new work so you can determine which existing work will be impacted and can get sign-offs for those delays. Then make it clear to the people requesting the new work that it will not go into the schedule until it's been prioritized and management and business agree to any impact. The selling point to management is that this is to insure that once work has been promised by a given deadline you won't miss the deadline because of new projects without management and business knowing about it and agreeing on which projects are more important. This goes against Agile in that the whole point of the process is to prevent the development team from adjusting to new requests as they come in. Eventually you will, but you're adding "stiffness" and delay to the process to deliberately act as a stumbling block to new work and force management and business to get together first so when they come to development for an estimate they've already agreed on priorities and development can proceed to revise the schedule without anyone being surprised at delays.
When you are in an environment where they will not stick with the commited stories during a sprint Kanban is a good way to go. They can add anything they want whenever they want. But all they can control is what is the next item worked on, not when it is due.
If you change one, you can only keep one of the others fixed. This is an immutable law of any sort of work.
Where I work, we have an agile process, but we're rigid about one thing...sprint plans don't change. Once a sprint plan is finalized and developers have accepted it, managers have two options...blow up the sprint and create a new plan (with a new deadline) or wait until the next sprint. The former option is supposed to be an extreme case and all checkins for the sprint, whether complete or not, are reverted to the previous sprint state. This allows management the flexibility to not wait in emergencies (i.e. we signed a multi-million-dollar partnership with XYZ but their shrink-wrapped software releases two weeks from now and we need our integration by next week) and yet provides enough of a penalty that they don't do it very often.
"Don't blame me, I voted for Kodos!"
The whole point of a sprint is that once it starts, it doesn't change. This protects developers from your exact problem - managers screwing everything up. If it's not possible to do weekly sprints (build/release friday morning, plan friday afternoon, start work on monday, or some variant thereof), that are static once the storiesare decided on, you need to move to a "feature" model, where you can build/deploy whenever wherever and your "feature teams" just pick up the next feature in the list, which is prioritized by management. This way builds / deploys can never be late, because they happen all the time on a per-feature basis. If management decides a new feature / fix takes priority over whatever the feature teams are working on at the time, they can either wait a day for their request or they can pull a feature team immediately and start them on the new story.
Alrighty, let me put my onion on my belt, my teeth in, and give you whipper snappers a history lesson:
That's how it has always been. We have never been given enough time. Specs change and when you miss the deadline, it's your fault. Period, no excuses - there's ALWAYS something you could have done to meet it - that's what my manager told me at NCR.
And ...the point is?
Software development and IT (same fucking thing in my day) have been like this. You have to work 10 -12 hours a day or more, at least 5 days a week, and you dealt with it. Because back then, we were paid 80-90 thousand dollars a year in my area - and now, you're paid 80-90 thousand dollars a year in my area - after 20 years of inflation, btw. Just saying - with inflation, developer salaries have been going DOWN.
And after you spent your 10 -12 hours a day coding or engineering software, you went home reading a goddamn O'Reilly animal book on the latest and greatest "technology" and tried to figure out how to work it into your daily job because the next time you looked for a job, that technology that came out will be required along with 2+ years of experience in that technology. Things change.
Stopping now because it's banana pudding night, the TV room opened, and there's Matlock Marathon on!
MATLOCK!
Simple really, add both extra pay and extra time when new requests come in. The amount of extra pay and time are to be determined by the developer on a sliding scale. Developers don't need to adjust to unreasonable demands from clients, clients need to adjust to reality.
And retire to a Mexican beach resort
The first issue is bad management. The second is lazy developers. Development should unit test their code before check in. If the project is 'that big', then there should be a server to run 'test builds' prior to actual check in.
Yes, I've had to fight these issues at multiple companies.
Have your 'clients' test any released builds in a non-production environment and sign off indicating their acceptance.
If they want the code fast - they can accelerate their testing process.
The problem is that you've got a deployment schedule. It implies deployments are scary, complex things. If you do things right, deployments are trivial, zero downtime events. Extra feature? Extra deployment. No biggie. If you stop deploying on a schedule, you'll also stop wasting time waiting for friday (or thursday, if that's your regular deployment day of the week). Of course, part of any feature being deployed is peer review, proof of testing etc.
Also, what other people said. If your employer wants to be professional, call them on it. Sneaking in last-minute features is not professional. Request a meeting, raise the issue.
Product Manager here. And I'm the guy who just can't help himself to add in that one more feature that seems so obvious now but was somehow hiding in my blind spot previously. Oh, and we're gonna make a TON of money if you can just implement this.
Step one: accept that life means constant change and these requests are always going to happen, like it or not. Nature of the beast but you can moderate this.
Step two: find a way to get your arms around it. A formula of: one feature = X days slip on delivery date is not sustainable since all X's aren't created equal. This will ultimately backfire when features take a long time to implement because expectations have been incorrectly established. You need to have a code freeze date for each build and stick to it. Managing code branches and merging will be key.
Step three: Make sure your product manager has solid use cases. Features wrapped in a story tend to stick together. If the feature doesn't play well into an already defined use case (story) then it is likely superfluous to the main goal of the product and can wait. If the PM needs to change the use case to accommodate the new feature then the PM needs to get his or her act together (while understanding that PMs are human and can sometimes make mistakes, but this should not be a standard operating procedure, changing fundamental use case scenarios). Sales organizations are typically coin operated so they'll always ask for just this one feature to make the big sale. It's a lie. If they didn't need it last month then they can wait another month.
In my opinion, this in not something a developer should have to be concerned with, this is a product management issue. What is probable in these situations is that the PM is not including all stake holders in the requirements doc. All stake holders need to understand to some degree the end user's mental model (assumptions, motivations, goals, etc.) and if so a lot of these things will get vetted during the review process. But Sales, that darn Sales team... can't ever keep them happy; can't run a business without the revenue the bring in. They will always try but only the lesser ones will need said feature NOW to make the sale.
I'm not a developer, so take what I say with a grain of salt, but I *do* manage technical projects on a regular basis. I try to stick to the rule that deadlines are dependent on requirements. If you ask for something to be added to the project, then the deadline (and budget) must be reviewed and altered to account for the changes. It only makes sense.
But if you're trying to make a regular release schedule, then I'd suggest that you basically stop accepting new requests for each release some number of days ahead of time.
The key sentence in your post is, "But we still have to deal with constant incoming feature changes and requests that are expected to be included in this week's package." Change that expectation. Maybe tell people that all requested changes for Friday's release must be submitted at least a week in advance, and then set the task on Monday morning to review those requests and set them on a realistic schedule.
Of course, I might be talking out of my ass because I don't program and I don't really understand what Agile development is.
After you miss your next deadline and/or push out a bug riddled release, task the initiative and get your group and sponsors together to formally find out why deadlines are missed.
As discussed by commentators above, the reasons are definitely a lack of a change management process (and possibly a lack of a clear scope / requirements definition). But somehow people are more receptive to the obvious when it comes from a discovery process, rather than being told.
After you set up a change management process figure out a way to get an estimate of how much the impact will be. So the next time some change comes up, tell them how much the release will be delayed up. front and ask them if they would want it for the next release.
Remember, the more time you spend on management, the less time you have to develop, so factor that into your schedule as well.
Keep a written record each time something unreasonable is requested.
After a few months (6?) show the documentation to the manager of whomever is making the requests.
Then crack open a beer and wait for your new middle manager to arrive.
Political or not, software project or not, someone in the management chain isn't doing their fucking job and you should not simply accept that.
I'm a 2000 man.
"All of our development bandwidth for this sprint is committed. Which item would you like to delay to make room for this one?"
In the spirit of my title, the second sentence is, of course, the important one.
Do agile proper. Choose a method. stick to it. Not a bit waterfall and a bit agile. then basically you are doing waterfall with some new term, but not new procedures.
Eg. with Scrum at the end of a sprint, the product is Done, or it is taken out. The development team decides how much work to take on. Since the development process is transparant business can guess that extra criteria will add load.
Business decides what are the priorities. Development determines how much can be done in a timebox and how they engineer it.
If there are more requirement that take extra time, then those requirements are taken to the next sprint. If there are delays, then those delays are the time of a full sprint, (3-4 weeks). And realize that 80% of perfect often is enough.
Things like "not an option" "business decides". "too costly"... are all in the big excuses book.
To be perfectly honest, you as a developer probably shouldn't be defining timelines. That's what management is for. If management is failing at establishing stable timelines, call them out. It is their job to redefine the release process when it is needed, not you.
And don't keep the quitting option off the table. Typically the only time I've seen management change in a majorly positive fashion is when they have to deal with a mass exodus of developers.
Far too many companies partially adopt Agile development practices. They integrate the rapid iteration, sprints, iteration planning, and standups but avoid the real work which is writing unit tests, and integration testing. It's impossible to work at break neck speeds while maintaining the integrity of the build without unit testing and integration testing even if you have embeded testers on hand to test things for you and most non-programmers don't really seem to understand this concept.
Developers who are being introduced to Agile development practices most likely need some basic training beyond "how to use the iteration planning software" which addresses the concept of Agile development and the importance of writing good unit tests to validate that the code that was written is still working as expected after developer C, B and A have checked in code they've modified which may break someone else's features.
The whole point of Agile development is to essentially be able to jump from one thing to another, but that doesn't mean things shouldn't be planned out. If a customer comes in asking for changes that are not planned for this sprint it should get added to a backlog, prioritized and then added to the next sprint or delayed even further into the future.
So essentially what I'm suggesting is to first put in place development practices that will help maintain a working build. And secondly, you need to manage the change requests in the same way any service industry does. A request is made, it is assigned a priority. At an iteration planning session that task is either assigned for this weeks worth of work or pushed into a backlog where it will eventually be assigned. Any work not assigned for this weeks sprint which would include all feature change requests that come in after the iteration planning meeting will not be addressed until the next iteration planning session.
Granted this is all ideal talk, because most of the time the players involved don't really want to make a commitment to the Agile development process.
If you can't get buy in on integration testing and unit testing/training for the developers, and sprint planning go back to waterfall development practices. Otherwise you will wind up with a bunch of very stressed, unhappy programmers who are rapidly iterating themselves and the code base into a tangled constantly broken mess.
Step 1: Create expectations for contractors and then change your mind to something else. Step 2: ???? Step 3: Profit...oh wait, lawsuit, anti-profit.
Okay, let's assume this since you say you're doing agile:
If they want to put their new story in the current sprint then I've used the rule you have to take double the number of points worth of story out. So if they want to add a three pointer, then they'd have to take out at least six points worth of promised stories. The double points is to account for the fact that anything at the last minute is hard. This *also* has to come from work that hasn't been done yet - so say you've got a sprint with a six pointer in it that is two thirds done (going by your daily burndown) they can only get two points back by saying that this doesn't have to be delivered.
Fix the economy so that bosses can't get away with fucking over their workers because there's nowhere better to go.
Your bosses know damn well they are forcing you to make bricks without straw here. Don't overestimate how much power you have over this situation.
Since you say quitting isn't an option it appears you are captive to the situation and are going to have to put up with it. Still, now would be a good time to build references and find another employer.
Rummaging through the thread I see a few good pieces of advice. One, somebody in the food chain needs to grow a set of balls and learn how to say no. Yeah, yeah, yeah, the-customer-is-always-right, if-I-piss-the-customer-off-we-may-never-get-any-more-business BS. It's BS. Grow a set! Second, even with a set and some compromise you will still need to start setting REAL (as in hard) deadlines for feature requests with regard to release dates. Project management 101, actually, and even the current methodologies assume there are still dates for changes. If your boss(es) don't know this already from experience before they became management or while managing elsewhere, they are incompetent and need to be rooted out. Not easy, mind you. Third, keep looking for other opportunities. Even if change starts you may not want to wait for the positive effects, as you may not retain your sanity/joie de vive long enough to reap them where you are.
Don't push back the deadline.
A new requirement / feature is given a priority and added to the product backlog.
It's not added to the sprint backlog.
I'm sure the customer can wait one week longer for a proper release with the new functionality.
If the feature request is so important that it ABSOLUTELY has to be in THIS release, restart the sprint from the beginning.
But that should be an exception, since it disrupts the production cycle.
Of course you explain these procedures with the customer and make sure he knows why it is important to stick to the production cycle (quality, productivity).
Also work on you Definition of Done.
Make sure you put "all unit tests passed" on that.
Privacy is terrorism.
We're not doing agile or scrum or anything - honestly, we have a development process that's like waterfall but with no falling. Nobody's charted out which parts are dependent on any other part.
But even with the complete lack of project management, scope creep is still a bigger problem.
The initial specs for the project ("R2" of a project we did earlier) started with about half a dozen items on the scope list. When the contract was signed (our company is technically contract working for another company), it had expanded to about ten.
Now it's around thirty, maybe forty depending on how you count things. We're about a month from the due date (we started in March), and we're horribly behind. Some of the things still don't even have specs. I'm trying to get them to trim scope - we've cut half a dozen things just this week, after I blew up on a phone call with the person in charge of managing the project (a combined VP/marketing lead/programmer with commit access, the most dangerous combination I've ever seen as he will sell a customer something, code it in a sleepless weekend, then expect us to help him when it breaks or needs more functionality).
Your project manager and sprint master are not doing their jobs. Perhaps you should apply for the position.
If feature requests are still being added to the iteration, then development needs to immediately cease and the sprint needs backlogged.
>(Unfortunately, quitting is not an option)
Unless you're an indentured servant or your family is being held at gunpoint, then quitting is always an option.
"A lack of planning on your part does not constitute an emergency on our part."
Another option: Use the power of bureaucracy to your advantage. For example, create a fairly confusing Mid-Sprint Change Request Form that needs to be signed off by 2-3 people that are never in the office.
A third option: Make sure that the work that was requested properly gets released on time, while the work that was requested mid-sprint will get released when it's ready (which, if you're doing things right, is always later than on time).
The idea is to use the carrot of on-time quality delivery plus the stick of annoying bureaucracy and late delivery to push the people making requests towards doing the right thing.
I am officially gone from
My wife is an attorney. When she's working on a case for a client, there isn't a lot of face time if she able to work out what needs to be done for the client in the initial meetings. Often she'll get clients who want to change or add things mid-stream. Fine... If you want to make a request after I've started working on your case, it's $150-$300/hr for me just to listen about what you want different. If I agree to do the work, it's another $150-$300/hr beyond the initial estimate of how much the total was going to be. And if it's more than a few hours I'll have to charge an additional retainer. This has cut down a lot on clients who bug for too many trivial reasons and don't realize it's just delaying things.
You don't get deadlines until you get requirements. Once you have them you give them a quote. "If you want all of this it will be done by February 1st" if they complain, tell them what they'll have to cut out to meet their deadline. Once this processes is done EVERY CHANGE TO THE REQUIREMENTS COMES WITH A CHANGE IN THE DEADLINE. No matter how small it is, they send you a change, you send back your revised estimate. You keep track of all of these changes. When the projects complete you do a post-completion review and explain why you missed your original date, siting line by line all of their changes.
This is how it should always happen. The only real catch is you get into arguments with them about "what is a change?" and "We really meant this, not that" So it's important to have the requirements really nailed down. You can take short 1 day classes on these sorts of things, how to word stuff. You want to avoid requirements like "Make the billing application faster" Well... what does "Faster" mean? does 1% faster count? Do they mean LOAD faster? Or that the reports return data faster? You need to be very specific because they're rarely satisfied. Also, define what the billing application is... Very specifically. You really have no idea what they are doing on the business side and sometimes you can get all done and find out they are doing 90% of their work in a spreadsheet and then dumping it into your application. You could likely meet all your requirements just by doing away with the 10yr old lotus notes sheet some dead guy wrote back in the day.
You think you're fighting manager's lack of understanding of software development. You are wrong.
You are fighting politically savvy people who have found a way to blame you for their problems. They don't want you to solve the problem and will actively work to prevent you from solving the problem, because then you can't be the scapegoat.
If you don't have a VP or C-level manager who will fight this fight for you, then you've already lost. Don't bang your head against the wall. Play the same game as everyone else and find someone else that you can use as a scapegoat. Meanwhile, start looking for a new job.
Even if you miraculously "fix" this problem, someone else is going to claim credit and you're going to get nothing.
Agile is all about adaptive development. This includes management, not just the developers. Part of adjusting to new requests is to prioritize them relative to existing commitments...by getting management to prioritize you're just bringing them into the Agile process.
Get a manager. This is the appropriate role for a manager, to stand as the gate keeper between the development team and the incoming barrage of requests. It doesn't matter what process you use if the manager is able to be an effective buffer. Ie, a new request comes in and the manager estimates how long it will take and then tells the requester either that the release will be delayed or that the feature will go into a subsequent release.
If the manager can not manage this process, then it will not matter at all what process is used because it won't fix the problem. Of course you could always be unlucky and be stuck with a bad manager, one who always sides with the requesters and is actively working against the developers, but no process is going to fix that problem either. Processes can and WILL be ignored. In fact, processes can often hide the problem of having a bad manager because nothing covers an ass better than a process.
I worked for a number of years as part of a small team of web application developers for an educational grant. When I came on, change requests came in at random without respect to deadlines, much like your premise. A couple of years ago I finally convinced our sole client that a semi-annual development cycle would not only get them a better product, it would also improve morale.
They reduced the funding a year later. I am currently unemployed.
Today: new requirement, new story, a new task goes in the the backlog.
Tomorrow's scrum: something gets put back in the backlog and the new task gets assigned.
TANSTAAFL.
And show a little backbone – push back. Don't be a door mat.
If a restaurant customer changes their mind while the chef is cooking their choice of meal, or maybe forget to request "no mushrooms" until 20 minutes after ordering, they may get the new dish, but they won't get it on time, and reasonable people understand that. Of course there are always unreasonable customers. Management reserves the right to not serve them.
You have to learn The Power of No.
You are not to be a jerk. This is important! You do not say No to everything. Instead you give your client (or management) a reasonable choice. They can:
1). Have the new feature, with a push on the delivery date;
2). Have the delivery date and take a pass on the feature.
If they go the Sparkle Pony route and ask for the new feature plus the original delivery date, then your answer is No. And you have to stick to that. Don't waver, don't even attempt to deliver that. Never say or imply that both factors are possible together. You are the mature one and simultaneous delivery is like violating the software development Pauli Exclusion Principle. It cannot be done.
In our architectural practice, we use the term "additional services" to quantify scope creep, basically anything beyond the scope defined in our proposal/contract.
This reinforces the need for making that initial statement of expectations clear AND the implications for any deviation thereafter.
There is no need to use a SlashDot sig for SEO...
What you have is a common issue - management doesn't want to change, so they make the developers learn Agile (or the next great buzzword), but never change their ways.
As others here have said, if they want Agile, make them be Agile - no changes to the content of the current sprint - it either goes as-is, or is aborted and a new sprint planning session starts. Enforce the need for a full user story for any new feature, whcih means a full meeting to define it, then the meeting to estimate it.
"Unfortunately, quitting is not an option."
Quit. Start your own company. Do your own development. Learn about the real world so you will better understand the full scope of the problem and you'll have complete control. Then if you don't perform you can fire yourself and go back begging for your old job with a new understanding (but outdated skills).
I just let them add it to the schedule and tell them we will charge every hour and get paid overtime aft 48 hours. Amazing how things get prioritized better by my management when it COSTS them more for items they want right away. You can have 2 of the three COST, QUICKER, BETTER QUALITY (LESS Technical Debt)
Pivotal Tracker and similar tools do an excellent job of addressing this kind of cognitive dissonance. "I need you to do this ASAP. Priority ZERO!!1!!" "Adding scope? No problem! We'll estimate the new task. There's the queue of user stories. Prioritize however you'd like." "Two older stories just got pushed out of this iteration." "Yes... Yes, they did. Any questions?"
"Agile" is something of a misnomer... it's about committing to the work items you've estimated into your current sprint -- and no more. If someone wants to add a feature or request, it goes straight into the backlog for consideration during the next sprint planning session.
"Agile" is more about setting up a consistent delivery schedule... the build train leaves the same time each week, carrying whatever passed QA testing... and no more. The build train is never delayed, only derailed by an Act of God. That's right, if some exec really thinks that something is so important that it needs to be done *right now*, you completely stop all work, scrap the current sprint and start a new sprint planning session with all of the overhead that entails.
Anyone who practices differently is not practicing Agile according to the way it was intended. There are no "sprint schedule extensions" in Agile, since it's a measurement and estimation tool... the same way you don't measure with a longer "yardstick" when something is too big to fit in a 1-yard container.
The reality is working at Facebook has got to suck. No pity for you.
If these requests are made in person or at least by someone in the office, you should have a task board with your current work on it. When someone comes in with a request, say "I'd love to, but this is our current feature list, and my job is to work on those tasks. You're welcome to go argue with the people who gave me these features which one should be removed for your new request. Otherwise, talk to the product owner and get it added to the backlog and prioritized appropriately."
If these are coming in from external sources, it's the same deflection, but harder without the physical board unless they can see a virtual task board / backlog somewhere. "Talk to the product owner to get those added to our backlog."
If they are coming from your product owner / manager, beat that person with an Agile Methodology book until they understand that features aren't added mid-build-cycle without at the very least pulling an equal-difficulty feature off the current feature list.
~Anguirel (lit. Living Star-Iron)
QA: The art of telling someone that their baby is ugly without getting punched.
...eventually. First try to have management implement changes to the process that will address the issues that you are having. If they don't, GTFO!
Seriously. It's not worth working under constant conditions like that for your career.
The problem is: ....
a. management
b. stakeholders
c. scrum master
d. customers
e. hardware
f. testers
g. co-workers
but never developers, i.e. you and your immediate sprint team
Agile is not perfect. You balance what goes to the next sprint and be done with it. Crap happens, it's what happens in the real world.
How is quitting not an option? Incriminating photo's? Doorknob too complicated?
Get an iteration manager with a backbone.
If a story is injected into an iteration that has already been planned, then something of equal size must come out. That's the product owners responsibility.
Its on your iteration manager to PUSH BACK.
If they don't push back, raise it in your retrospective... no retro? then your team has no feedback loop and you're hosed.
Move over to Management or Marketing. You can thank me later from the deck of your trophy yacht.
putting the 'B' in LGBTQ+
The short answer is that you can't. Your boss, if he / she is a programmer, will go to bat for you, and say "this won't happen; deal with it." If they aren't, you're screwed.
See, in the business world, much to its caricature, there are people who think they are business-savy. They watch 'The Apprentice' with a notepad in hand, and think that when it comes time to handling outside work, it's all about how fiercely you negotiate. Your non-programmer boss, who got his start in sales / marketing, is used to promising people stuff that others need to deliver on...as well as combing over any problems when a 'whoopsie' happens (missed deadline, etc.); he is also used to the idea of pandering to the client, and doesn't understand the intricacies of telling the client, in non-subtle, but non-insulting language, that something simply cannot happen.
So, when your client comes to negotiate with your boss, he's going to give them everything for nothing; he doesn't know this, but he does it. He's going to ask for time estimates from a programmer, where things operate in a completely different kind of world (every project is a new set of problems, first rule; ergo, all time estimates are vague and unreliable...even for 'easy' projects, because of some stuff I will touch on later); he's going to take these time estimates, and shave them down...asking the programmer, "Can't we try to get this done by Tuesday? And we can fall back to Friday if it doesn't work out." The programmer, of course, will tell him the truth (the programming / mathematical truth), which is "Sure, we can try to get it done faster." But in reality, it's not a magic button that gets pressed to make things 'go faster.' So, your boss tells the client his truth, which is that the project will possibly be done by Tuesday. The client, hearing this, thinks that it might be done by Monday, but will begin annoying your boss via phone calls as of Tuesday.
Now, let's take a moment to look closely at some of the elements around this scenario: your boss is going to charge the client for a certain amount ($2K), based off of your low wage, long hours, and another project that will be coming up a few days later for another client (it's all about volume). The actual cost of the project is $3K, but after your boss is worked down in negotiations ("We need to keep this client / build a relationship. We'll make it up to you with more work down the line / another project from them that will be worth more at some point in the future."), it'll be $2K. Bear in mind that the Tuesday deadline is actually negotiated by this client as well...so from their viewpoint, they've gotten a pretty sweet deal according to Apprentice 101: by dominating your boss, they got him to place their project at the top of the 'critical priority' pile...and they saved themselves $1K.
Your boss, believing the lies of his industry, thinks he's building a relationship with the client...he's not, since the client will bounce as soon as he tries increasing the costs anywhere near market rate, and they know that they can tweak him at will to speed things up / shave costs because he's already done it once before. Meanwhile, you, the programmer, are doing $7K worth of work, and enjoying near constant panic attacks because -> the client submits development requirement changes piecemeal, via email, telephone, SMS, Skype, and toilet paper. Your boss, of course, will come to you, and ask you if you can just do these extra tasks...that they won't take too much extra time, right? Of course not...changing the backend from SQL to NoSQL, and the frontend from ASP.NET to PHP shouldn't take any extra time at all...you're a programmer...you're second-kin to a magic elf...you can just not sleep, and reach into your magic bag of tricks, and pull off this thing by Tuesday's lunch. And skilled salesman that your boss is, he's either giving the changes away to the client for free, or taking on an absurdly low number for the additional work ("It'll pay for itself in the long run, you'll see!").
So, Monday
I am John Hurt.
As has already been mentioned, this is an age-old problem. One of my co-workers many years ago addressed it thusly (after long and bitter discussion): "Okay, you demand that I finish it quickly? All right, I give up; it's finished. Now, when do you want to discuss fixing the bugs?" Never, ever, EVER let anyone talk you into a time commitment you don't think you can meet. If you do, you'll get fired just as surely -- and with much better cause -- as because you won't knuckle under to unreasonable demands.
Don't worry so much about deadlines, they make a whooshing sound as they go by, that's all. Focus on making a great product, if its good and sells nobody will care how long it took or how much it cost, they will only ask who made it.
You need a full service, deep Kanban implementation to evolve your process into a flow based one. You can have separate classes of service, with separate prices, for new development, and support and operations tickets with different urgencies. Once upstream stakeholders are clear on team capacity, they can negotiate amongst themselves about which items should be done next, which can be done later, and which are probably not going to be done at all. I think with a good system design, concrete measurements of throughput and lead time, paying attention to the way work flows through the system and full transparency you can at least make upstream stakeholders aware that they cannot have it all.
Proceses, and more processes.
It is tue magic world that protects you from being called a non tem player, and keep things working as you want them.
Just get a business process in place that does allow to submit change request only for the next week, no exeptions. Any change of course, bug fixes and implementation errors should still be fixed as they come in.
And the secon point is to discurage the people to submit insignificant changes and trash to your department. How to do that? I found one metod being very efective: mandatory testing.
The person who requested the feature or change must mandatory participate on its testing before it goes live. Usualy this happens on weekends in my company.
So, you are nit ready to test it this weekend, then fine, this is not included in this weeks release.
You get less requests and all will be more relevant, and you get people for testing it.
...generally placing one of them on top of the conference table before code or deadlines review should do the trick...
I like that yardstick analogy, mind if I use it?
Over the past few months, my org has moved from waterfall to scrum, and while the transition is painful for some people, I think it's really simple to grasp. We no longer have the 3-day meetings of adds/cuts to figure out what features we think we'll be able to ship in the next 2 years, we just have an idea of the bigger items we want to accomplish. Things that we need to do sooner rather than later are more detailed, such that we know whatever work we think we'll be doing over the next 2 sprints is very clear and detailed, with increasing fuzziness the further out we go.
We only work a sprint at a time (2 week schedule), make sure everything is as tested as we can before calling it complete. If we find gaps in our coverage before the sprint is complete, we fix them then and there (no checking in things with "known issues"). If we find gaps after the sprint was completed, we add them into our backlog for handling in the future, so we don't context switch out of our current work to service code. It's actually quite a blessing.
We're not a well-oiled machine yet (will take a while), but I think people are getting it more and more.
I don't think agile is necessarily the right tool for everyone or everything (I know some agile purists would disagree), but for the way we were doing waterfall before and all of the problems we felt year after year, I feel agile development is exactly what our org needed.
where features are on. the left of the whiteboard and the ungoing activity to the right (in progress, done, blocked) then tell the manager he can switch out or reorder things on the left but he has no power of ongoing activities. realisticly to meet deadline old task of equal size and complexity should be replaced by new ones. kanban is ideal in a support organization with fluid deadlines
You're describing scrum and variations. There are other, less stiff, ways of running agile. Please don't define agile as sprints, etc.
As more developers I talk to, I see a lot telling the scrum was of doing things don't work as good as expected, a lot are already going to kanban or just revert back to the old way..
In the end a lot of us already were developing 'Agile' but it just didn't have a name..
You should do what works for you and your team, and not try to force a specific method onto a team just because it's hip.. Just use parts of the 'Agile' methods that work for you and your team. There is no definitive or best way to do development, just do what works best for you..
Soulskill, after over 25 years in this business, I find the situation you describe so very, very common. Management is like little babies, all they know is they want and they want and have little understand of what it takes to get what they want. I am soooo sick of it all. I go into programmer because I like it but the brain-dead, souldestroying zomies have been it a an unreal nightmare.
If the story is agreed upon then it will be built that way, and I can hardly imagine cases where it is acutally business critical if a requirements change is implemented this week and throwing everyone out of the loop or next week where it can be correctly scheduled.
Thank you. Best, most concise description of Scrum ever.
"Agile" is more about setting up a consistent delivery schedule... the build train leaves the same time each week, carrying whatever passed QA testing... and no more. The build train is never delayed, only derailed by an Act of God. That's right, if some exec really thinks that something is so important that it needs to be done *right now*, you completely stop all work, scrap the current sprint and start a new sprint planning session with all of the overhead that entails.
I'm currently working at my first gig where they're actually doing Agile properly, but even here, it has happened that something needed to be included in a current sprint. And we have accepted it (if it was small enough) but dropped other stuff out of the sprint. And even then, it always cost problems. Putting it on the backlog is nearly always the best approach. Disrupting a current sprint is always messy.
There are still a lot of things we're struggling with, but even so, it's the best work environment I've ever been in. I'm convinced now: I want more Scrum.
It's like you say, mostly political. Until you have understanding at the top of the food chain (VP of Marketing, a CIO, or the CEO themselves), you're not going to have understanding on how development really works. It's coming, but right now very few upper management types can properly scope the time it takes to work on IT features. If you're unfortunate enough to be working in a purely political situation, you'll never get the timing right because you're a political 'asset' rather than a proper development center.
You should have a system in place that will allow you to prioritize the features, something simple like 'high, mediun & low' will suffice. The system should allow a maximum number of high requests to be put in place (otherwise, all requests will be high). the idea being that high requests are critical for the next release and low being nice-to-have-but-not-important-at-all.
Then additional to this, somebody reviews these requests and then puts a implementation timeframe to it. You then have something on paper to show that to implement all high requirements you will need at least X amount of time. If this exceeds to deadline, either they can drop it (reducing the priority) or extend the deadline.
"Get the flak" -- in other words, someone (the business line) expresses anger.
Anger happens when people have expectations, and these expectations go unmet.
The answer is: Lower Your Expectations. (Management, I am looking at you.).
I'm guessing you can't actually say this to management, without coming off as sarcastic. But I am quite serious, this is basic psychology. If my expectations are lower, then my unhappiness (when things go wrong) is also lower.
It's entirely possible that your organization cannot improve processes. Maybe what the company is doing right now is the best it can do. Let's not get all "Rah-Rah, We Can Do It" -- maybe you can't do it, maybe the effort is a waste of time. After all, the people before you were not idiots, yet look where things are today.
If you can't fix the problem, you can at least minimize your unhappiness, and help your colleagues do the same.
-kgj
...this is a common problem. You start the sprint, you know what to do, and something urgent happens and you have to prioritize that.
Which breaks "agile", just in itself, since you no longer focus on your work packets.
The method that I've seen that seems to work the best - still under experimentation - would go something like this:
* make two branches, one for fixes and one for development
* allocate enough time to do the normal work packets in the sprint to allow for extra fixes
* do work packets in the development branch
* do fixes in the fixes branch
* on a successful test of the development branch, make a new fixes branch from the release build
* make work packets the following week to move the fixes into the development branch
* if too much time is spent on fixes, drop complete work packages - that's why they are isolated packages anyway
That way, you can get fixes out quick, without risking bugs from the normal development. You can also keep your normal development flow.
You will risk never being able to do a release from the development tree. This is usually solved by having a sprint focused on catching and finishing a new non-fix release.
Depending on your needs you might want to do more than one fixes-branch. For instance if customer A needs one emergency fix, and customer B needs a completely different one - you might not want to mix those two up and accidentally causing more bugs.
You risk just working 100% on fixes on the release branch. Then you have to rethink your priorities... again.
Mod parent up.
Commit to your short-range deliverables. Restate your commitments daily, keep your team informed.
-kgj
Yes!
You communicate with anyone by speaking to them in their own language. Maybe they understand your language as well, all good and fine. But the best way to reach someone is to speak his language.
Money people speak money. If you want to get through to them, speak money.
For that matter, if you are a developer who doesn't speak money, you might want to learn -- it's a useful skill with a variety of benefits.
-kgj
Yes! I also learned the "say no" trick, and use it to good effect.
-kgj
I work for a large multinational (well, for a subsidiary. Parent company is massive. Global subsidiary is quite large. We're a regional offshoot).
We get a fair amount of our deadlines set by head office, with a "We've put out a press release saying it'll be out on this date". You can't say no, it won't work. This sort of thing isn't restricted to big companies. In smaller companies I've had bosses tell me (and this pre-dates Agile as IT design tool) that I have to have the code finished before the end of the week, as they've got an advert in Saturday's paper.
Like in Mythbusters, failure is always an option.
... and today's pet project has
While my shop implements Agile rather well--we generally work on 2 or 3 week sprints (we are developing a brand new application and a more rapid release cycle presently isn't particularly valuable). Under committing can help when critical features show up for the current release. An alternative to pushing a release back because of features injected at the last moment is committing to less than an entire release's work and adding additional stories when possible.
Personally, in your situation I would probably recommend going full tilt into the formal Agile process. Strictly speaking, the formalized definition of the "Agile process" does not necessarily coincide with what most people think of as "agile". In an "agile" world, maybe you can make changes at a whim, simply because the customer says jump... but in the "Agile process", you don't really make changes to the current sprint; all changes get dropped into the product backlog, and prioritized appropriately to be included in a future sprint. If you're changing the scope of tasks included in the current sprint, then you're not really doing Agile.
Mind you, there are infrequent times when a new requirement or a bug report might actually trumps the sprint altogether... personally, I would classify those as emergencies. And let's be frank: How many actual emergencies do you think take place in the world of computer programming?
(Source: My employer sent me to ScrumMaster training a couple of weeks ago.)
I'm a former software and system engineering team supervisor/task lead, not a lawyer so you'll need to talk to your company's lawyer to provide more expert advice.
Tell your chain of command to grow a pair (in a more eloquent and enlightening manner), that what they are asking can't be done, and your estimate of when it can be done so they can inform the customer. Or they need to hire additional team members and adjust the price to reflect rush jobs.
When that doesn't work make sure to follow your appropriate reporting routes to your HR/Legal team that these requirement changes are negatively impacting not only the software team but the company's reputation. You'll need to make sure they get: a copy of the original requirements, the original schedule, the new requirements, your estimated schedule deviation due to new requirements, the dates of when you informed your chain of command, the actual schedule deviation, and the test results. Then ask them what else they may require from you to do or their advice to you. Now this won't "fix" the issue but you need to do it to cover yourself for the next step. Stop caring so much about meeting unattainable requirements and only work according to the original schedule + whatever free time remains in your scheduled work period (40 hours a week max unless your contract says otherwise).
If you end up getting lower than normal performance reviews because you haven't met unattainable requirements, you'll need to go to your company's HR/Legal team again for conflict resolution. If it ever gets to this point you and the HR/Legal team will have documented history of these problems which also show that you had attempted to handle the issue through the appropriate chain of command for your company. At this point if the companies lawyers aren't trying to mediate the issue contact an outside lawyer, but usually a company rather's handle issues internally so you shouldn't need to.
As an aside,
To help educate our customers when requirements will be met:
-We had weekly status meetings (no more than 10-15 min for any single project) where we used a diagram showing the workflow of the development process with the associated deadline/man hours for each step. Double check with your company before showing them a diagram like this though because some companies consider their development process proprietary information (they're all based off of similar schedule management techniques but companies are crazy at times, go figure).
-We also put up estimated dates for next releases of requirements with no more accuracy than which quarter of the year we thought it might fall into. This way the customer could think about changing around requirements if needed but wasn't going to assume they would get done by a simple date.
I assume your team is the developer pool for the company. If not, my comment is not usable.
Learn to use the forces piling onto you to achieve. If Bob in accounting is the client for whatever you are currently working on and Jim in sales asks for something, forward the request for pushing back Bob's feature to Bob. Then watch Bob and Jim wrestle on your behest.
Why does this work? Because people unconsciously know you are a finite resource. You are not tasked to choose what the company's priorities are, so let management choose the priority of each request.
Oh, and ALWAYS make an estimate of the work needed for development include the testing and some meetings. Meetings are like gravity: you can't escape them all the time.
You're not old until regret takes the place of your dreams.
This is a great way of describing Scrum, but I would be more clear about the fact that Agile != Scrum. Agile is the abstract base class (or maybe even just an interface/protocol) described by the Agile Manifesto: http://www.agilemanifesto.org/ and Scrum is a subclass that implements/extends Agile.
That said, the yardstick analogy is great and I'm going to use that right away in a class I'm teaching about Scrum!
Thanks!
PS. BSP below...
Helping with organizational effectiveness is our job.
Here is an article I wrote a few years about about this. Probably helpful: http://www.agileadvice.com/2012/01/08/scrumxplean/seven-options-for-handling-interruptions-in-scrum-and-other-agile-methods/
Helping with organizational effectiveness is our job.
Forgive me. This needs to be a quick note.
This is politics, promises, and a perpetual "Get out of Jail Free" card for your clients. (Not your customers.)
An ad hoc process isn't a process. GET _YOUR_ C-LEVEL EXEC TO EARN THEIR MONEY! You need an actual business process. Create, acquire, or adopt a process that takes these problems into account and implement it rigidly. This isn't the same thing as a complex process. This isn't the same as having a rigid process. It means that there are parts of the process that describe how usual development needs are handled AND how unusual development needs are handled. These documented process steps and their impacts are then are shared across the organization.
I'll bet your sales staff know exactly how their compensation is determined, including what will delay a commission to be paid and what may cause a commission to be withdrawn. This compensation development process is just a process- that works. Everyone affected by it knows how it works. This includes what role is responsible for what activity. This includes who determines the amount, who documents the amount, who verifies the amount, cut-off date for changes, who signs the paycheck, that it gets delivered, that errors can be corrected, & etc.. The systems and software development also is just a process- that needs to work. Remember that a process doesn't define "how" something is done. A process just defines what needs to be done and requires that you can prove that it actually was done. (For many positive and helpful reasons I don't have time to address here.)
Analogy: Get your boss to remind The Boss that a game isn't a game unless it has rules. A player that doesn't play by the rules isn't considered a player but as an obstruction on the playing field. The Boss needs to clearly establish the process for the start of the Development game, time length of the game, what is included in a game, how fouls are handled, the process by which a game is won, & etc.. Otherwise they won't be considered a Player in their organization. The Players will get The Boss removed or play around them. You've got to have a Process to be a Player.
-BW
The details of the problem are not as interesting as the underlying problem. It is human nature to want something for free. Agile beats waterfall because the tools are nearly free. Waterfall beats agile because it gives management some measure of control. Most development activity lie somewhere in between. People with political pull demanding features unreasonably usually have some measure of political pull to get their way.
The features themselves may be incoherent to everyone except the person demanding them. This of course means an analytical problem becomes a political one. Once real problems start to get solved politically the downside becomes unbounded.
Software is an engineered deliverable whose cost is very difficult to minimize yet it takes little to no skill to drive it in the opposite direction.
Requirements->Analysis->Specifications that can be coded to and are testable are like thermodynamics: inexorable laws of the universe. The circle of life. A century from now this will still be the central problem in systems design and software development. There is no solution. If someone says they have a guaranteed solution then they are liars, fools or both.
Ha ha, yeah, thanks for the correction! I only took a Scrum class, not an Agile class :P
But no, you and drawfour may not use the yardstick analogy unless you convert it to metric first ;-P
OK, J/K
At a company I worked for, the project request document template resembled a contract. Included on it was a statement acknowledging in advance that any change to the stated requirements would result in a mandatory delay AND cost increase. This quashed any kvetching about delays (because now they're not really late), and encouraged people to do two things: 1. Only insert changes that were actually absolutely necessary, 2. Save "nice to have" features for subsequent iterations.
Agile actually has some answers to these problems.
I think the first problem you need to work on is code quality. Nothing should get released that has not been thoroughly tested. There should be no compromise on this. You're already seeing the consequences -- things don't work, developers get blamed, and nobody is happy.
Next, realize that deadlines are bullshit -- especially in larger companies. I've found that arbitrary dates are chosen, and then they're treated like they've been set in stone. The Agile solution is for everything to be put in priority order. Always work on whatever is most important. Or put another way, work on whatever will provide the most business value. Management should be the final arbiters of what's most valuable/important. Once you start working this way (or even just realize that deadlines are made up), your stress level will go down significantly.
Being Agile means being able to adapt to reality. (I think that's Agile's main reason for success -- it realizes such things like the fact that we're terrible at estimating and works with that reality.) Your reality seems to be that people want to re-prioritize frequently and get features turned around quickly. So change your process to something that can do that. Stop doing iteration planning, since you don't know what you'll need that far ahead of time. Instead, allow stories to be re-prioritized until the developers start on them. And consider doing continuous deployments.
Alternatively, factor in the amount of extra work that gets added to every iteration, and leave that much extra time. This should actually already be factored in to your velocity, because velocity for iterations is defined by the amount of stories (or story points) completed within the iteration -- but only for stories that were discussed during the iteration planning. So for example, if you had 20 stories defined at planning, and 8 stories were added, but you only got 12 of the original stories and 5 of the added stories done by the original end of the iteration, then your velocity (what you can expect to accomplish each iteration) would be 12 stories.
Another reality that Agile accepts is that you can either get everything you want when it's ready, or whatever is ready whenever you want. Most shops tend to go with fixed time periods, but a lot are starting to move to continuous deployment. If you go with fixed time periods, they need to be fixed. If something misses the deadline for this iteration, it has to wait until the next. If things were correctly prioritized and your iterations are short enough, this should not be too big of an issue. If you go with something like continuous deployment, people will get what they want quicker, but will have to deal with more uncertainty about when they'll get it.
Of course, Agile can't solve every problem. If management is unwilling to prioritize things, or people are unwilling to deal with the reality of what can be accomplished in the given time, then you'll have to deal with those in the same way as in any situation. Which probably means learning to set expectations better, playing office politics, or finding another job.
Software sucks. Open Source sucks less.
once e'one is signed off on a version, it gets implemented and used before ANYTHING else happens. and the version must include how long the thing is to be used before the next one is designed. without this principle, yr dead. and if they don't like it, go find other work. sh*t flies.
Agile works best when it's both easy and cheap to build, test, and deploy changes to an existing product. If your testing processes are slow and painful, or if your finished product is a very well defined embedded unit, like a toaster, Agile becomes only one of the many approaches you might choose.
For example, if you're building a pacemaker or an airplane flight control system, there's an awful lot of engineering and design that has to go in up front. It all has to work together and be perfect the first time, because if you get anything wrong, you've got dead bodies on your hands. Agile may not be your best choice here.
If you're burning a CD-ROM release of software, you better be putting out a disk you can live with. What goes in the box needs to have a reasonable chance of working well at every client's site. If you have an automatic online updater, it's somewhat less important, but still, you don't want to ship a bad version.
But if you're building an iPhone app, you can put out a crappy 1.0 version just to see if people will fall for your advertising gimmicks and download it. Next week you can push a version 1.1 out to add a few features and fix a few bugs, version 1.2 can follow a couple weeks later, and so on. Most iPhone users are conditioned to clicking the "update all" button every day or two, and many people are tolerant of feature-poor apps. Same with web sites. You can release a new web site to the world four times a day, if you want. The trick is that it costs you almost nothing to push out a new version of software.
Even when you can't deliver to all your end users quickly or cheaply, you can almost always use an Agile or iterative methodology to evolve the product with your clients and beta testers. The ideas for a car start out as rough designs that are tested and iterated until a working prototype exists.
If your problem is with development and testing, though, then your options are limited. If you can't start by running automated unit tests and automated system tests of your software, Agile is just one more way to throw your money away quickly.
John