Slashdot Mirror
A Decade of Agile Programming — Has It Delivered?
snydeq writes "InfoWorld offers a look back at the first decade of agile programming. Forged in February 2001 when a group of developers convened in Utah to find an alternative to documentation-driven, 'heavyweight' software development practices, The Manifesto for Agile Software Development sought to promote processes that accommodate changing requirements, collaboration with customers, and delivery of software in short iterations. Fast-forward a decade, and agile software development is becoming increasingly commonplace, with software firms adopting agile offshoots such as Scrum, Extreme Programming, and Kanban — a trend some see benefiting software development overall."
A team in my the company I worked for was designing a new set of software and were utilizing Agile development. Well, seems they spent so much time going back and changing the software due to the changing demands that they never got anything finished to demonstrate so they wiped out the project team.
Quoth TFA: "Despite potential pitfalls, experts in the agile space agree that implementation of agile practices has benefited software development overall."
And in other news, despite potential pitfalls, the pope agrees that Catholicism has benefited religion overall.
The original manifesto's key message was "people over processes" which I still agree with. On the other hand the word "Agile" now means just as many processes as waterfall or RUP meant in the past. Shame.
I've worked in 100% waterfall, and in good agile environments, and I've found the key is to keep things small-a "agile", not to concentrate on capital-a Agile. Some places embrace Agile as a process, and fill binders with process documentation around their Agile process - at which point it's no better than any other.
I think the key to success is summed up in this line from T3FA:
if waterfall has delivered?!
It seems most projects work in spite of waterfall, not because of it.
I'm not saying it doesn't work. That is for small/medium projects with a very tight scope it does.
What's the name of that FBI project again?! Virtual case file?! Oh well...
how long until
In my experience, software development methodologies are more of a way of describing how teams already work. Sure you can put a name to it, polish it a bit, and other people can work toward that name, but there were tons of "Agile" projects and "Agile" groups before the manifesto. Maybe "Software companies" didn't do it before 2001 I don't know... but "software departments" of other companies have pretty much doing this since I have been around, which is a lot more than a decade.
The truth is that electronic delivery created agile development, at least for software companies. Internal departments have had the connectiviety for 2+ decades to deploy new releases often, but it wasn't till the late 1990s or early 2000s that a "boxed" software company could provide regular working releases to their customers (who wants to mail a CD or a stack of floppies every 3 weeks, and what customer would actually apply them?) Web-based apps and self-updating software just brought dedicated software development in line with corporate development practices, and then it was given a name.
Bits and pieces of agile are good. I think if you take the spirit of agile.. that is doing things because it's a good idea, not because you have a compulsive need to fill binders with documentation.. it can work well.
I like unit tests for regression testing (that is, verifying that the software still does what it used to).. but I think test driven development is more hassle and riskier than it's worth (now instead of just changing code when the requirements do a 180, you have to change code _and_ unit tests, ($cost + $time) * 2).
I like eliminating "because we need documentation" documentation.. but I think there is great value in documenting stuff that is complicated or weird. Having a binder full of "the UserAccount class represents a user account. It is comprised of a username representing.." is useless. The same goes with diagrams.
The worst is when agile is implemented as a buzzword. "We are agile! We have a binder full of documentation describing the rigid agile process we follow!" (not saying agile or processes in general shouldn't be documented, but in my opinion the whole point of agile is to be flexible).
2. It at least notionally asks the management to consider resources while committing to features.
3. It allows some kinds of software project to managed better. Things like merging products when companies merge, or when porting software to a new platform etc.
4. It works when the project has a large number of people with identical or largely similar skill sets. If the project has large number of specialists (like "Frank here is the only one who can even touch the turbulence modeling code") development will be slow, agile or not. At least the management should know not to commit to too many turbulence modeling features.
5. Rally proponents broad brush all skeptics as "people not willing to change".
6 Too many Rally concepts come from manufacturing and some of the examples and analogies don't even translate correctly. Take the famous, "burnt toast" example. A company decides to burn all the slices, and then scrape off the charred crumbs to get the color desired by each customer. Supposedly Rally will toast them right in the first try saving all the efforts that go into scraping off charred crumbs. Well, in software it does not cost me any money to char a toast and scrape the crumbs. Often times it is perfectly ok to render all the pixels of each body, even if another body is going to come back and override a lot or most the pixel later. You waste time trying to predict the final pixel value to render it just once.
7 Some times it is funny to see the Rally proponents not using Rally methods to develop their presentations, not using Rally for their own internal websites! Too many of them recite from a text book or a holy book instead of using actual code/project examples.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
I appreciate good management. I can live with no management, but I can't handle bad management.
SCRUM has sort of become a device for a manager to avoid managing. The human in the picture is replaced with a sprint chart and backlog tracker. It lets bad managers get by, while good managers are really thrown out of the picture.
I hated scrum in my old job. But the new job just throws out a list of features to implement, ranks it and throws it at one of us. There are no punishments for missing a day, no tracker to guilt-trip you and most importantly, the stand-up meetings are just before lunch. And mostly, serves to keep our communication channels open across the team.
I hated the time-keeping TPS report style scrum, but I'm cool with the new approach. I love the idea of sprints and taking a week out of a month to hammer something out. But this system only works with motivated teams with a fair scrum-master.
But I repeat, it is not a replacement for good management. It is as good as a way of letting me manage my tasks,but please (for the love of God, please) do not use it to manage me.
Quidquid latine dictum sit, altum videtur
In my experience, Agile works for two things:
1) Maintennance and support development
2) Extremely strong engineering teams
Formal processes with more up-front planning and review like waterfall allow you to turn out high quality product with average teams. You can't use the typical Agile processes with average teams and do substantial new development, but I've seen it work well enough for incremental development or support engineering/bug fixing.
A good manager will know his or her team and use the methods appropriate for success with that team, and not chase buzzwords.
In my experience, though, these days there are a LOT of bad managers.
From languages I've seen, programming has regressed in the last decade. When Grace Hopper wrote the first compiler, her dream was an english-like language that computers and people could both understand easily with a minimum of training. She co-wrote COBOL as the second compiler.
Older languages, like the mainframe DBMS NOMAD and the PC DBMS dBase were IMO brilliant. They needed little documentation, and no curly braces or other such nonsense you find in modern languages like C or Java.
The "language" I hate most is MS Access. Convoluted and unintuitive, you don't really write anything, and what's produced is "visual basic" and the equally unreadable SQL.
I hate it. I'd rather program in assembly, so I can know what registers are being accessed, whether I pushed or popped a stack (and which stack), etc.
I hate programming in a language that doesn't let me feel the metal. Hell, I'd tale old fashioned 1980 BASIC over the newer languages; it doesn't take long to learn not to write spagetti.
Free Martian Whores!
I work in customer support for Agile Web Solutions, the makers of 1Password (a password management system) for Mac, Windows, iOS and Android. Agile development seems to work well for us.
I think that there are two reasons why this has worked well for us while not for others.
There are drawbacks, of course. What we like to think of as "surprising and delighting" and delighting our customers usually works, but sometimes we have to take steps back from something visible which we've tried. I think that over all this still is a "win" for us and our customers, but it can sometimes be disappointing.
A perceived (but imaginary) drawback is "wasted effort". We put a great deal of effort into getting data syncing among machines and devices to work via webDAV (and in particular, MobileMe). For a variety of reasons we had to abandon that approach and go with Dropbox (with which we are very very happy). To some this might seem like wasted effort, switching to a different approach after a great deal of effort has gone into the first one. But in fact, agile principles in this case simply mean that we don't fall victim to the sunk cost bias.
Prime numbers are exactly what Alan Greenspan says they are -S. Minsky
"People over process" is an important tenet in more ways than one, and not only in IT. Often, this message is taken to mean that teams and individuals are given more freedom to organise their own work, and are managed on outcome rather than activities. This is true in some ways, but it's more than that.
"People over process" to me also means acknowledging the value of each individual's skills and abilities. That starts with resourcing: instead of posting jobs for "2 junior programmers, 1 senior programmer, 1 encryption specialist (part time) and 1 PM", one would think "Together, Alice and Bob are up to the task of writing this module, Alice knows enough about encryption to cover that part of the job, and Bob is a good team lead". No two people are alike, and if you value people, that means that you account for their differences as well. Yes, it also means that if Alice calls in sick, you have a problem on your hand, but it's by no means an insurmountable problem.
And it goes beyond that. "People over process" to me also means that you let people do what they are good at beyond the call of their assignment. (as long as it benefits the company). As an individual, it means that you sometimes take responsibility for stuff that is not strictly your job. If someone contacts me with a problem I can solve, I do not answer with "Contact the helpdesk"... if it's likely the problem will get kicked down 3 layers, taking 2 weeks to come up with a non-answer. Instead I will take an hour off my regular work to provide a helpful answer (of course with the suggestion that they contact the helpdesk, and copying in support as well). This helps everyone, including the company.
Yes, accounting for individual differences makes budgeting, resourcing, managing continuity, and people management a lot harder. That's why managers hate "people over process" and why Agile IT gets loaded down with more process: resources are so much easier to manage than people.
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
IT, or at least some people in IT suffer from what I call the "Oprah" syndrome. No, not the weight issue. The idea that a problem is just one problem and that it can be solved with a single solution. Or the silver bullet that will solve all your problems at once.
Software development is bloody hard, partly because it doesn't deal with a real product and testing a software product is extremely abstract.
If I design a better brake pad, I can test this fairly simply AND thorougly. We know exactly what it is supposed to do, how it does it and how we can compare performance. There is no magic, if the brakepad reaches 198 degrees on a sunday, the trunk will pop open. Also nobody will ask a brake pad engineer to add a christmas special to it, on the 25th of december, at 8 in the evening.
Web development especially suffers from the idea that if only we implement magic method number #45324521, all the hassle of web development will go away. Writing specs is long boring and not very sexy. So lets do agile and not write them anymore. Problem solved... eh no because spec writing is long boring and not very sexy because getting the requirements out of a user is like sucking cock. The user might enjoy it but you are just left with a bad taste in your mouth and a desire to throw up over the user. Or maybe that is just my dates.
"But if you do agile right", but if you do documentation driven development right! Any method, done right will most likely work. That is the definition of doing it right. Why is it so hard to do agile right? Why are there so many variants already?
I see a lot of "magic" solutions in web development.
Database abstraction, so we can magically switch database!!!
Question: When have you EVER switched database on a web application and HOW easy was it? Is there ANYONE out there who only use pure SQL that is 100% understood by all databases the same? Then go stand in the corner, because your code lacks any optimization. Real developers optimize their code for the specific environment they are using. This includes making use of database specific features. Don't use Mysql auto-increment and PDO and think you are database independent.
Frameworks take all the hardship out of writing code!!!
Question: How smart do you think it is to hire a coder who hates to code? It is like hating a singer who hates to sing, a spokeperson who doesn't like talking.
This is where a lot of the silver bullets come from, from people that don't actually want to be a developer but just cash the paycheck. You don't hire a cook who only knows how to nuke frozen meals do you? If you go to the supermarket and buy a microwave meal thinking wow, this makes cooking easy, then here is a hint: IT AIN'T COOKING. It is heating up food.
I see time after time some framework or tool promosing a complete working website without writing a line of code... EEK! That is my JOB! Luckily they are all wrong because what at most they do is create a straight CMS for your database, which is rarely what the customer wants. After all, there are already plenty of tools to graphically maintain your database content.
No, Agile hasn't delivered... for those looking for a silver bullet to the problems of development. For others, it is a valid method with its own problems that puts it along side more traditional models, not ahead of them.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
Reminds me of an old poster from my CS department. It features legendary crotchety old fart John McCarthy angrily looking at the camera, with the caption "Programming: You're Doing It Completely Wrong."
SJW: Someone who has run out of real oppression, and has to fake it.
In my experience plenty of teams "do Agile" without understanding what it's supposed to achieve. People just cherry-pick and adopt well known bits of some Agile methodology or other (for example the stand-up meeting) but then don't do it properly or miss the feed-in practices that are necessary to make it work.
It's all about "Being Agile" (i.e. fashionable) instead of "Having a way of developing software that can consistently cope with fast-changing requirements minimizing wastage, chaos and cross-team overhead".
Just as an example, in the last 7 or 8 years since Agile became fashionable, having been in and out of multiple teams/companies that use Agile to a smaller or greater degree (I'm a freelancer), I have yet to see a proper Use Case which describes in a consistent and well defined way a feature that the system being designed must provide, including handling of error conditions. In fact, 9 out of 10 times, people forget to account for errors (even user errors). Use Cases are the basis for each development cycle, the point of communications with those defining the requirements and the basis for prioritization of work and yet most teams can't even get those right.
Then there's the flaws in analysing the requirements to do make sure the system has all the data that it needs (if for example one Use Case says "The user will select one value out of a list of options for field X" you need to have those options coming from somewhere, potentially ending up with a whole bunch of other Use Cases dealing with maintaining those options).
Agile doesn't solve the essential problem in IT which is a lack of understanding of the software development process, lack of preparation and lack of method - it just provides cover for those developers which do everything in an ad-hoc, reactive way and will then excuse themselves as being "Agile".
There are simply not enough people around that try to understand the process by which people create a software system or application from a "not so well defined set of needs from the users" but lots of people focusing on the quasi-aestetic details of some language or other - too many people asking "How?" not enough asking "Why?"
I think that you spent your time in a wrong kind of stand-ups.
You see, if you have up to 10 people on your team, and each one gives precise answers to those three scrum questions, you are gonna spend no more than 15 minutes in that meeting. After that, you know what all other people are busy with, so should any issue pop up, you know who to ask. This is less time to spend than if each one were asking another one by one.
Scrum done wrong: 1) a half an hour stand up; 2) scrum done in teams with more than 10 people; 3) off-topic discussions during the stand ups.
If you are saying you can fix half a dozen bugs in 15 minutes, then what the heck were you doing yesterday? Pushing trivial to fix bugs so that you can overclock your bugfixing karma the next day? Doesn't look like competent, anyway.
If you read the original manifesto, it is very carefully worded. It does not say "abandon processes", it says that the priority should be on the developers as creative individuals, instead of mechanized drones.
Also, it is not against "processes" (with small "p"). A build system is itself a process, a source control software also enforces a process. The manifesto is against Process with a capital P. It is hard to explain, but easy to give an example what I mean about this:
http://www.ogcio.gov.hk/eng/prodev/es3.htm
The above link is about the SSADM Process. Read it and you will immediately understand what I mean.
Also, lot of people misunderstand "Working code over documentation" and think that documentation is not important. In fact, it should be read primarily as "project documentation", the things that most old fashioned processes mandate. Again, look how many and what kind of docs SSADM needs.
The original manifesto's key message was "people over processes" which I still agree with.
This is I think, a totally key point.
Agile is supposed to describe a way in which people can work together to improve productivity. If you think about it, what it's really doing is attempting to document how a successful team (or one kind of successful team) interacts, like Goodall studying mountain gorillas.
So then other companies want to have successful teams too, and study what was done. But instead of saying, here's an approach, everyone read this and lets try something similar, instead of that companies all over have managers reading agile books and then using that as a template into which they cram all the developers they have, ignoring the differences in the way a given group interacts with each other.
And that's because it's easy to do one thing, but very hard indeed to adapt something for the individuals doing it. The payoff is enormous if you can get individual tailoring right, but it's so hard to get right I don't know that you can even try except to try and recognize signs of success in groups and encourage them (which in turn leads to cliques if you are not careful).
It's something that happens to any methodology, in short order it will ossify into rigid methodology because the communication of how something works spreads to most people in the form of text that looks to managers like a template for how to arrange "resources".
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Absolutely not. Never before have I experienced a method of programming that can be more aggressively mismanaged than agile. Many developers think agile is a means to produce better code, faster, and with better specs. Most management sees it as an excuse to provide no specs, to change their minds on what they want every 5 minutes, and call all of that "agile" development. These things ruin the name of agile development and provide such a bad experience for anyone stuck working with it, that it simply can't have managed to do anything but fail utterly to deliver. People have been fired over misinterpretations of what agile is. Others have left because of a misinterpretation that was shoved down their throats by management. And that's just all where I work... Perhaps it's better in other places, but I've never heard a story of it going well.
This is pretty much the ultimate "get off my lawn" post.
Having worked under the Agile whip for the past year, I have to say the methodology is horrible. The planning is ridiculously costly, and the constant pressure makes people manic about their work. Ultimately, people stop caring about the sprint deadline and just resort to letting tasks go undone.
No; I do - let's say try to do Agile and I'm a CSM, since its inception and on different locations and cultures; in the last 5 years, when I did it mostly in North America I never seen it succeeding; mostly the reason was hijacking the Agile and masquerading waterfall or chaotic processes under the Agile terminology/dictionary. As always, human greed and immorality overtook the basics of the Agile manifesto and lead to disastrous mini-waterfalling with dire consequences as accumulation of huge technical debt, broken estimations missed deliveries and developers psychological burn out and collapse. Of course, the poor slaves pushed to work crazy overtime and crunching tasks where always at fault. The core problem I always pointed at in the Agile communities I had the chance to talk about, is that there are no *Agile Certified Organization* certification criteria and accreditation auditing processes and trained, certified and experienced auditors. This leads to a gazillion of organization declaring themselves as Agile by saying that they do or try to do Agile but without *being Agile* in reality. Until we will have this accreditation mostly we will see failures and this will lead also to Agile going to the garbage bin of the various trials in doing better software development and having a decent profession - not one well known and renowned for all possible worker abuse and wrong doings. At the end, maybe there is no method to do better software development because of the nature of the people and the context of being a no regulated profession - this opens the door to everything and in this libertarian approach always the winner will be the one owning more power - in this case the positioning in the monetary interest stack of the project. This is nothing else than feudalistic primitivism which in time will destroy the whole industry as the older ones will pursue different career options - and I know lots of them who already did it, and the young ones will not step into this industry anymore. Wherever I have the chance to participate at interviews I see mostly immature kids and not only, some are close to their 30s, but their professional and social skills are so low that many years ago these people wouldn't have gone even over the phone screening phase.
Yes, anyone adhereing to a strict waterfall model today is probably being silly.
If you are doing internal-only development you can get away with constant change. When there are real outside users there is real, external documentation and marketing materials. There is a real date for a product launch, and things have to be stable for it.
Marketing materials need to be prepared and printed. If they take a bunch of screen shots and sent stuff to the printer Marketing will not be happy with the announcement that those are the "old" screens and the "new" ones are much better now.
An really funny scenario is some marketing type is going to give a demo of the product to some big customers (prospects, really) and no longer understands how the "new" product works after the latest round of changes.
Yes, I have seen that happen. The result is there are some new developers working on the product and the launch is delayed. Sometimes for a year. Sometimes the original developers have a hard time finding a new job.
Change is good, stability is good. The intersection of these two is really great. Anything that is too far away from the intersection tends to be bad, especially at the outer edges of either.
The two take away message I got from that was that working at Google is awesome and that the stupid blog poster can't understand why everyone doesn't work like they are at a cash rich company developing internal research projects without external clients setting the requirements.
Puzzle Daze is now my job
While you have a valid example of why database abstraction CAN be useful, you have not actually proven that it must be done every time just in case you might switch sometime in the future.
Seen to many developments throw speed out of the door for a migration that will never happen.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
Also, it is not against "processes" (with small "p"). A build system is itself a process, a source control software also enforces a process. The manifesto is against Process with a capital P. It is hard to explain
I don't think its really hard to explain, though I think the best explanations I've seen are in books that focus on "lean" methods rather than "agile" ones (which, really, aren't all that different; the principles are largely similar, though "lean" seems to focus on an enterprise wide approach that lean software development applies to software development, while "agile" seems to focus on software development with applications to the broader enterprise particularly the interface between software development and business):
It is important to have clear processes that people follow, because otherwise you have wasteful churn as extra effort is spent doing everything ad hoc. But you have to respect the people doing the work to the point where they are expected, on encountering a problem with the existing process, to call a halt, propose and test an alternative, and then that alternative, if it is an improvement, is adopted as the new process.
That is, just as software being developed needs to be developed in small increments and able to respond to emergent discovery of requirements, so the processes used in software development (and, really, throughout the enterprise) need to be able to be adapted rapidly in response to emergent requirements. And, to do that, the people that do the work have to own the processes by which the work is done.
This is opposed to devotion to enshrined Processes that are treated as received wisdom and not subject to question or revision by the people actually involved in doing the work.
Old School here too. I do what I have to, to make a living. For fun, I program in FORTH and store my source code in 1K Screens represented as 16 lines of 64 characters each.
Simplicity and elegance is what I am looking for. A new Forth definition should be about 7 to 9 words long (not including noise like + - / ). If I somehow end up in a situation where I run out of room, It means I have used 15 lines for creating my definition. Which is another way of saying "Hey! your doing it wrong".
I think programming should be taught on an emulator of a Commodore64. Once you learn the computer from one end to the other and how to take advantage of all of it and understand all of its concepts, then you can move onto programming in a more abstract environment.
vi +
Frameworks aren't "automatically generated code" (typically), they're libraries of code written and tested by other teams in the past, along with a skeleton code structure to fill-in.
I agree with your sentiment, but it doesn't apply to frameworks.
Comment of the year
I think programming should be taught on an emulator of a Commodore64.
No, 64k is too much memory. Teach them on a TS-1000 emulator with its 4k or RAM. You learn to write really tight and fast code when you're programming in 4k on a chip that's only 1 mHz clock speed.
Free Martian Whores!
You like COBOL because its a nice verbose language. But you had verbose languages because you can't tell what stack you are pushing? You love BASIC, but hate Visual Basic, I guess because there are no line numbers? Make up your mind. You know COBOL doesn't give you any more control of the underlying structure than C# does.