Slashdot Mirror
Organizational Patterns of Agile Software Development
Organizational Patterns of Agile Software Development starts by describing the foundations of the authors' research. There are definitions of a "pattern" (but "your intuition about the meaning of the term will take you far") and of a "pattern language" (read the book), the history of their research, and some information about how the book is laid out. The authors recommend you read this section, and so do I; but if it's too dry for you, by all means move on.
The meat of this book is four pattern languages: how to manage a project, how to grow it over time, what can make up an organization's "style" (I'd use the word "culture"), and how the people fulfill their roles and interact with each other. These are not prescriptions or algorithms; they're elements of how successful organizations have worked.
Each pattern describes one aspect of some effective software development organizations. Some patterns are found in more than one pattern language; "Community of Trust" is common to all. Others are less general; "Moderate Truck Number" applies only to the "piecemeal growth" pattern language.
How valuable are the patterns? Some (such as "Get On With It", proceeding with an effort before the planning is considered complete) are common sense. Others (for example, "Don't Interrupt an Interrupt") are things you probably know, but might need to be reminded of ... or might need to remind your boss of. More than a few (my favorite is "Architect Also Implements") might help you understand how something could or should work. Finally, there are some patterns here (such as the "Day Care" pattern for training new members) that might be new to you.
The rest of the book puts the patterns and pattern languages into perspective. There are chapters on organizational principals and (seriously) anthropological foundations of this work. Then there are two case studies of very successful projects. On one, "[about one] million lines of code were written over a period of 31 months by about eight people (that's about 1,000 lines of code per person per week) -- that doesn't include code in the [two] prototypes." It's easy to crank out code at that rate for small bursts, or on small projects. To stay at that pace constantly for over two and half years is nothing short of astounding. The resulting product was released to great reviews. (It then did poorly in the marketplace when it went head-to-head with a directly competing product from Microsoft. Sound dissatisfying? Consider how very long people waited impatiently for Mozilla and its successors such as Firefox. More directly, look at Robert Glass's assertion of the "disconnect between managers and their programmers" as to what projects are seen as successful; it's Fact 13 in Glass's book reviewed August 30th on Slashdot.)
What's imperfect about this book? A couple of things.
First, sometimes the language gets too academic for easy reading. Example: "We have also seen a lighter though almost equally destructive form of this phenomenon, which we describe as schismogenesis.... Symmetrical schismogenesis occurs when two factions each rise in power (or in fear or distrust of each other) and form cliques or splinter groups that tend to focus inward rather than resolve issues in the dialogue with each other." Clear enough if you work on it, but a little intimidating.
Second, the book is surprisingly partisan on some subjects. The book is not kind either to ISO 9000 or Extreme Programming; it could serve as a sort of litmus test, delighting critics and coming across to supporters as unfairly harsh.
What's good about this book? It's a collection of good information, well presented, with information on how to apply it, on a topic where not much knowledge has been accumulated. For some specific circumstances, this book sometimes points out different likely alternatives, with information on when each is applicable. Don't expect Organizational Patterns for the Complete Dummy; then again, don't expect anything useful to be superficial.
How could Coplien and Harrison's work apply to open source development? For starters, they point out the value of people working physically together, and of individual code ownership; these aren't easily applied to open source, but at least it points out forces that need to be resolved somehow. On the other hand, some patterns here are hugely relevant to open source: "Work Queue," "Informal Labor Plan," "Self-Selecting Team," and "Team Pride" come to mind.
Organizational Patterns of Agile Software Development is no panacea. If your organizational practices are the opposite of what's found to be effective, you may find this book frustrating. A book can't take your organization where it needs to go; but Coplien and Harrison have put up some road signs.
You can purchase Organizational Patterns of Agile Software Development from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
...about patterns and architecture and such - On the Nature of 'The Nature of Order'.
It's a fairly short paper and can give you an idea of his style - or at least his style as it was 7 years ago.
The Army reading list
Simple. l33t haxorr sk311z.
An Indian-American Hindu committed to non-violent thought/speech/action alarmed by the global explosion of radical Islam
Those blessed to work in a programming environment with only one or two other coders always enjoy a productivity advantage over our friends in much larger teams.
.
I honestly think that most of these patterns could be synthesized simply by looking at things that good small teams do. Each of the patterns described seems like a no-brainer that we've likely had some experience with -- but the same harsh criticisms have been levelled by less-than-Humble Programmers against the GoF's Design Patterns: they're window dressing, they're an attempt to claim intellectual real estate that everyone with a brain already uses .
The critic would be well-served to read The Humble Programmer again; I mean, it's ridiculous that it has taken the time it did for the notion of Design Patterns to come about when Djikstra specifially described the concept in '76.
This book, and it's potential benefits, should be looked at from the same viewpoint that intellectually honest people looked at Design Patterns.
Unfortunately, many bright (but not bright) programmers saw Design Patterns as an attempt to undercut their own intellectual conquests, an attempt to lend mass production to the concepts that they had discovered for themselves, inch by inch. It takes considerable humility and intellectual honesty to benefit from DP's -- programmers are probably one of the lines of work where those two qualities are so strongly stressed (Code Complete, etc).
I would probably buy this book when and if I went independent (likely not within the next five years), but I don't hold out a lot of hope for it for the simple reason that it seems to also require intellectual honesty, and humility.
In MANAGEMENT!
after taking software engineering, it seems everyone feels the need to write a book about the best way to develop software. What about all the other previous ways? They aren't valid anymore? were they ever valid in the first place? is this *new* way to develop software more valid than the rest?
did you forget to take your meds?
2. All the soda you can drink before 5pm.
3. All the beer you can drink after 5pm.
4. A dev machine with the best graphics card, raided SATA drives, 3+GHz, and a raw, unrestricted net connection.
Free food and drink, Half-Life parties after 5pm...what more could any geek want? They'll pump out more code than you're slowbie QA people can handle.
Mentors, the Freshmaker!
(I apologize for this, really, it's just been one of those days).
It's ...
... Hey, what do you mean, Agile? I saw a fluourescent green book on 'Agile Extreme Programming' at Borders, I thought you didn't follow all those stupid fads."
The Teach Teach Yourself eXtreme Test-driven Agile Development Patterns in 21 Days By Example Bible.. JAVA EDITION!
Seriously I believe strongly in Agile methods (even before they got the name "Agile") but please let's not dilute the name "Agile" to the point that I'm embarrased to use it, okay??
Me: "I think we should stay agile and use this simpler algorithm until we have some real-world benchmarks."
Coworker: "Okay.
Me: *grumble*
Successful software projects have great leaders. Leaders that understand what needs to be built and what doesn't, what can be done and what can't, and who works well on what. Good leaders are accountable, have the respect of there people, and know there shit.
If you're leading a software project and reading books like these your project is already screwed.
From what I've seen, the thing that makes or breaks a leader is whether they want to get the job done or whether they want to get recognition. Those that lead to get the job done, are the ones that tend to be very effective, and those that want to be constantly recognized tend to not be. It's really common sense. When you're secure enough about your own abilities, you don't need to have them constantly reinforced and recognized.
What I think America needs more of us is a military-influenced leadership culture. Not too heavily influenced, but enough that people understand that when you work in a team, even as the leader, your success and the team's success are one-in-the-same.
Of course schools often discredit this by having teachers who will grade down an entire team because of a slacker or two's stupidity, whereas in the real world the slackers would get fired for bad performance. I have had a class before in college where I got graded off because another team screwed up and made it impossible for my team to finish our work.
It seems to me that the best anti-dilbert solution is to instill from an early age the value that if you screw up, you'll be accountable, and that when you succeed that you share your success with those that made it possible.
Click here or a puppy gets stomped!
The gist of the article is that first-rate problem-solvers are not well suited for "agile" methods, because they tend to be more individualistic (and the agile methods tend to force programmers to interact continually).
I especially thought it was interesting that programmers with good people skills do better in agile methodologies, and that there are many more programmers than there used to be who have good people skills. This could be the death of the Dilbert stereotype....
Have you read my blog lately?
Agile is meant for projects where change is a fact of life. If your project has explicit, written in stone, unchanging goals then you should consider another design methodology.
Agile development focuses on rapid delivery of working (but not polished) software. It should be lightweight, but still accomplish the functional goal.
Agile projects should provide value long before they are finished. Each increment of functionality should be useful for a customer.
Agile is not meant for large teams. Agile does not work well with customers who can not provide frequent feedback. Agile is not optimal for projets which have stable needs that are known from the beginning. Agile is not a good choice for mission critical products.
These are the basics as taught to me, they may not be the same as presented in this book.
The book is On Line. Go, read it.
The most successful projects I have worked on were always blue-sky projects, were where:
o Any existing utility libraries had been heavily tried and tested
o Everyone was enthusiastic about the particular area of work they were doing, as new code was being designed.
o Everyone had separate areas of work, and there were well defined API layers between each module.
o There were only two agenda's affecting everyone
- Management wanting the work complete
- The programmers/software engineers
wanting the work experience
The worst projects have been:
o No well cleared API layers, and programmers belting code into each others routines without checking with each other.
o Senior management (in head office) messing everything up by adding a third agenda and switching everyone around for the sake
of having the brighest graduate working on XXXX instead of YYYY.
Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
If your project has explicit, written in stone, unchanging goals...
Then you've drunk the Kool-Aid. Go throw up and quit trusting the customer not to screw you over. Now!!!
Agile is not a good choice for mission critical products.
Riiiiight:) Mission critical (as in "this needs to happen or the company folds") products need to spend months doing BDUF (Big Design Up Front) in order to stay in line with the 70% failure rate that business has come to expect (and resent) from software development methods.
The two day class sounds like it might have been softened up for newcomers so as to represent Agile methods as not being too disRUPtive (get it? ha ha) to industry leaches who push big frameworks and methodologies wich people stake projects, budgets, and careers on.
Disclaimar* - Keep in mind, I am part of an XP Team.
*** Sigs are a stupid waste of bandwidth.
Like in DePalma's The Untouchables
Like when a Yakuza thug fucks up, but worse.
-kgj
-kgj
Let's not be naïve; obviously, successful programmers -- as is stressed in one of the books I quoted as a reference, Code Complete -- always need to know how to work with others and have a realistic, comprehensive understanding of how the needs of management, and the requirements of the project, work together.
:)) of us live in, management is a necessary shield from inefficiencies of a beaurocracy, and more importantly, is possibly the only way that programmers can get their information about the requirements of the system. The burden is laid upon the manager of really communicating effectively, of communicating requirements, of clearly expressing the company's needs and position on a project, of ensuring that programmers have the correct context available with which to successfully approach a problem.
.
However, it's naive to think that the team of good coders -- and by now it should be obvious that I'm not using a narrow-minded 'creative hacker' definition; I mean good programmers, good architects, good with people, good communicators -- is really better off, in terms of potential, when they have a manager -- even one running interference -- than in an ideal situation where each programmer can know the point of view of the customer, can sit down with the customer and get inside their head and their business and bend their minds to the problem of delivering software that lets their customer take over the world on the one hand, and works within the budget of and accomplishes the goals of their company.
Manifestly, an ideal situation is one where the programmers know the needs of their company -- because they know its state, possibly have some ownership. Where they know the needs of their customer -- really know them inside and out. There is a reason that some of the most successful stories when it comes to programming are those involving companies and teams of very small size. Viaweb (Paul Graham) is an example of an agile product; features were often rolled out within a day of introduction by a competitor.
In the less-than-ideal world that many (but not all
That is a huge burden. And the fact is that most managers of IT professionals do not know how to manage the special case of programmers. The fact that the programmers often have to prod their management, gouge them for any sliver of information that could lead to an actual understanding of the real requirements and goals behind a project -- this fact is not a proof that coders need to know how to "work with" management.
Programmers will often go above and beyond the call of duty to solve the problems of the organization in a very real way -- often, as we have seen in the cases of small, successful, programmer-driven companies, in a real enough way to rocket a company from obscurity. A Manager is not going to do that.
He can, however, try to present a more ideal environment for his problem-solvers to solve problems effectively within. He can communicate ceaselessly, so that he's not excluding what could be data that is crucial to a complete or correct understanding of a problem. He can educate himself by reading the last few chapters of books like Code Complete and Rapid Development (these I picked just because they're on my desk right now; there are plenty others. The Pragmatic Programmer is solid, and there are a good dozen or so more that should be requird reading, including some of Joe Celko's later articles in Intelligent Enterprise, like "The Logic of Failure"). A Manager will likely be surprised -- if he has good coders, he will find that many of them have read all of those books already, and he will be amazed at the amount of practical knowledge they have about the realities of scheduling and organizing software projects. Rapid Development alone . .
Realistically, most programmers with that kind of an understanding of the issues involved -- and, of course, knowledge of the appropriate principles of abstraction -- will probably ha
Not all customers will screw you over. Many of the will, but it isn't always helpful to mix customer relations with extreme skepticism.
Mission critical is no so much "this needs to happen of the company folds", but more "this needs to happen or (bridge collapses|spaceship crashes|patient dies)"
I couldn't agree more.
At its core, that is the problem.
And actually, coders have been marginalized for a long time -- I don't think that many non-programmers realize that the amount of time that a programmer spends thinking about his work, and actively working on the problems related to it, is easily on par with that of other creative problem-solvers -- architects, mathematicians, engineers everywhere.
However -- and this is something that almost NOBODY outside of programming/mathematics/compsci realizes -- programming is so very different from almost all of those disciplines (not counting mathematicians, obviously, since nowadays an increasing number are also computer scientists by necessity). Programming ends up requiring a deeper committment because the possibilities are so much greater.
An architect may work his whole life, design thousands of structures, and finally write a book about some of the fundamental principles that the good designs share.
Likewise, a programmer may create a system every bit as complex, and frequently more complex, than the overall complexity of the CAD for a medium-sized office building. However, the good programmer immediately looks for the common elements, and is in effect able to -- because of the radically different nature of programming from most other disciplines -- from that point on, deal with those principles in the abstract. This is something that you can only do in a limited sense with non-programming jobs.
After all, repetition is a part of life, and life experience, but in programming you can solve a problem "once and for all". The only use for experience is, not in having solved a problem before, but in learning how better to understand a problem.
In effect, then, good programmers are never solving the same problems twice, and thus everything they are working on is new to them. If it's not a fundamentally new problem, the effort required is often trivial in a large subset of problems where the architectures exist for a high degree of abstraction (some of the problem areas being GUI programming; toolkits like Glade are still a ways away from providing the kind of abstraction we can enjoy in, say, the persistence layer).
All good programmers are constantly trying to understand problems better, trying to understand methods and techniques better, and most of all, trying to understand themselves better so that they, as humble programmers, can better understand how to use the limited capacity of their skull to accomplish potentially unlimited things.
What this means for managers: if your company hires good programmers, they have the capability to design a solution based on a problem. The more they understand the problem -- all facets of it, including what the user expects to see -- they better the result will be. There literally is a direct connection between how "good" a programmer is -- in terms of problem-solving, not hacking -- and how overall effective and useful he will be, and the quality of his results.
So -- and stop and think about the implications of this -- programming may be one of the few job categories on the planet where regular, continual, honest self-evaluation and training and personal improvement are a job requirement.
I don't think that most programmers need to worry about yoga classes or philosophy classes. All they have to do is continue to elevate their level of understanding, and these improvements come with the territory.
This is what I, and others, mean when we say that we doubt that Design Patterns will catch on with management! Design patterns are there to take the mystery out of abstraction, to take the mystery out of efficiency, to take the mystery out of success! This is the kind of thing that is fairly unique, and doesn't tend to be well-received, let alone become a cornerstone of an industry.
Contrast this with management. As someone who's had to take a managing role in a lot of different contexts, I haven't heard the
Regardless of the buzzwords, early prototyping and change management integrated into the base release are the best way to ensure a manageable system/human process in the future. Business gets more value sooner and understands the cycle sooner.
A lot of maintenance issues come in transition from the base release to later releases. If the typically more senior programmers that start a software product behave (clear code, documentation, training, whatever) so that they or more junior programmers can quickly understand what they did (automated Unit Test) there is a much easier transition later.
The real problem comes in when the senior designer / programmer abruptly moves on to the sexier product (or company) and the junior guys are left for the next design/implementation. This is ususally where products get fscked up. If the handoff to the junior dudes is not clean including working unit tests, the product starts down the doom slide. If the extension designer is on his first non mentored system, watch out!
The fundamental problem in the software business is that it is our job to learn and apply learning every day. We like that. We like it so much we usually do not stick around to work on our systems in the maintenance mode (always another new/better system/tool/language to learn) which is also why the academics do not address it. Good methodologies going forward will minimize or address the impact of the learning programmer.
I like automated Unit Tests (used 'em pre xP) because they capture more of the code level design and clearly indicate the extent to which the inital programmer considered the problem -- along with well written code itself, this is as clear a knowledge transfer as you can get without the old boy around.
Why do software projects fail? - Hanging on to long-evolved, poor design instead of refactoring when needed. Sometimes starting over is the only solution, but it is nearly impossible to convince a company to rebuild, from scratch, an entire system that they had custom developed in a timespan of over 10 years. - Attempting to build a super-system that 'does everything' instead of serveral smalls systems that just do their part. A small, modular system (even being part of a bigger whole) is easier to replace than a huge one. Results in the situation described above. - People entering and leaving projects. When people leave a project, so does the knowledge that they built up and invested in the project. Documentation is harder to find than the tooth fairy. - Managers pushing the wrong technical decisions, not being able to forsee their implications.
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book