Scrum/Agile Now Used To Manage Non-Tech Projects

jfruh writes "Agile and, in particular, Scrum, have been popular project management methods for software development for more than a decade, and now its use is spreading well beyond software. For example, NPR is using Agile for faster, cheaper development of new radio programs. 'I was looking for some inspiration and found it one floor up inside our building (where Digital Media sits),' says NPR vice president of programming Eric Nuzum. NPR has used this 'Agile-inspired' approach to create several new programs, including TED Radio Hour, Ask Me Another, and Cabinet of Wonders."

Comments

Agile / Scrum method vs Brainstorming

[Taco Cowboy]

I am adapting some principles from the Agile / Scrum method in managing my business

However, I find that in our monthly brainstorming sessions - the traditional brainstorming way still generates more leads than the Agile / Scrum method

Anyone have any experience to share ?
 

Re:Agile / Scrum method vs Brainstorming

[eulernet]

Agile/Scrum is not very good for brainstorming because it's focused on the process.
In fact, brainstorming creates a lot of ideas, but most of them are garbage.

I would recommend that you use the retrospective tool as follows:
  - first, use a timeboxed meeting (for example, 1 hour)
  - when the meeting starts, explain the goal of the meeting: to generate good ideas
  - then ask people to list the problems or the parts that could to be improved
  - then ask them to vote for the most urgent problems (give 3 points to each participant, and ask them to place their points where they want)
  - then take the most voted problem, and talk about it WITHOUT searching for a solution. The deeper the problem has been discussed, the better the problem is understood. If the problem is not clear, use the 5 whys.
  - NEVER search for solutions, since you'll fall in the "shitty instantaneous ideas" syndrome.
  - there is another syndrome, which is that people tend to defend their ideas even though they are bad. Most of good ideas are a mix of different ideas.
  - as long as you have time, continue taking the most voted problems

I call this process: the Reality Check.
It's necessary to detect what can be improved, and most of all to challenge the existing product/process.

Now, the tricky second part:
  - at the end of the meeting, ask people to propose solutions on a wiki page, or on a wall with post-its

Generally, finding good solutions requires to take a break, and cannot be found in a short amount of time (in my case, I find my best ideas after a good night sleep).
Ideas can be iteratively improved, so a wiki is the perfect way to do that.
Once a good idea has been found, reward all the participants, for example invite them for a lunch.
Some people like competition, so you may use a chart about the most creative guys, and reward them at the end of the year.

If you need more ideas, just contact me, and I'll provide you some other tricks, like ASIT methodology.

Re:Agile / Scrum method vs Brainstorming

[Taco Cowboy]

Many, many thanks for the most informative post !

It'll take me sometime to digest what you've written and we'll get in touch later

Thanks again !!

Re:Agile / Scrum method vs Brainstorming

[eulernet]

You are welcome.
I'll give you another trick, inspired by TRIZ and ASIT:

if you have no idea how to improve your product, just describe all its functionalities graphically (it's similar to Design Thinking, which is a great way to solve problems for visual people).
After that your product has been drawn on a board, just remove each part one after another (and replace the removed part after).
Ask yourself:
- what is the value of my product without this part ?
- what could I use to replace this part ?

This process is called Extracting, and is essential when improving systems, it helps to challenge your own conceptions.

ASIT's creator, Roni Horowitz, proposes the following exercise:
suppose that you have a TV without image.
What could you do with it ?
What is its value ?

Architecture

[TheLink]

So how does this sort of stuff work in the architecture field?

e.g. say you have a team of people designing a new large building with an innovative design.

Re:Architecture

[Anonymous Coward]

The fundamental idea of Agile is that you do things incrementally, see how they work and then fix the problems afterwards. This is based on the (definitely somewhat valid) claim that it's easy to change software after the fact but difficult to know in advance what the best solution to a problem is. Moreover, the idea is that you can properly test a piece of software to see that it does more or less the right thing before using it. Basically it's saying that the hardest part of software is the design and it may be worth trying different ones and throwing them away as they fail in order to be sure that the design is sound.

Architecture is different. It's very difficult to change a building after you have poured the concrete. If you try building a concrete building without incuding steel to see if it works and then it falls down with people in it then that will not be acceptable.

In other words; the consequence of applying true agile methodologies to Architecture is likely to be well deserved jail time.

Re:Architecture

[K. S. Kyosuke]

So how does this sort of stuff work in the architecture field? e.g. say you have a team of people designing a new large building with an innovative design.

It works pretty well, thank you.

Re:Architecture

[frank_adrian314159]

Agile can work very well in architecture - it simply has to be private, small-scale architecture, on the scale of a house or on a neighborhood layout level. Check out the work of Christopher Alexander (who invented the concept of the pattern language, from which software patterns were derived). His early work emphasized organic growth of architecture over large-scale planning, use of local materials, and energy-efficient designs.

What if...

[MonoSynth]

What if Agile is better suited for other tasks than software development? I think Agile is an elegant way of approaching some kinds of creativity, but it just doesn't seem to work for most aspects of software-development.

Making radio shows is more of an iterative kind of creativity with lots of loosely-coupled ingredients where throwing away an item and replacing it with another won't destroy the whole format, so you can start off with a format, broadcast it, and add/remove items as you go.

Software is completely different. You create it once and after the first release you have to support it for eternity. Every new addition adds another layer of complexity, you can't just remove a feature without breaking other things or add a feature without duplicating functionality. For every iteration you'll need an overview and a deep knowledge of the whole system.

Re:What if...

[RD_Sid]

I think Agile is more of a "getting things done" sort of a methodology. It doesn't care what you're developing or how stable the thing will be. It just forces you to create and manage something which can't be created through a static set of rules. For e.g. you don't need Agile to assemble a car, it's all taken care of by machines. But you can apply Agile to the unprectictable areas, be it S/w, management, research etc.

Re:What if...

[under_score]

I teach and coach teams on how to adopt Agile for their work. I have a computer science degree and worked as a software developer and architect for several years... using Agile methods. I no longer do lots of hands on work, but I mention all this to help you understand that what I am saying is based on experience, not theory.

Scrum is one Agile method. It is good for creating high-performance product development teams. Unfortunately, it cannot succeed without some additional things: you _must_ also use the Agile Engineering practices from Extreme Programming: refactoring, test driven development, pair programming, continuous integration, user stories, architectural spikes, agile modelling, and acceptance test driven development. These are the practices that directly address your concerns about the problem with changing an existing software system.

Probably the most important, and the least understood is refactoring. Refactoring is defined, in agile methods, as changing the design of the system without changing its function. There are different types of refactoring: UI refactoring (e.g. radio buttons to drop-down selection list), code refactoring (e.g. pulling a method from a subclass into its parent class) and database refactoring (e.g. splitting a table into two or more tables along columns). Refactoring can be done on its own, but it is much better to do it with the other practices, particularly TDD and Pair Programming. Refactoring is essential for maintaining internal quality of a system in the face of constant change (which normally results in ever-increasing amounts of cruft and technical debt).

In my practice with software teams, I no longer think of developers as professional if they fail to use these practices. They are hackers in the most pejorative sense of the word. Think of a surgeon. If a surgeon fails to wash her hands before operating, she is failing as a professional. These agile engineering practices are like surgeons washing their hands.

Re:What if...

[under_score]

pair programming

Oh no, are we still harping on about this? I though it had died 10 years ago.

Paired programming: one compenent programmer carries the lazy one. Why should I have to explain every line of code because he is too lazy or thick to read it himself?

In the end the competent guys went off and coded the app on their own in a quarter of the time.

Actually, I also recommend this to most organizations: if you have 100 people working on a system, you can probably find the top 10, get them to work together and let the other 90 surf the web... and you will get a better product faster.

Still, most organizations don't have this luxury (for whatever reason, good or bad) and so they need a system and tools to help with the problem of poor performers. A good agile environment forces (yes forces) efficient cross training. If you are a top individual contributor, maybe you resent this. That, in my opinion, is also unprofessional.

Re:What if...

[Instine]

I no longer consider any manager to be 'professional' if they get so dogmatic and process obsessed that they underline the word 'must' before asserting the need for a given practice or methodology. What you describe here is a soulless, creativity sapping mess of ass covering. It will not make better software, but simply reduce the attack surface of you in the boardroom/excecs' meetings (I've been dev, lead dev, Arch, Lead Arch of 30 devs, CTO and owner - if that makes any difference). It helps justify larger teams. It helps eat precious time. It helps track nonsense constructs, such as 'velocity' (please - as a physicist this term misuse makes me want to commit violence - what are the units of a scrum's velocity exactly?). It helps do many things, but not make better software more productively.

Re:What if...

[fatphil]

Having had the misfortune of witnessing many different methodologies (a word that I hate - it's a nonsense word when used in that context, it is not a study of methods, it is merely a method), and a wide range of teams with varying experience, I think that Agile does have its uses. It's good for making sure that sloppy non-self-motivated people keep making progress. There's nothing software-engineering-specific in that, it would work just as well in a many contexts (such as producing a textbook with many authors, or even creating an album if you're a band like 60s/70s Pink Floyd). Of course, whilst the micromanagement would be annoying, more than objecting to that I'd rather not be working with such a team of sloppy and non-self-motivated people.

Re:What if...

[bertok]

They are hackers in the most pejorative sense of the word. Think of a surgeon. If a surgeon fails to wash her hands before operating, she is failing as a professional. These agile engineering practices are like surgeons washing their hands.

Washing of hands is based on the scientific germ theory, which is backed by mountains of evidence.

Every buzzword you just listed is a mere whim. A preference. A particular style that works for some teams, and not others, mostly for mysterious and unknown reasons.

What you just said basically amounts to some priest admonishing a fellow man of the cloth for not properly anointing the bust of Christ with scented oil. He's doing it wrong, doesn't he know?

I've seen these software development fads come and go, and without exception they fail at being scientific, they fail at consistently achieving their goals, and eventually get replaced with a shiny new method which is just as un-scientific.

Okay, enough waffle, let me give you a concrete example: test-driven development. Sure, it sounds great. You write tests, they pass, and if you break something then the test will tell you. Except that it's not so simple:

- You can't predict unexpected problems.
- Expected problems aren't really problems, are they?
- Tests can give a false sense of security by reporting 100% success despite the prolific presence of bugs not tested for. This can result in major scheduling issues when the software fails unexpectedly.
- It is impossible to write quick & simple tests for a HUGE range of things one would most want to test for: ACID compliance, memory leaking, safe multi-threading, security, etc... Some of these things just have to be done exactly right the first time, like a maths proof.
- Tests take time to write. In many cases, doubling the amount of code written.
- Changes to the code like refactoring often require changes to the tests, potentially doubling the cost of maintenance.
- Tests are usually written in the same language as the code being tested. Gotchas with the language like Javascript's and PHP's "==" vs "===" are likely to be repeated in the tests too, leading to false results.
- Someone who doesn't know how to write code correctly probably won't know what tests to write to really put their code through its paces. Programmers who do know how to write code correctly, probably won't benefit quite so much from testing.
- Tests are most popular with developers using languages with weak built-in protections, like weakly-typed or dynamic languages. Where's the study comparing the relative merits of dynamic languages plus tests vs using a statically-typed language on its own? Nowhere.
- If a test fails for the wrong reason (badly written, test engine issue, etc...), then time may be wasted fixing a bug where none really exists.

That's just off the top of my head.

Now don't get me wrong, I'm not saying that test-driven development is bad -- far from it -- but what I'm saying is that it's not always the best approach, and NOBODY knows when it is or isn't the best approach. Neither you, nor anyone else, can give me a convincing argument of when it is right and when it isn't, because there is no evidence either way. There is no "theory" of project management that can be applied, like germ theory. There's just guesswork, and rules of thumb, and blind application of a rule that worked for one project to another where it may not actually apply. All I ever hear are anecdotes: "Oh, TDD worked great for my PHP website written by beginner programmers, you'd be an idiot not to use it". Meanwhile, I'm a veteran, write code in a safe and statically-typed language, and rarely if ever find bugs in my code after it ships. When I do, it's something obscure that tests would not find, like the incorrect use of case-insensitive collation in a SQL sort statement.

Same thing goes for refactoring, or pair programming, etc...

I've tried both. I like refactoring, but again, i

Re:What if...

[todrules]

So true. The biggest proponents of Scrum/Agile are the "instructors." It's just propaganda to make you think that Scrum really works. While it might look great to the MBAs and other execs, it just doesn't work in the real world, at least at any company that has a budget and wants to actually make a profit. I'm sure it would work beautifully on side projects, non-profit projects, etc... where you're not concerned about money, though.

Oh yeah, and a case in point on the GP's propaganda:

I no longer think of developers as professional if they fail to use these practices.

So, he resorts to telling programmers, basically, that they're not professional if they don't use Scrum. Trying to appeal to that person's guilt and shame. Sorry, but I don't fall for these "shame" tactics. Just a tactless, tasteless ploy to try and lure people to Scrum.

Re:What if...

[Joey Vegetables]

I had a lot of trouble learning to do TDD well because for a long time I confused it with unit testing. They are related, and complementary, but separate disciplines, and it is best not to confuse them. The way I approach TDD right now is to try to gather testable (though not necessarily complete) requirements, write tests to verify them, design smaller components that will implement these requirements, write the tests and then implementation, and continue to divide the problem into smaller pieces until the implementation is complete. The unit tests validate the functioning of the individual components, but they do not drive the overall design; functionality tests do. The functionality tests tell you if you've broken something in a refactor or redesign or in whatever other way. The unit tests, ideally, help you to find the broken piece(s) quickly. Both kinds of tests help to ensure that your architecture and design is testable, and thus verifiable, and thus far less likely to break. Mocking can be used to simulate the behavior of other systems outside of one's control (e.g., databases, legacy systems, etc.), including error conditions these systems may encounter or errors connecting to them, so that the software within one's control can respond in an intelligent manner if things beyond it go wrong. I've been accused of being a TDD skeptic, and, indeed, I don't think it's appropriate everywhere, but it is broadly useful in many areas of software development and it helps to teach and enforce other good architecture, design and coding practices. Unit testing is great but it's only one component of TDD, and, often, much less important than comprehensive behavior and integration testing. It should still be done where possible, because seemingly small changes and refactorings can cause subtle breakage that the unit tests can catch quickly, especially if they are automated as they should be, possibly before an integration build is even attempted.

Re:What if...

[JDG1980]

A good agile environment forces (yes forces) efficient cross training. If you are a top individual contributor, maybe you resent this. That, in my opinion, is also unprofessional.

So the top individual contributors are tasked with two separate jobs: programming and training. I trust they get paid a lot more for this? Otherwise, they have every reason to "resent" this: They're being asked to carry everyone else, to do double duty, and getting nothing in return.

Re:What if...

[JDG1980]

Actually, top individual contributors are tasked with two separate part-time jobs. So, no, they don't get paid a lot more for this.

For some people, being tasked with two separate jobs is more stressful than being able to focus on one specific task. This is true even if the total hours are the same (and I'm rather skeptical that this is actually the case in practice).

Furthermore, the skills needed to be a good programmer are different from the skills needed to be a good trainer. Many of the best programmers aren't exactly very social types.

What you're doing is driving away and/or burning out your best programmers by making them do a job they probably don't want to do, aren't very good at, and essentially punishes them for their talent.

OpenAgile for non-software

[under_score]

FWIW, OpenAgile was designed for non-software agile use including management, sales, marketing, operations, etc. and there are some very interesting uses out there. I'm on the board of directors for the OpenAgile Center for Learning so I'm a bit biased, but I strongly recommend this approach to any organization or team that wishes to improve effectiveness. I think it's very exciting that Agile is spreading beyond software, but there are also many challenges. In particular, Scrum is really best for new product development and many practices are not applicable outside that environment.

Oh, great...

[__Paul__]

...more Cult of Management techniques being inflicted on the world.

Scrum is not originally a software methodology

[jools33]

I first heard of Scrum from my wifes hospital ward - where they were using the technique to manage the activities of their staff. This made me curious as to its origins and it turns out it was first and foremost a product development methodology. So its not that Scrum is spreading from its software origins - it never originated in software in the first place.

Re:Scrum is not originally a software methodology

[Xest]

Yes, the NHS in the UK has used it for a number of projects for many years.

Creativity Killer

[mnt]

Working in the game industry i found that creativity was heavily hampered by scrum, even after doing several adjustments to the process.

SCRUM stands for SCRrewed Up Management

[nickmdf]

... especially when non-technical people are in charge of this process. One can't map short term deliverables properly to a creative process.

My biggest problem...

[pauljlucas]

... with Agile is the "Daily Stand-up." I don't care what anybody else is doing on a daily basis. Actually, for most people, I don't care what they're doing -- ever. All I care about is that people I work with (1) answer e-mail in a timely manner and (2) if I depend on their work, that they'll have it done when they say they will. (If they're going to miss a dead-line, only then do they need to bring it to my attention.)

What some other set of people whose work I don't depend on is doing in no way helps me do my job. I'm paid to do my job, not concern myself with everybody else's job.

Re:Transfer of knowledge

[TheSkepticalOptimist]

I hate that expression with a passion.

What it means is that instead of a team of highly focused and specialized developers that are all skilled in specific areas of the project, you end up with jacks-of-all-trades that are mediocre in all areas.

"Transferring" knowledge, doing it properly, constitutes a significant amount of overhead simply to have another developer take over a task and do it poorly.

If you have already invested time and energy to investigate and work on a specific problem or feature in a product, why on earth should you hand it over to another developer during the next sprint? The only reason given is that if one developer becomes ill, or sick, goes on vacation, leaves the company, or dies, then the knowledge is contained by the team and so the project can continue on unencumbered by your demise. I mean, what a cold hearted process to assume you mean nothing to the company that your absence will not affect the outcome of a quality product? How is that motivational?

It should not take long for a good developer to ramp up in any given area of a product and I would claim it takes less time for a developer to become "specialized" in an area of a product, and results in better code, then the time wasted trying to spread and maintain the knowledge across the team during development on the off chance that someone might die unexpectedly.

I am not saying all developers should work in a fortress of solitude, never exposing their work until they are complete. Periodic updates (NOT DAILY) can go a long way to ensure that developers keep on track AND other developers are aware of what is going on and even become interested in other areas of the product, but in general the concept of Transfer of Knowledge is bullshit.

I hate daily stand ups because the reality is that most people really don't care what other developers are doing. Often these stand ups run long and get mired in details between specific people and all I keep thinking is that its taking time away from me completing my tasks, so its not a motivational tool, and what knowledge I am gaining will not help me in any way if I should need to take over an area of development. And believe me, I have taken over from other people in different areas of the code, and never found it detrimental to not be intimately aware of that area right away.

There is a quantity of overhead associated with Agile, and where I worked using it, it was upwards to 30 -40% of my time was invested in process NOT development.

So maybe Agile is better for non-software related projects that have generally less requirement for focus and skill in specific areas of a project, but for software Transfer of Knowledge is another myth and lie about Agile that needs to be squashed as a software development process.

I get it...

[gosand]

I understand what he meant by that, and I don't think it is as bad as it reads. I think he just meant that those things are the key components, they are just part of the process, if you want to do it right.

I spent the last 3 years managing a testing teams on an Agile project. And it was at a very very large company that is most certainly concerned with money. What I saw Agile do was amazing... and yet, we had to compromise it somewhat. We didn't do pair programming. Our TDD wasn't as good as it could have been. We faced challenges with it, but we had contraints that we had to deal with, especially after our first release. But I will say that the quality and volume of what we put out was far and above anything else around us.

IMO, Agile isn't for every software project, and there are some that I think it simply just wouldn't work for... but it is very very useful when it fits. BUT - you have to really adopt it. It really is a team effort, and if it's not, or if you ignore or sabotage some of the key components of it, you will fail. To one of your points, calling velocity a "nonsense construct" tells me immediately that you've never done Agile, at least not successfully. I'll take a moment to explain....

You actually aren't far off... it kind of is a nonsense construct. What?! Yeah. It is a unit of effort. Here is how we did it. Each story is written to describe the functionality desired. It should follow good story principles of INVEST (look it up). Once you have that, the development and test team review it, and quickly put an estimate on it. We used a point scale. 1,2,4,8,16,32. You have to pick one of those values, there is no 12 for example. This forces a decision on it. If you had a 1 to 10 scale, there's really no differentiating factor between a 7 and an 8. You get the idea.

So what we did was the dev team came up with their estimate for each story, and test did the same. Then the story was assigned the larger of the two numbers. Since you have to dev and test it during the iteration, larger number wins. Then as a team you commit to X number of points for an iteration (we used 2 weeks). At the end of the iteration, whatever stories are accepted as delivered are counted up, and that is your velocity for the iteration. The NEXT iteration, you can only commit to doing that number or less. You can certainly deliver more, but you can only commit up to that. Over time, your velocity will fluctuate, and then STABILIZE. That is the point where you know as a team how many points you can deliver in a 2 week period, in theory indefinitely. Now some people want to know how accurate your estimates were - i.e. we said this story was 16 points.. how many was it actually? Don't do that. That is exactly why we didn't use hours. It's irrelevant. What is relevant is how many points you delivered. By making the points a non-quantifiable number you can't do that. It let's you focus on what is really important, and that is determining the team's sustainable velocity.

It is a foreign concept. But we did it for 3 years. Actually the project is still going, I just left and took on a different positon in the company. Yeah, we had challenges, funding and otherwise, but we were able to deal with them. We had to cut about 1/2 our team at one point, and our velocity suffered. But we got to the point where when we said we could deliver something by a certain date, we could. I really don't see that very often, and it wasn't the case with all of the projects around us struggling with Waterfall. Again, it's not a panacea, but it can work and to discount something just because you don't understand it or have never actually done it is foolish.

Re:Me no get it.

[frank_adrian314159]

Good software engineers will make things work be it with XP or Waterfall. Bad software engineers will fail regardless of methodologies.

This. A thousand times this.

I often feel like writing a watershed paper called "Software Methodology Considered Harmful". In the end, the people involved on the software team and how well they work together are the most important determining factor as to the success and/or failure of a software project. Process in organizations often exists as a way of trying to paper over the problems of not wanting to pay for decent practitioners or making a corporate culture that's so awful that no one decent actually wants to work for them. If you like methodologies, you generally like looking at people as functional units rather than people - and that's a problem in itself.