Slashdot Mirror


The State of Agile Software in 2018 (martinfowler.com)

On the surface, the world of agile software development is bright, since it is now mainstream. But the reality is troubling, because much of what is done is faux-agile, disregarding agile's values and principles, writes programmer Martin Fowler. The three main challenges we should focus on are: fighting the Agile Industrial Complex and its habit of imposing process upon teams, raising the importance of technical excellence, and organizing our teams around products (rather than projects), he added. An anonymous reader shares his post: Now agile is everywhere, it's popular, but there's been an important shift. It was summed up quite nicely by a colleague of mine who said, "In the old days when we talked about doing agile, there was always this pushback right from the beginning from a client, and that would bring out some important conversations that we would have. Now, they say, 'Oh, yeah, we're doing agile already,' but you go in there and you suddenly find there's some very big differences to what we expect to be doing. As ThoughtWorks, we like to think we're very deeply steeped in agile notions, and yet we're going to a company that says, "Yeah, we're doing agile, it's no problem," and we find a very different world to what we expect.

Our challenge at the moment isn't making agile a thing that people want to do, it's dealing with what I call faux-agile: agile that's just the name, but none of the practices and values in place. Ron Jeffries often refers to it as "Dark Agile," or specifically "Dark Scrum." This is actually even worse than just pretending to do agile, it's actively using the name "agile" against the basic principles of what we were trying to do, when we talked about doing this kind of work in the late 90s and at Snowbird. So that's our current battle. It's not about getting agile respectable enough to have a crowd like this come to a conference like this, it's realizing that a lot of what people are doing and calling agile, just isn't. We have to recognize that and fight against it because some people have said, "Oh, we're going to 'post-agile,' we've got to come up with some new word," - but that doesn't help the fundamental problem. It's the values and principles that count and we have to address and keep pushing those forwards and we might as well use the same label, but we've got to let people know what it really stands for.

34 of 315 comments (clear)

  1. A new pile. by Anonymous Coward · · Score: 5, Insightful

    Seems like with each new manager and team lead they change things. From codewords to project phases to agile. The not invented here mentality is the problem

    1. Re:A new pile. by Darinbob · · Score: 4, Insightful

      When I see things like "because much of what is done is faux-agile, disregarding agile's values and principles", it sounds like the church arguing against reformers and splitters. Agile started life sounding like a religion, and over time that has only intensified. Sure some people do it badly, but Agile is not a magic bullet that solves everyone's problems all the time as long as it's done perfectly.

      The problem is the adherents insist that there are only two models - agile or waterfall, nothing else. And then they try to put the two in conflict with each other. But waterfall is mandatory in so many environments. If you must tell your customers what features will ship on a certain date in the future, then you have to have project planning in advance of building or implementing, and that's where waterfall works. Agile is good for web sites where you're just tweaking small things and doing "continuous delivery". But it becomes clumsy with complex systems, and where you need specialists and not generalists.

      Sure, I have seen some advocate that you just put the design phase into agile, but you're still stuck with the silly scrum model where you have to have results every two weeks even if a design task takes two months or more.

      If a company is failing because of waterfall, they are almost certain to be failing with Agile as well.

    2. Re: A new pile. by HornWumpus · · Score: 3, Insightful

      Agile is a manifesto full of truisms. Nothing to argue with, but PHBs are not the target audience and DON'T GET IT. Only take the parts they like, such as (para via PHB brain) 'specs are useless'.

      Scrum is just a bad idea and is often called 'Agile'.

      --
      John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
    3. Re:A new pile. by HornWumpus · · Score: 2

      You disagree with the authors of the Agile manifesto.

      I bet you have _never_ shipped code.

      --
      John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
    4. Re: A new pile. by Anonymous Coward · · Score: 2, Interesting

      Agile is judged by its own metrics. Of course it is done "correctly". Whether they laugh at others' implementation of Agile is irrelevant.

      Yes, I have read the manifesto. Unfortunately, extracting my commentary on it from my plain-text document will not look good, and I wrote it a long time ago without fully reviewing what I wrote, but I don't care, since nobody will read it anyway.

      Here's the Agile Manefesto, with commentary by me:

      > We are uncovering better ways of developing software by doing it and
      > helping others do it. Through this work we have come to value:

      > Individuals and interactions over processes and tools

      Processes and tools empower individuals to get their jobs done
      without uncertainty and with great efficiency. Also, many agile
      shops insist on using certain methodoligies that require certain
      tools (e.g. Rational Rose, Microsoft Visual Studio), so this is
      clearly not a shared value.

      Valuing individuals is good. The proper way to do this is to
      adjust the processes and tools to meet the needs of the
      individuals, rather than dictacting what sounds best to
      everyone without feedback.

      Wikipedia provides elaborations:

      > ... self-organization and motivation are important, as are
      > interactions like co-location and pair programming.

      What does this even mean? Are you saying non-"agile" methods
      do not encourage self-organization and motivation? Motivation
      in particular is completely dependent on the individual.
      Agile methods do not motivate me, for example. Self-organization
      sounds a lot like "we have no clue how to manage this, so we'll
      rely on you to, even though you probably have no clue, either".
      ISO 9001 compliance, which was popular for a time, requires
      that every task an indivdual performs be written down. This
      prevents management from inventing new tasks outside of the
      employees scope, helps orient new employees, and when it
      comes time for status reporting and performance reviews, it
      gives the employee a reference against which to evaluate.
      Self-organization can be part of that, by not forcing
      procedures that dictate organization, but I don't see a lot
      of that coming from the "agile" programming folks. Instead,
      a new set of rules is forced on people, claiming that they
      are somehow superior.

      Co-location is fine, and sa

    5. Re:A new pile. by gtall · · Score: 2

      These are not the problems with Agile. The main problem with Agile is that it doesn't scale and when large projects are attempted, it tends to produce dirty snowballs. Those snowballs become too large for any agile project and then the size makes them essentially unguided. They crash, burn, and then the team realizes they should have done a real design first, and then filled it in making accommodations as necessary.

      In my own experience, Agile reduces engineers to cogs whose only mission is to add some extra pieces of chewing gum to an unruly wad.

    6. Re:A new pile. by laird · · Score: 5, Interesting

      When done right, it lets managers get out of the way of developers, because it replaces the boss being in charge with the team being self-led, with the "boss" clearing the path and facilitating when things get stuck, and you empower the product owners with real authority to make decisions, empower teams to do real estimation, etc. When I've seen agile fail, it's because the management couldn't give up control to the team, so they basically ran waterfall development using agile terminology. That is, when you make fixed scope and fixed time commitments, because the boss (or product owner) says so, you're waterfall - it doesn't matter if you use story points and sprints to implement your waterfall product, because you've missed the fundamental point of agile.

      Call it Scrum, XP, Kanban, etc. - it's all about measuring and optimizing a process by empowering the people who do the work to improve how they do the work. Everything else is just a means to that end. The techniques work, which is why companies who do them tend to outperform the companies who don't. But a bad team with bad management who uses the form of agile without the substance is still screwed.

    7. Re:A new pile. by PPH · · Score: 3, Funny

      At some point it becomes necessary to behead all the architects and begin construction.

      -- Abi-Bar-Shim (Project Mgr. - Great Pyramid)

      --
      Have gnu, will travel.
    8. Re: A new pile. by dskoll · · Score: 2

      I founded a successful software company that I sold after 19 years, so I have shipped code. Agile is a satire written by drunken programmers on a lark that accidentally got taken seriously.

    9. Re:A new pile. by zmooc · · Score: 3, Insightful

      I think the main problem is something else: proper agile requires buy-in from just about any organisational layer above the agile team. Usually the time, attention-spam and understanding to achieve this is almost impossible to achieve. Agile requires trust while company hierarchies are usually organized around distrust; agile requires doing things together while traditional company hierarchies demand clear separation of responsibilities. Agile requires thinking about what to do next while companies revolve around yearly planning. Agile requires accepting uncertainty inherent to software development while companies revolve around containing such uncertainties, usually in a superficial way. Those kind of things make achieving true agile in a commercial organization next to impossible.

      --
      0x or or snor perron?!
    10. Re: A new pile. by ichimunki · · Score: 3, Insightful

      Absolutely brilliant response to the Agile Manifesto. As someone who used to be an Agile fanatic, I've never once seen it work the way it's supposed to. The fact that it's so prone to abuse is just further evidence to me that it's a flawed ideal. I especially dislike the "welcome changes in requirements" part... WTF. No. It's one thing to *clarify* requirements as you go along, it's another thing to scrap them completely. And even the most rigid waterfall system always had a robust system for change control, updating requirements, and responding to new information/understanding. The best thing I can say about Agile or DevOps is that it's got management types actually taking things like automated deploys and source control seriously. But as you say, that's "tools and processes" over individuals.

      --
      I do not have a signature
  2. No True Scotsman by Anonymous Coward · · Score: 4, Insightful

    Just the latest management fad in a long line of management fads, watch as we wave away criticism of our touted silver bullet.

    1. Re:No True Scotsman by laird · · Score: 4, Insightful

      In reality, lean production techniques are why the Japanese car companies outperformed US car companies for several decades, until the US car companies copied their management techniques and (somewhat) caught up. Very real stuff, worth $billions.

      If MBA's don't understand lean manufacturing and copy the form without the substance, the problem is the MBA's, not lean manufacturing.

    2. Re:No True Scotsman by PolygamousRanchKid+ · · Score: 3, Funny

      Just the latest management fad in a long line of management fads,

      Agile will soon be enhanced to create Freestyle Recursive Agile Software Development, known by its short form, Fragile.

      Similarly, the imprecision of integer DevOps will be improved by implementing floating point operations, to gift us DevFlops.

      watch as we wave away criticism of our touted silver bullet.

      watch as we wave away criticism of our touted Bitcoin bullet!

      --
      Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
    3. Re:No True Scotsman by DNS-and-BIND · · Score: 2

      This is a partial truth. The real truth is that America gave Japan a gigantic bribe in the form of totally unfair (to Americans) trade deals. Japan largely got a pass from the United States when it used protectionism and state subsidies to build its industries after World War II. The same protectionism that people decry today apparently worked spectacularly well when anyone but Americans did it.

      Japan was free to dump its autos on America and put our people out of work, but American imports to Japan were tightly restricted. This bribe was to stay on the free world side in the Cold War. You'd think they wouldn't need a bribe, but even today Japan has an active Communist Party, and even back when Russia was 1000 times the threat it is today, Japanese and Western leftists sought to join their nations to the Soviet Union as Soviet republics. More info here.

      Follow me back to 1946. Europe is ruined, China is still a backwater. The USA puts together an alliance that allows everyone within it to trade goods into the US without significant trade barriers. This allows for Europe to export their way back to prosperity after WWII. It gives them room to grow their economies beyond what their somewhat depleted populations will allow. In addition to allowing the free flow of goods, the USA becomes the security guarantor of the free world by using our navy to police the world's oceans and enable low risk (and hence low cost) global trade.

      But there was a condition. In order to deal with the US, you had to be on our side in helping to combat and contain the Soviet Union. Essentially the US traded some of its economic/manufacturing capacity for increased security and strategic assets to assist in the cold war. Only one problem: We won. The Soviet flag went down in 1991, and we didn't really make any changes to the world order.

      They only beat us on trade the past few decades because we let them. We're not letting them any more. Globalization is over. Free trade is over. The rest of the world riding on America's back is over. Atlas is shrugging.

      --
      Shutting down free speech with violence isn't fighting fascism. It IS fascism!
  3. YMMV by ZiakII · · Score: 4, Insightful

    YMMV I still find it better then the old system and waterfall. My biggest problem you would get stuck in a 2-3 year death march when you know the project is going terrible due to the sunken cost fallacy. Now at least with agile the most you waste is 2 months.

    1. Re:YMMV by Darinbob · · Score: 2

      If you have a strong team dedicated to making a process work, any process will work.

    2. Re:YMMV by angel'o'sphere · · Score: 2

      areth, with all respect, you are wrong.

      The easiest example are external resources that have longer delivery time.
      Then adjust your sprint length or mock them away till they are ready.

      But also internally, if you have to e.g. run a 4 week test for regulatory reasons
      Then adjust your sprint length. The original recommendation for sprint length in Scrum and in XP is 6 weeks to 12 weeks!!!!
      Not one week, not two weeks!

      or prime a machine learning algorithm with real-time data that happens in a particular time frame.
      That is usually outside of the sprint, completely irrelevant for ordinary software development.

      The agile answer is then "sorry, we cannot commit to this",
      That is nonsense. The agile answer is: adapt!

      --
      Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
  4. Handy cross-reference for job seekers by 93+Escort+Wagon · · Score: 4, Funny

    “We are an Agile shop”: Your expected standard work week will be 80 hours

    “We are a Post-Agile shop”: Your expected standard work week will be 95 hours

    “We believe in work-life balance”: You’ll be required to wear a beeper whenever you’re not in the office

    --
    #DeleteChrome
    1. Re:Handy cross-reference for job seekers by halivar · · Score: 3, Insightful

      My old scrum team of 10 years had a policy of only considering half the available work hours when placing hours on tasks. More often than not, it was spot on. After 6 months our velocity projections were spot on, every sprint. After that I worked on a "scrum" team that packed everyone's sprint board with 40 hours/week of work.... but sprint "planning" ate one of those days.

      Underestimating work availability and overestimating needed work was pejoratively called "sandbagging" on that team, and yet the former delivered project on the estimated date, and the latter never did, even once.

  5. Re:The trouble with Agile... by arth1 · · Score: 2

    *Solution: Write the plan on True Scotsman brand paper.

    Yeah, the "you're not doing agile right" mantra as the explanation for failure, and repeated in every agile "state of the union address" is becoming tiresome. You'd think that if there were a right way that made the companies and customers more happy, we'd all be drifting that way and things would get better and better, simply because those doing it right would outcompete the others. This is not what we observe happening.

  6. Re:Agile by halivar · · Score: 2

    Is "agile" the reason why paying customers are now unpaid beta testers for the vast majority of crapware foisted on us by software houses?

    If they won't pay for proper QA under agile, what makes you think they'll do it under waterfall?

  7. The only thing wrong with Agile is... by Assmasher · · Score: 3, Funny

    ...its pervasiveness.

    It's a valuable tool IN A TOOLBOX.

    If you're NOT an ISV, and you make software solutions for individual customers (which is what the vast majority of software developers find themselves doing) - then Agile is probably the right way for your company to go.

    If you ARE an ISV (and Independent Software Vendor makes products, unlike the folks above who work on PROJECTS) then use Agile at your peril.

    ISVs that use agile are those that tend to have "Product Managers" who are not actually Product Managers, but are instead "Project Managers" with a penchant for making products up out of virtually thin air ("Oh, a customer wants this, so this must be part of the 'product.')

    ISVs that know what they are doing start with Product Managers - because a Product Manager has DOMAIN EXPERTISE that means they actually know what the market wants - not just a single customer. The problem for wanna-be ISVs is that real Product Managers are expensive and difficult to find (but they're out there) due to the Product Management market being swamped with Project Managers calling themselves Product Managers.

    If your company has a technology or product related vision - you should not use Agile.
    If your company has a market dominance or growth related vision, and you make software tailored to a specific customer's needs - you should probably use Agile.

    --
    Loading...
  8. The real problem is... by QuietLagoon · · Score: 2

    ... Our challenge at the moment isn't making agile a thing that people want to do, it's dealing with what I call faux-agile ...

    ... focusing far too much on what to call the process, and focusing not enough upon what problems the client needs to solve. Perhaps the move to "faux-agile" (as you call it) is really more of a move towards actually solving the problems the customers have, and less of a desire to make sure the process to achieve those solutions has the correct name.

  9. Re:Agile is like communism... by ShanghaiBill · · Score: 2

    ...Sounds good on paper, disastrous in practice.

    Another similarity is that when it fails, the proponents will say that is only because you weren't doing it right.

    My opinion:
    Good agile practices: TDD, FBF
    Ok in moderation: Standups, sprints, stories
    Bad: Everything else

    TDD = Test Driven Development
    FBF = Fix Bugs First

  10. Re:Agile is like communism... by ilguido · · Score: 2

    My opinion: Good agile practices: TDD, FBF Ok in moderation: Standups, sprints, stories Bad: Everything else

    TDD = Test Driven Development FBF = Fix Bugs First

    The fact is you don't need agile to do TDD and FBF. Every _good_ software developer would do that. On the other hand, if you have bad developers, no matter the method, you're going to get mediocre software.

  11. How customers and managers see "agile" by ffkom · · Score: 4, Interesting

    If I learned one thing from experience with "agile development", it is how totally different from the IT-perspective it is seen from non-IT people, especially customers and managers:

    Customers: "Agile is great, because now we can change requirements whenever we like, and don't even need to think of what we really need in the beginning. Ah, and by the way: We don't have time to sit in your sprint-planning meetings. We just expect you to deliver."

    Managers: "Agile is great, because now none of those IT wisenheimers can obstruct our great vision by telling us how time and effort consuming our projects will be before implementation starts. Once it becomes too obvious how much more work it will take to turn the software into something useful, it's too late to cancel the project as a whole, and we'll just demand long hours from the programmers."

  12. Agile ain't by SpaghettiPattern · · Score: 2
    • A way to make mediocre programmers any better. An excellent programmer can effectively work for 10 mediocre ones. And if the mediocre ones aren't well organized, then the number may be factors higher.
    • A guarantee that any project will succeed and produce viable stuff. The success of any project almost always relies on the best coder. Nothing more than the Pareto distribution.
    • A protection against taking the will for the deed. Bad managers make the organization "feel good" about itself without actually delivering anything near to an acceptable level of functionality and maintainability. Ever witnessed a project being abandoned even if it really "deserved" it?
    • A guarantee that the methodology will be fully embraced. Indeed the failure of doing so will eventually be the crap excuse that nothing in the failure is your fault because the methodology wasn't fyll embraced.
    • A silver bullet. A crap manager that never delivered anything useful will usually maintain that agile working will solve all current problems. Agile will also not make mediocre managers any better.

    It's a waiting game until this fad blows over. Each fad always highlights its own successes and downplays successes achieved with anything else. And then we welcome the successor.

    --

    I hadn't the slightest objection to his spending his time planning massacres for the bourgeoisie... (P.G. Wodehouse)
  13. Re:Agile is like communism... by OneHundredAndTen · · Score: 2

    ...Sounds good on paper, disastrous in practice.

    I don't agree - it doesn't sound good on paper. It does sound like a lot of hand-waving, with very little in the way of fact and substance.

  14. Re: Agile is like communism... by ShanghaiBill · · Score: 2

    if I spent time writing tests I'd never get anything done on the timeframe demanded of me.

    You are doing it wrong. TDD saves time. Even during death marches, I write unit tests. The time spent writing tests is way less than time spent chasing bugs.

  15. Re:Agile is like communism... by ShanghaiBill · · Score: 2

    I was always taught that tests should be written by a QA developer instead of the application developer

    You are doing it wrong. Unit tests should be written by the developer, and the class/method/function and the unit test should be written simultaneously by the same person. Code test commit code test commit code test commit.

    QA people should focus on high level functionality and usability, not implementation details.

  16. Re:Agile is like communism... by iMadeGhostzilla · · Score: 2

    I used to think so about unit tests until I read "Why Most Unit Testing is Waste" by James Coplien (https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf). It caused quite a stir on Reddit and YC in 2014 when it appeared, and Coplien published a sequel (https://rbcs-us.com/documents/Segue.pdf). From time to time is rediscovered and creates the same controversy.

    I was very opposed to it in my first reading, but on subsequent readings and checking out arguments on the boards I slowly came to be in favor. Then as an experiment I ditched almost all unit tests and started only writing functional tests and I'm sure I'm seeing a better payoff. Just the other day my functional test caught a very nasty bug introduced by a coworker where the results of some key calculations were off by 2-10%. I'm sure no unit tests of mine would have caught that bug of his. So for the moment I am a believer, and Coplien gives some good empirical and logical arguments in favor of it.

    That I still do TDD about a third of the time, making a functional test before I write the code.

  17. Who is agile IRL? And how did they get that way? by sbjornda · · Score: 3, Insightful
    Athletes are agile. Ballet dancers are agile. Acrobats are agile. Martial Arts experts are agile.

    How did they get that way? Discipline and practice; practice and discipline, more practice, more discipline.

    Too often, Agile as applied to software development seems to mean the opposite of discipline, and I'm not sure where practice fits in.

    --
    .nosig

  18. Steven Lowe puts it best. by michael5727 · · Score: 2

    The clearly-titled Project Management: A Surefire Way to Kill Your Software Product by Steven Lowe hits the nail on the head. If you've ever found yourself frustrated or overwhelmed by a process—purporting to be agile or otherwise—you'll find this essay is a refreshing read.