Survey Finds 'Agile' Competency Is Rare In Organizations (sdtimes.com)

An anonymous reader writes: The 12th annual "State of Agile" report has just been released by CollabNet VersionOne, which calls it "the largest and longest-running Agile survey in the world." After surveying more than 1,400 software professionals in various roles and industries over the last four months of 2017, "Only 12% percent responded that their organizations have a high level of competency with agile practices across the organization, and only 4% report that agile practices are enabling greater adaptability to market conditions... The three most significant challenges to agile adoption and scaling are reported as organizational culture at odds with agile values (53%), general organizational resistance to change (46%), and Inadequate management support and sponsorship (42%)...

"The encouraging news is that 59% recognize that they are still maturing, indicating that they do not intend to plateau where they are." And agile adoption does appear to be growing. "25% of the respondents say that all or almost all of their teams are agile, whereas only 8% reported that in 2016."
The researchers also note "the recognized necessity of accelerating the speed of delivery of high-quality software, and the emphasis on customer satisfaction," with 71% of the survey respondents reporting that a DevOps initiative is underway or planned for the next 12 months.

Comments

Agile takes a rare group

[Anonymous Coward]

Agile has always seem to take longer than it should, never works as promised. Simple staging plans and a short meeting to discuss issues seems to work well enough in my group

Re:Agile takes a rare group

[ctilsie242]

Agile, or more specifically Scrum is pointless. When you have a daily stand-up meeting that can take six hours while the Scrum master chastises, badgers, yells, and excoriates people, one by one, for not making deliverables. During this everyone else is pointing at someone else and saying, "I'm blocked... he did it!". This isn't productivity; it is a game of kangaroo court.

Then the Scrum master tosses more crap on people's swim lanes at random (because marketing wants them done, and because they make the sales, they get what they want, without challenge), without really knowing or caring how difficult the task is. Finally the Scrum master closes the meeting with how everyone has been in a sprint for the past year, and says the sprint will continue until marketing is happy.

I do not see Agile adding any productivity whatsoever. It turns a dev team against everyone else, which may be great for management, but it creates a workplace that is at best hostile, and at worst toxic, because every day you have to go in and defend yourself against everyone in a multi-hour blamestorm. Eventually the good people leave for greener pastures.

Re:Agile takes a rare group

[fahrbot-bot]

Agile, or more specifically Scrum is pointless. When you have a daily stand-up meeting that can take six hours ...

Well... Daily scrum meetings are *suppose* to only be 15 minutes, but either (a) they aren't and are a waste of time (as you described) or (b) actually are and are a bigger waste of time. What it does ensure is that everyone is micro-managed into being at (or dialed into) that meeting every day at, like, 9:30am -- even though many (most?) companies have "flextime" -- 'cause management loves managing people.

Look. I *imagine* these meetings could be useful if you have a team of inexperienced people that need constant "guidance", but otherwise, working with experienced, responsible people, I've never run across anything that couldn't be more simply handled with emails to the appropriate people *if* something *isn't* on track.

Re:Agile takes a rare group

[ctilsie242]

In the Agile/Scrum environments I worked at, it wasn't handled by E-mail. Calendar appointments would magically appear, because the Scrum master, PM, manager, team lead, and backup team lead all had delegation authority to add meetings without approval, and these were "The Apprentice" like boardroom confrontations that lasted for hours.

I'm glad I'm away from that. My current place uses a modified waterfall model, and it works quite well, with projects getting done on time.

Re:Agile takes a rare group

[Anonymous Coward]

Very few of the proponents of "Agile" seem to have actually read and understood the Agile Manifesto.

Where I work, the introduced Scrum about 2 years ago. We were more agile before that, but our process, which was well adapted to our situation, didn't have a name since it had evolved internally (together with the customers) for about 10 years. We made a mistake not to connect a buzzword to it.

The output has dropped considerably with Scrum, much because of the planning meetings, but also because of the treatment of all the developers as "resources" instead of individuals. So instead of letting the team divide the work between them as they see optimal, everyone now has to work on all parts of the system regardless of knowledge or ability (this is a big system). So far no sprint has resulted in a working release and every sprint starts with tasks to make the previous release work. The testers are upset because they don't get _anything_ from the team until the last days of the sprint, whereas we used to have a constant flow of work to test (and subsequently release).

I guess Scrum can be an improvement if the organisation is completely non-agile from the start, but for us it was a great step backwards.

True agile development is wonderful, but management has to let go of some control for it to work properly, which is why we get "Agile" instead.

Re:Agile takes a rare group

[apoc.famine]

Welcome to the 88% that don't do agile correctly.

I've worked in a shop which did, and it was honestly amazing. The company embraced it, institutionalized it, and ensured that they had the expertise to do it right.

Daily standup was great. No more than the team and 1-3 product owners or managers in the room, 10 people, 1 minute each. What did you accomplish yesterday, what are you working on today, and any issues you're having or expecting. That was it. And the scrum master kept people from going on and on - two guys would regularly get "your minute is up" warnings, and everyone learned to make them as uncomfortable as possible or just start their 1 minute talk over top of them because standup was serious business.

What happened in there was that everyone on the team knew who was going to be touching what, and that lead to either avoiding code that was going to be touched or collaboration when two members were working on related stuff. The team leads and more experienced members could pinpoint where the newer members were getting stuck and help them over humps, or steer them away from bad coding decisions. (Outside of standup, of course.) The product owners got a thermometer on what was getting done, so they could update their roadmaps and tell the bosses, marketing, and other people what was going on. That meant the developers didn't have to waste time dealing with those people.

It also meant that sprint planning meetings were done with a broader institutional understanding of the current state of development, which tended to lead to more reasonable sprints.

Overall, it was a well-oiled machine, and it was very productive. In great part it was because the head of IT effectively used the agile framework to shield developers from institutional stupidity. When 10-15 minutes of your morning is all that management gets to waste of your time, you can get a lot done. It meant they all felt included in the development process, and had a structured way to interact with the devs rather than just calling random meetings and dragging everyone away from their work. And by having product owners closer to the work, they could answer the questions that management wanted, rather than the developers doing it.

I'd happily work in an environment like that again. What people normally describe agile to be like, however, seems to be a nightmare.

Re:Agile takes a rare group

[fahrbot-bot]

Daily standup was great. No more than the team and 1-3 product owners or managers in the room, 10 people, 1 minute each. What did you accomplish yesterday, what are you working on today, and any issues you're having or expecting. That was it. ...

What happened in there was that everyone on the team knew ...

While presumably a nice idea, it's still a waste of time. Everyone on the team doesn't *need* to know what everyone else is doing - and certainly not every freaking day. It's simply a way to ensure everyone is working like the busy little bees management wants. Team members don't -- and shouldn't -- interact with everyone else on the team; that's inefficient. Perhaps getting everyone together is appropriate for milestone events, but there are better, more flexible and productive, ways for people to interact as needed in normal situations. This becomes more true as people become more experienced -- said with 30+ years experience.

Re:Agile takes a rare group

[SimonInOz]

When I was involved in Agile, it was indeed pointless. It was in a major bank, and basically descended into institutionalised micromanagement.
It was pretty horrible, and I hated it.

There was no opportunity to spend time to actually think about things, it was all rush, rush, rush.

The is a saying "More haste, less speed", and I think it applies here.

Agile and Scrum Are Like Communism

[DatbeDank]

All sound great in theory, fall apart in practice, and there will always be someone who says, "You just didn't implement it the right way!"

Re:Agile and Scrum Are Like Communism

[Anonymous Coward]

I thought it was more of a cargo cult. Do the right motions, and the gods will reward you with deliverables.

Re:Agile and Scrum Are Like Communism

[Anonymous Coward]

The problem with implementing Agile the right way is simple: it's too hard.

Management absolutely hates facts like:

1) Each team must do their own estimates, and estimate numbers are not comparable between teams.
2) Early estimates cannot be treated as commitments, especially since they are wildly inaccurate (due to lack of key requirements).
3) Changing requirements comes with the consequence of changing the costs and delivery dates.
4) Velocity cannot be used to measure productivity.

Their hatred of these facts is so deep that they will always reach in to any well-formed agile process and break it in a misguided effort at escaping these facts.

Business Analysts fall into similar traps, either treating agile as an excuse to go hog-wild with requirements changes, or sticking with waterfall planning processes and not understanding why that doesn't just work with an agile process.

Developers fall into traps too, sometimes thinking that Agile means they will never have to push to hit a deadline again, or that they need to have a higher velocity than other teams to look good, etc.

So, in sum, Agile actually can be done right....just not by humans.
 

Re:Agile and Scrum Are Like Communism

[ctilsie242]

I have seen weeks where the entire 40 hours were all Agile/Scrum related meetings. This meant that there was no significant coding done whatsoever.

In all of my IT work, I have never understood why some managers think that calling meetings will enhance productivity, and if that doesn't work, call more meetings. I don't know if this is incompetence, or an issue with ego. Either way, it hamstrings actual productivity.

Re:Agile and Scrum Are Like Communism

[Pseudonym]

I remember when Agile was called XP. That was before the seminar and certification industry got hold of it.

XP made sense, but only by comparison with what came before it. If there's one thing I've learned after 25 years in this business, it's that fundamentalists make shitty products.

Re:Agile and Scrum Are Like Communism

[Junta]

I particularly struggle with management on estimates. I insisted that for 70% of the work, we don't need projections, and that allows us to properly focus on the minority of work that has *real* deadlines. Management feels like if you don't know when you'll get to it, and no one is specifically going to be looking for the delivery, why would we do it? So we end up with BS no-one cares stuff effectively blocking sudden and important real requirements because we are flagging stuff as 'need to do in three weeks time because someone demanded a random guess'.

Re:Agile and Scrum Are Like Communism

[Required Snark]

RS -232 actually worked most of the time, unlike the other examples.

Re:Agile and Scrum Are Like Communism

[JaredOfEuropa]

You know: that's an apt description of many managers I've worked with. And it's not just Agile or Scrum either: they devour management books that vary between the blatantly obvious and the hilariously ridiculous. They are desperately looking for the right cargo cult, the right set of motions: "If I just find the right steps and follow them exactly, my projects will be on time and my department will run like clockwork".

With that said, and reading the comments here on ./ about horribly dysfunctional Agile environments, my first thought still is that these organisations aren't implementing it the right way. Having 3 hour standups, screaming Scrum masters, and weeks fully booked with meetings means that you're not doing agile wrong, and perhaps you're not doing it at all. Agile is about taking small steps in small teams, removing process or demoting it to guidelines, getting closer involvement from the business, and what have you. But it won't turn cookie-cutter developers into "full stack" rock stars, nor martinet team leads into effective facilitators, and it won't provide management with a clue. And like those management books, it's not a panacea for all organisational ills.

Re:Agile and Scrum Are Like Communism

[Jahta]

I have seen weeks where the entire 40 hours were all Agile/Scrum related meetings. This meant that there was no significant coding done whatsoever.

Then you weren't doing either agile or scrum. The only routine "meeting" should be the daily scrum; 15 minutes max - what did we complete yesterday, any problems (team blockers) that need to addressed, what are the priorities for today? If you are not spending 90%+ of your time working on producing the software, then you're doing it wrong.

In all of my IT work, I have never understood why some managers think that calling meetings will enhance productivity, and if that doesn't work, call more meetings. I don't know if this is incompetence, or an issue with ego. Either way, it hamstrings actual productivity.

Traditional management types really don't like agile because - budgets and time-sheets aside - there is not much for them to do. The team manages itself on a day-to-day basis; the scrum master is a facilitator, not a manager. And the staples of traditional management - like meetings, powerpoints and progress reports - are either eliminated or kept to the absolute minimum necessary. As a result, I've seen cases where traditional project managers deliberately set out to sabotage agile projects to show that "agile doesn't work".

But mainly whenever I've heard people say "agile is crap", what they have really been doing is this. I've worked on many well run agile projects that were both successful and good places for developers to work.

Re:Agile and Scrum Are Like Communism

[gtall]

Agile's biggest problem is that it tends to turn out systems that are dirty snowballs. Gluing bits and bobs onto a system using Scrums and Sprints as a filtering device only encourages the bits and bobs becoming unglued later as the system wavers on its rickety foundations. And even the term "sprint" is a term to keep the entire "team" working as if it were always running the last few yards of the last leg in a relay race. Micromanagers using it are telling their people in precise terms they think the people are lazy dolts who require constant needling and pushing to produce. The people get that message loud and clear, and will find ways to push back.

And Scrums are a godsend for micromanagers who think ordering their flocks daily activities is somehow managing. The point scoring is also tailor made for micromanagers to show their bosses how "much" progress is being made..."Lookee here, see this magic number". Those numbers are pink unicorns and pixie dust to micromanagers.

Re:Agile and Scrum Are Like Communism

[Chris Mattern]

But that's the problem. You can say, "Well, that's not Agile", but so many attempts to do Agile wind up like this. If the method seems to create problems in implementing it, it's not a good method.

Re: Agile and Scrum Are Like Communism

[Anonymous Coward]

What I see are a ton of Agile shops turning out crap, and Agile Evangelists handwaving it away with the excuse that "it's not being done right."

Well if 90% of the places can't do it right, then it's not really a useful system, and all those Scrum Master Certificates are worthless.

The primary purpose of Agile seems to be selling Scrum Certs, loudly defending the model in order to sell more training, and getting Scrum Masters to loudly defend their decision to dump a pile of money on a Cert which is effectively useless.

AGILE is utter shit

[Khyber]

Look, spend the time (and if needed, money) to actually make a solid product from the get-go instead of relying upon adaptability. This is what will net you the best results, customer satisfaction, and fewest warranty/support issues (thus saving TONS OF MONEY.)

Around the 90s is when software development was truly in its prime, despite the shit languages and lacking hardware. It was that shit hardware that forced programmers to figure things out in effective and proper manners rather than relying upon huge amounts of error-correcting glut and hardware to cover up for their n00b-level mistakes that even a TI-BASIC programmer couldn't make.

Our current hardware is literally overpowered for every task we need it to do, if proper coding would be taught and implemented. How can I say this? We've been doing this exact same shit since the 90s. Online video? Yea, back then it was 320x240 if you were lucky, and a fake 640x480 (upscaled 512x384 IIRC) using RealPlayer's codec. Still, we had it, and when the P4 came around, 720p video was a breeze if you had something like a 64MB GPU.

But people tend to ignore history, so there's your historical quip for the night for good measure.

Re:AGILE is utter shit

[Penguinisto]

If you think about it, back in the 90's, do you remember TQM? It spread like a frickin' religion across Corporate America. Every company from GE down to your local strip-mall-based franchise retail outlet preached the Gospel of Deming. The justification boiled down to pretty much the same thing: "This is why Japan kicked our asses back in the 80s! We need to implement this!"

Here's how TQM actually turned out: Some orgs implemented it beautifully. Some gave it lip service then ignored it. The rest picked out what they wanted and shit-canned the rest. Eventually most of it got ignored while a few good bits got absorbed or were mutated to meet the C-level's expectations of it (basically they neutered it except for the bits where they could take good ideas from the proles and claim them as their own.)

Pretty much like how Agile (and its bastard spawn, such as Kanban, etc) is turning out.

The more things change...

Re:AGILE is utter shit

[The Evil Atheist]

Kanban is Agile's bastard spawn?

And people make fun of China claiming inventions from other countries.

Methodologies Are For Hacks

[NicknameUnavailable]

Management and leadership styles need to depend on your team, if you use agile/scrum/kanban/etc it means you are trying to make up for shitty management skills, and in turn are making everyone else waste 10-30% (50%-75% in extreme cases) of their time to make up for it. There is no one management or leadership style (two VERY different things, mind you) to bind them all.

Managers are glorified communal secretaries, they exist to arrange meetings, sit between upper management/clients and developers, and ensure the developers have the appropriate resources while having out extraordinarily high-level orders (e.g. "we have a new project, figure it out,") they don't make decisions but ask for input from all parties involved and arrange the information such that people who make decisions can make them quickly and accurately.

Leaders are just the most applicable guy for a given project the others will listen to, they're in the trenches do the work and can (often should) change from project to project both due to differing skillsets and to prevent burnout of the leader.

Nearly every shit place has the same problem, and it has nothing to do with Agile/Scrum/etc - the shit problem is when you have a person who thinks they are capable of both leading and managing at the same time - nobody is.

This is actually good for agile believers

[Anonymous Coward]

Considering that agile is a giant scam to sell consulting, training, books, etc.... low adoption only means more customers to fleece!

Re:This is actually good for agile believers

[Anonymous Coward]

Anything good will invite bullshit artists to pretend like they can teach you how to do it. So the presence of crappy consulting, training, books, etc....doesn't automatically mean agile is in-and-of-itself a scam.

Much like programming, doing agile right requires above-average intelligence, specifically in one's ability to think abstractly and understand the process deeply. The agile community is brimming with people who only have a surface-level understanding of how it all works, and they ruin everything they touch.

So, while I am in the group that believes agile works well when done right, I qualify that by saying that it is so hard to do right that it may as well be a scam.

bah

[ruir]

Agile and ITIL are a load of BS.

I love Agile

[WaffleMonster]

Agile is amazing. All of our competitors should adopt it.

Agile was doomed by the name

[NotSoHeavyD3]

I was watching a video where one of the guys who came up with it mention the alternative name that one of the other guys really liked. He wanted to call it conversational development. That is so much better of a name for what you're trying to do. Everything is supposed to be a conversation. A conversation is a 2 way street which is the entire fucking point. There isn't supposed to be a deal where say a project manager dictates what he wants and the devs have no say, just do it bitches. You're not supposed to have dev dump a release to QA, deal with it bitches. Hell, you're supposed to have conversation between junior and senior developers to turn the juniors into seniors. I get they called it agile to sell it to corporate but the problem is that by doing it they convinced corporate agile is just "Go fast be stupid"

Agile and Scrum in real life ..

[Anonymous Coward]

I have been in the tech field for almost 40 years, and I have witnessed a lot of so-called 'methodologies'

Agile and Scrum are two of them

They come, they boom, they wither, and then, vanish

No matter what they are, the bottom line stays the same --- software are still buggy, projects are still over budget, and delays are routine

And the basic fact still remain --- the output of a top grade super coder is better than 30 garden variety (American) code monkeys, and if we include the outsourced labor, a top grade super coder produces more and better output than 50 Indian H1-B slimeballs

Re:Agile and Scrum in real life ..

[Junta]

I have said point blank to my management that I don't *want* a team of 50 programmers they 'help' get for me, I want 4 or 5 programmers I actually vet. This makes no sense to them, because more people == better, but in software small teams almost always do better.

Re:Agile and Scrum in real life ..

[religionofpeas]

More people look better when you show the org chart to investors. That's all.

Re:Agile and Scrum in real life ..

[Cederic]

Lower wage bills look bloody fantastic when you show the accounts to investors.

Re:Management likes metrics

[Junta]

Add to that knowing *precisely* what people will be doing 6 months from now. Yes I know this is nominally exactly *not* something advocated in the words of Agile, but because so many in management demand it, the consultants have found ways to present epics with stories planned for months as still 'Agile' to collect their check and let the company have warm self-affirmations that they are now 'Agile'.

In addition to metrics, they have the hope that applying project management can magically allow them to use cheap random developers to either accelerate or replace development.

I've seen projects that were surprise successes, largely because a really good and motivated technical team did something unbidden. Then seen such efforts utterly ruined when the business tried to 'help' by putting it under project management and reassigned developers to be 'more effective'.

Re:First item on the Agile Manifesto

[Junta]

Sorry, that went out the window when consultants saw money to be made. Processes and tools are big money makers. Telling people "just talk to each other" isn't enough to make a lot of money.

Re:No one actually does this, it's just me-too wan

[Junta]

In my experience, those who act in accordance to how Agile describes itself, generally do not call themselves Agile, they just aren't interested in the formal labels. They also don't need anyone to tell them common sense about how to be effective, it comes naturally.

Agile in practice is institutionalized self-delusion for managers in dysfunctional organizations to fool themselves into thinking they can be a good organization by waving a magic wand and getting some certification from a consultant.

Re:First item on the Agile Manifesto

[gweihir]

Quite possible. Fortunately I am a technology consultant, not a management one. So while I do not tell my customers processes and tools are mostly bullshit, I sometimes have a chance to demonstrate it. Of course, a high daily rate helps because then they take you seriously. But overall, I think you are quire right and it is the reason why corporate IT typically is a mess and often going for a train-wreck. We have customers that cannot even implement basic things anymore because their IT is so wrecked by constant new things and doing things "better" as suggested by some business consultants that do not care one bit about the well-being of their customers. What I actually expect is some Fortune-500 folding because of IT problems pretty soon. Sony regrettably survived, but apparently it was close. I know of at least one more where it was a really close thing recently. This stupidity really has to stop, but we need a "reference catastrophe" or two before we can make that happen.

Math

[Sir Realist]

"Only 12% percent responded that their organizations have a high level of competency with agile practices across the organization, and only 4% report that agile practices are enabling greater adaptability to market conditions.."

Look hi. I'm not going to comment about whether the Latest Greatest Fully-Buzzword-Compliant Management Trend is actually backed by reproduceable research or anything. I'm just going to comment about maths. If 12% of your respondents report a high level of competency in a system, and 4% report that that system is actually doing any good whatsoever... If we assume roughly equal levels of response to both questions then we have a system that, when implemented at "a high level of competency" self-reports that system as having a positive effect roughly 1 time in 3. Random chance should have a positive effect 1 time in 2. And self-reported success rates run notoriously high...

Re:It's a crutch

[NicknameUnavailable]

The same goes for developers who have never managed anything, believe me.

This is a thing a lot of people miss, a leader who believes they're a manger is every bit as shit as a manager who believes they're a leader. In the case of a manager attempting to lead it might even be easier to stop them from damaging things because nobody takes them seriously, while in the case of a leader attempting to manage they will be able to actually sway people and completely miss every target be it abstract or refined. The two types of thinking simply don't cohabitate in the same mind at the same time (though the illusion that they do/can is often there for the person attempting it.)

What is this Agile?

[TJHook3r]

Would be interested to know if a majority of companies could actually give an accurate definition of Agile. Have heard from too many managers that a project will be 'Agile-like'. Generally meaning 'half-assed'.

Sacrosanct Schedules

[SwashbucklingCowboy]

"and only 4% report that agile practices are enabling greater adaptability to market conditions..."

"The three most significant challenges to agile adoption and scaling are reported as organizational culture at odds with agile values (53%) ..."

"The researchers also note "the recognized necessity of accelerating the speed of delivery of high-quality software ..."

If it's anything like my company, it's because schedules are sacrosanct. Anything will be sacrificed to hit schedule. It sucks.

Knowing that Agile isn't SUPPOSED to be fast

[raymorris]

According to Scrum.org, a big part of Agile competence is understanding what the entire point of Agile is, and it's NOT supposed to make development faster or cheaper.

The point of Agile and Scrum, according to an article on Scrum.org, is to provide a way to deal with situations where you can't go sit with users to understand what the actual requirements are, and you can't look at some legacy system you're replacing, so you have no way of defining the requirements except "try something and ask users if it's getting closer to what they want." Scrum is a system to quick iterate through different things, presenting each possible deliverable until you get close to what users need.

If you CAN go sit with users and take notes as you watch them work, if you CAN look carefully at the system you're replacing, if you have any way of defining requirements before you start coding, Agile iterations will take LONGER than just defining the requirements and then building something that meets the requirements (while being conscious of boxing yourself in, because next year requirements may change a bit).

Re: The best explanation was written long ago

[clawsoon]

I'd agree that there's an overlap in intent (and justification for violence!) between democratic socialism and Bolshevik socialism. But if you look at which regimes (actually, literally) worked people to death, you'll notice that it was those which had extreme ideas about property rights - libertarian 19th-century Great Britain, and dictatorial Soviet and Chinese regimes. I'm comfortable partway between the two, with a lean toward the Brits. (They did work people to death, but a lot less than the Soviets.) In practical terms, democratic socialism has resulted in the smallest number of people being worked to death, and that seems like a pretty good metric to me.