Slashdot Mirror
It's Not Developers Slowing Things Down, It's the Process
An anonymous reader writes: Software engineers understand the pace of writing code, but frequently managers don't. One line of code might take 1 minute, and another line of code might take 1 day. But generally, everything averages out, and hitting your goals is more a function of properly setting your goals than of coding quickly or slowly. Sprint.ly, a company than analyzes productivity, has published some data to back this up. The amount of time actually developing a feature was a small and relatively consistent portion of its lifetime as a work ticket. The massively variable part of the process is when "stakeholders are figuring out specs and prioritizing work." The top disrupting influences (as experienced devs will recognize) are unclear and changing requirements. Another big cause of slowdowns is interrupting development work on one task to work on a second one. The article encourages managers to let devs contribute to the process and say "No" if the specs are too vague. Is there anything you'd add to this list?
This is not exactly accurate. It hinges greatly on the type of manager we're talking about.
For example if the manager is very hands-on, goes into the details, produces proper mock-ups, flow diagrams, and everything is properly documented: This type of manager can actually accelerate the development process significantly since developers now know exactly what to do. But again, this manager has to really know what he's doing, and have some serious programming experience in his past.
All those moments will be lost in time, like tears in rain... time... to... die...
There is also the management of the manager: If the manager has too many products/projects on their plate they can't dedicate the time and focus to being that detail-oriented manager that gets things done. A good manager can turn into a bad one if they can't focus. That's where even higher-level management needs to set expectations and manage proper headcount.
You don't want to take managers out of the equation. They're the only people keeping the other departments from running you over. You see that most clearly, ironically, when you have an incompetent manager and you get run over in spite of it.
And a bad manager in this sense isn't the evil taskmaster, it is the guy who has no idea of his team's capabilities and taskload. He's also probably a little clueless about what is or is not possible, but in that sense, it's more of a feature of him making promises without talking to the rest of the team first. That manager goes to meetings and lets himself get cowed into submission when sales or marketing goes after him because he has no facts. Removing someone in that position just means that engineering is no longer even a speed bump to unrealistic goals.
Saying "No" to business people is not a valid strategy. You'll just find yourself replaced. Saying, "yes, but you'd need to spend 2 million dollars on it" with proof is a valid strategy. You don't want to sit around and come up with that data, that's what the manager is supposed to do.
I agree that indecisive managers and overwrought process is probably the top cause of problems with productivity. However, there are good managers and bad ones. It pays to understand the difference.
Take managers out of the equation and work gets done. Pretty simple.
Yes, but what work gets done?
Is it the sexy feature that the dev has been just dying to implement since he/she read about some new language/process/data type de jour?
Or is it fixing the hard to locate bug in deep in the back end that deletes all the users data on seemingly random occurrences (and can be brushed off in dev's opinion as merely an aberration)
I am Slashdot. Are you Slashdot as well?
That is a good one. Sometimes, managers are having too many projects to manage at once and are experiencing the exact same problem as lead developers. They have to switch context so often they lose completely the focus when time comes to a particular project.
Usually, when a position post description mention: "must work well under pressure", "must be able to multi-task", "seeking for rockstar developer", etc; you can be sure you don't want to work there. It is symptomatic of a shop which operates without enough staff and on low budget expecting miracles to come out from the process.
Achille Talon
Hop!
First, that doesn't seem to be what the article is saying. Second, I don't really believe that it's true.
When I say "don't believe that it's true", I'm saying, "I don't believe that the removal of managers necessarily gets work done faster." I'm not talking about programming specifically-- I'm not a programmer, and managing programmers is not my expertise-- by my general experience is that a lot of people think managers are just wasting everyone's time, when the reality is more that most people don't understand what managers do. Unfortunately, this sometimes includes managers.
A good manager often spends his day trying to figure out how to remove obstacles so that the people he's managing can just do their jobs. For example, the summary says, "The article encourages managers to let devs contribute to the process and say 'No' if the specs are too vague." That sounds right to me. First, a good manager will of course listen to the people he's managing. That doesn't mean doing whatever they say, but when I have managed programmers, I assume that they know what they're doing better than I do, so if they say there's a problem of some sort, there's a problem of some sort. I wouldn't always go with their recommended solution, but would I listen to their explanation of the problem and try to come to a solution that addressed the programmers complaint as well as meeting the business needs we were trying to address.
If specs are too vague, that seems like the sort of thing a good manager would help to work out. For example, I might suggest talking to the programmer, trying to figure out which aspect of the specs are too vague, and then meeting with the stakeholders to try to clarify the specs. I wouldn't necessarily make the developer get involved in the process of clarifying them, since unless they're needed for the discussion, they probably have better things to do.
But being a good manager is pretty difficult in general. It's often not clear what needs to be done, or how it ought to be done, and it's your job to figure that out. It's pretty much impossible to be a bad manager without annoying people, but even the best managers might seem annoying or clueless because you don't see what they're doing for you. Sometimes good managers are only noticeable in their absence-- when they go away, you suddenly go "Oh jeeze, things are falling apart a bit here. How was it that we never had these problems before?" And the answer is, you were having those problems, but your manager was dealing with them when you weren't paying attention.
On time completion: It will be done as soon as it can be done, only experienced teams can provide reasonable estimates, developers provide timetables not managers, there's a specific amount of work that must be done before release and putting dates on it won't reduce the total amount of work that needs to be done.
Unclear requirements: It's now the developer's job to talk to the stakeholders and find out what all the requirements are at the point in time they need to size or implement them. They get a vague 'story' that gives an overall concept of a requirement, and then it's up to the dev to talk to whomever needs talking to, then.
Changing requirements: This happens and everyone expects it. So you do only the least work required to complete each requirement so the overhead in a change will be the smallest, and then you just pop that item on the queue and it gets done, that's it.
Context switching: Tasks are assigned to a team by managers, not as a sole developer, so if context switching is causing a problem, it's up to the team to figure out how to minimize it.
Take responsibility: They are, and increasing the duties that a developer has, while removing certain responsibilities from managers and product owners, which more accurately represents the reality of the situation.
Caveat: Recent studies have shown that Agile is not as good as a waterfall-agile mix, where you do a good amount of planning (especially architectural planning) prior to the agile-like development, which makes a lot of sense. If half your work effort is refactoring, at some point you start take a severe hit to either efficiency, quality (robustness, maintainability, operational limits like memory or speed, etc), or time.
That's nice, you left out architecture. Who's doing the overall architecture? Has it been done before? If it is new, better spend a lot of time doing the architecture. Unless....
You have caught Agilitis. In this case, spend no time doing the architecture up front, do it on the fly. Add time to every single task you think you see for doing architecture related stuff. At the end, allocate a large blob of time to figure out the correct architecture rather than the one you built which now resembles a dirty snowball. And begin rebuilding the thing, reusing any errant parts you did in pass one. And because you are agile, be sure to appropriate time for execs who understand agile as they get to change what's necessary on the taste of their coffee that day, whether their secretary gave them a perky Good Morning, or whether their hair will turn sufficiently silver in time for that next promotion.
Scrums should be 15-20 minutes a day. Period. And they should be stand-ups. And they should only be used to unblock not for status.
Project status should be reported by the position of the cards on your board. You do Agile well and you don't need a daily or weekly status meeting for developers. If you are doing code reviews, you are evaluating code and progress at the same time.
If there is a weekly status meeting with product owners, the manager or scrum master should be attending and no one else. That's business stuff, not development.
I agree that meetings are problematic when it comes to specs. You usually need people who are knowledgeable technically to comment. That bogs things down sometimes, especially when product or marketing gets involved. At that point, the manager needs to say: "I need your minimum viable product for this date.", and then you work out a schedule and hold everyone to it.
When you start getting things into production instead of dithering, business starts getting a lot happier, even if they don't get everything they want. You absolutely MUST manage scope creep. The plan needs to be committed to and reviewed at specific points where demos are provided. The manager must run interference for developers and the business people should not be trying to alter the parameters in mid stream.
And yes, developers need to develop. If you are a manager and your team is in meetings all day long, you need to get more managerial skills or get a job at a place where your business people are going to let you do your job properly.
That's a manager who manages down. Most managers manage up.
In my ~15 years of experience "manage down" managers are hurting their career advancement prospects by helping the company deliver. It's one of those corporate blackholes. Normally if you take care of your house, you have a nicer house to live in. If you maintain your car, you save money and have a more reliable car for longer. But in the corporate world, investing in yourself is usually trouble.
Not everywhere, but 3 out of my 4 employers.
Yeah, my company does that.
75 features to be implemented by the end of the quarter.
2 weeks in, cut it down to 50
1 month in, cut it down to 20.
Actually deliver 12 features.
Planning & prioritization are all over the place for the first month. And code freeze is 2 weeks into the second month.
Every single quarter. Why don't the people expecting 75 features every quarter get fired?
I'm just glad I'm not a developer here.
There are two types of people in the world: Those who crave closure
Of course, one could theoretically argue that working for an employer who won't give due consideration to a developer's input isn't worth working for in the first place, but that nice little theory doesn't pay the rent. Oh, and try explaining to a future would-be employer about why your last job didn't work out... care take a guess how that will go down?
Of course, it might seem that working for yourself solves much of this problem, but even that still requires that you actually have paying clients, enough of them that you can support yourself on what they are willing to pay you. Of course, then you are actually back where you started, where saying "no" to a person who pays you money to do whatever it is that you do carries a risk of not getting paid by that person again. If you have enough clients, this may not be a problem to lose the odd one or two because they are dillholes, but getting to that point will take time... possibly many years... so until that time comes, you'll just have to do whatever the heck the person who is paying your salary tells you, unless you really have an affinity for living in a cardboard box on the street.
File under 'M' for 'Manic ranting'
I completely agree with your point, but would like to observe that senior or mid-level management always cares the LEAST about fixing old, broken stuff.
Every place I've worked has had serious ghosts in the closet, but projects to go clean up old messes never get approved. This has been true across business, IT, and development roles (in my experience).
After all, leadership doesn't get bonuses for reducing risk to the company, they get rewarded for the next feature/launch/whatever.
Especially funny. Rock Stars don't multi-task. They lock themselves away in a darkened office working on the currently interesting single problem until it is solved. Then they come out, decompress, and repeat.
Part of why they have rock star performance is that they don't multi-task and don't sit through endless meetings re-hashing yesterday's meeting.
Your argument is nonsense and can be extended to: he didn't even mention the servers, be sure to add a few megs of RAM for each feature to developed, and some HDD space too. Oh, and increase the chip speed for each new feature as well! Then later but new servers to replace the cobbled together disasters you built on the fly.
We use Agile for the *development* process which is distinct from the *design* process. That is, we do architecture up front, including DB design, and - surprise - we even build the server platforms too!
Business gets an opportunity once a month to change development priority order. Once. A. month. New features introduce are written up, added to the feature list and the business gets to include them in next months re-prioritization.
It's simple , everyone gets it, and the business (ie the people paying the bills) love it. And that's bad, how?
Seriously. Does no one read Mythical Man Month anymore? This stuff has been done to death for ages now...
I'm work for an organization that provides design services (as opposed to building and selling products). If you are ever, ever , realistic about the time it will take to deliver or what features you can include in a design for a given set of resources, you won't get the job. It's as simple as that.
Why do you think most construction projects go over budget? One big reason is they had to make a crazy bid because if they didn't, someone else would.
The bottom line is: if you say no, you're out of a job.
One problem with discussions like this is that there is a lack of consistency in titles and job role naming across companies.
For example, in some companies first level managers of developers are not very technical (fortunately, I've never worked at a company where this was the norm) and can only manage work units and people but not solve technical problems themselves or provide detailed technical guidance. They can be good people managers, good at working the politics, good at making sure that the project dependencies (both inward and outward) are being tracked, and good at protecting the group from abuse. However, they need to rely on project leaders/lead programmers for the technical stuff. In other companies (fortunately, the most of the ones I've worked at as a developer or a manager), managers are de facto project leads and/or psuedo-architects and are able to (and do) look at code, review specs, make technical decisions when necessary.
Similar story for "project managers". At some companies they just push lines around on PERT charts and note and track that there's an issue that needs to be resolved by next Thursday about if the asdfasdf is to provide some data to the lklkjfsdf or if the lklkjfsdf should independently fetch it. At other companies, "project managers" would actually know what asdfasdf and lklkjfsdf were and be able to understand, at some level, that lklkjfsdf couldn't possibly independently fetch the data because security policies don't allow it no matter how loudly the owner of asdfasdf insists otherwise (and, knows who/how to bring in to shut the owner of asdfasdf up).
Why is there an "insightful" mod and why isn't it "-1"? If I wanted insight, I wouldn't be reading
Good lord, I could not agree more. From what I've learned in the last three companies I worked for that used agile, agile means that an ungodly amount of time is spent in meetings and constant, meaningless record-keeping. God, I hate agile. It gets in the way of doing good, timely work.
We use Agile for the *development* process which is distinct from the *design* process. That is, we do architecture up front, including DB design, and - surprise - we even build the server platforms too!
Don't ever leave your job unless this is becomes no longer true. So many people think agile is free tick to skip design and when wonder why agile isn't saving them from creating a pile of crap.
Rock star performance is achievable by anyone who gets to focus on just one thing at a time. It's also called being a prima donna, not a rock star.
--#
It depends which parts are "agile" and which are "waterfall". From my not-exactly-vast experience, whatever mix you choose has to address four concerns:
1. WHAT ARE YOU BUILDING? Seriously. This is where the extra planning of "waterfall" -- itself a misunderstanding of someone else's comments -- comes in. One of my first jobs was a derivatives trading app in a perpetual tug of war between an outspoken exotic derivatives trader, back office and compliance folks wanting to automate trade reconciliation, risk managers wanting to manage risk, and the rest of the trading desk who just wanted to price whatever deal they were trying to do (vanilla or not). The end result was a kludgey mess that everyone hated.
2. HOW DO YOU ALLOW FOR GROWTH? There are right ways and wrong ways. Agile has a good tip: build only what you need, and as cleanly as you can. Better said than done, of course, and sometimes (as the Pragmatic Programmers have said) you *know* there will be a database in there sooner or later, so plan your architecture accordingly even if it's a little awkward short-term. Then there's the wrong way, which I saw in one project: build a "flexible" architecture with "configurable" components so that you can do "anything"! 1) KNOW WHAT YOU'RE BUILDING, even in broad strokes (see above). 2) For software to be reusable, it must first be usable for *something*. 3) If your "flexible" and "configurable" architecture takes more time to modify than writing straightforward code would take, it has failed; start over. 4) It's better to use open-source architecture than build it yourself. (Buying is also better, but beware of vendor lock-in and every-increasing fees, especially if you only use a small part of an elaborate framework.)
3. HOW DO STAKE-HOLDERS REQUEST CHANGES? No specification will be adequate out of the gate, and unless you're working for NASA or the military people CAN and WILL request changes, sometimes while you're building the product and certainly once people begin using it. Do you jump when they say how high? Do you have a long and slow review process? Do you work individual tickets? Do you have potentially disruptive "projects", and if so how do you integrate them back into the main project? Different applications have different rates of change and different tolerance for defects, but it's best to work out change process before the complaints come in.
4. WHEN AND HOW DO YOU CUT YOUR LOSSES? Despite your best efforts the whole thing will become obsolete or unmaintainable, sooner or later (hopefully later). As in point #1, have some idea which you're building, and at what point do you deprecate it to build something better. (This is the hardest part for many organizations; why can't the floor wax also be a salad dressing?) At some point make a plan, refactor your code base -- over time! -- to separate the parts you'll keep from the parts you don't need, and toss out the latter. If a radical rewrite is the only way to go, bite the bullet and do it, with appropriate safe-guards like regression tests. If you can't evolve or replace your product, a competitor -- maybe within your own organization -- will do it for you.
That's my long-winded advice, from someone who's watched many "successful" and "unsuccessful" projects die. Nearly all in-house software is a Potemkin village designed to keep the tzars happy. If the tzars aren't happy, the fake village is set ablaze and you, the fake peasants, go to Siberia. The only real question is how to keep your frantic peasant dance going as long as possible without dying of exhaustion or being sent to Siberia.
TL;DR: Plan the limits, goals, requirements, and large-scale architecture up front (erroneously called "waterfall"). But planning never really ends; stay agile to correct your course or take advantage of opportunities ... and be willing to throw the whole thing away if warranted. Also, keep your resume updated.
To my mind, the best managers set up conditions for their staff to succeed and defend them against higher-level managers. That's it. Managers should take care of head-count and hiring and meetings and all the other boring but apparently necessary stuff; they shouldn't be responsible for *any* analysis and design, let alone coding. Because all the "boring" stuff will eat up someone's time, and better the designated victim, I mean manager, than someone responsible for making progress.
The *worst* managers, in my not-too-atypical experience, are the rock-stars, micromanagers, timekeepers, and corporate climbers. True rock-star programmers should be writing code, not going to meetings ... and if they can't cooperate with the rest of the team they shouldn't be on the project at all. Micromanagers second-guess their staff and the staff waste valuable work time indulging the boss. Management by Gantt chart is even worse, since estimates in software are rough guesses, not contractual commitments; the work takes as long as it takes, for a set amount of work, and forcing the staff to work harder to satisfy a bogus schedule only makes work later (or buggier). Worst of all are the managers who are simply mouthpieces and boot-licks for their bosses, because they'll throw you under the bus for any perceived failure and claim credit for all your successes, often both at the same time.
This is provably false. There are certain people, we'll call them rock stars, that you can stick in a room and they'll get a better solution out faster that addresses everything you need versus 100 lines of doesn't work for 20% of the cases code. Sometimes the rock stars do this to existing code with a 1 line mod. It's because they actually take the time to understand the problem and can visualize the system, instead of writing some boilerplate garbage cut and pasted from stackoverflow.
The cesspool just got a check and balance.
Couple of big shops in my town. Take one for example. They had a 2-year window for a very important project.
Bought (expensive trendy tool) from (major software vendor). Spent 18 months drawing stick-figure diagrams with (expensive trendy tool). Realized they only had 6 months for implementation and panicked. Basically tossed the stick-figure diagrams because they had to drastically modify expectations to make it in 6 months of 100-hour programming weeks. Using contract laborers who didn't know the company and how it operated, because they'd taken a chain-saw to the ranks of the in-house experienced staff.
I'm sure that they learned a valuable lession from that and will never do anything like that again. I'm also sure that pigs fly and cows routinely jump over the moon.
Rock star performance is achievable by anyone who gets to focus on just one thing at a time. It's also called being a prima donna, not a rock star.
--#
Pri Madonna, a Pop star.
So often I see developers and often their managers being loaded down with things that have exactly zero to do with the project's success. For instance I have seen companies that had an 8am meeting every morning at a company that claimed flexible working hours. These meetings were nothing beyond empty platitudes from a manager.
Many companies that I have visited had sales dictating what needed to be built in that they would chase every whim of every potential client resulting in 100's of features being built for nobody (didn't get the client) but slowly turning the projects to crap. Then sales would regularly blame the programmers for letting them down by not developing an AI driven natural language database over the long weekend.
The single worst that I saw was a company where the president's wife suddenly started dictating features. These features were completely unrelated to the core product. It would be like a VW engineer suddenly getting instructions on adding a flower arranging module instead of working on a recall where the cars can catch fire. This had the entire project's staff leave one by one until the project just died.
But if I had to pin project failures down to a single consistent problem it would be that really terrible programmers who are complete blowhards often get way ahead of their co-workers leaving a trail of fuming resentment in their wake; the primary result is that the most skilled programmers are the first to leave resulting in a rapid plummet of the average level of talent as once the best go often the next best are soon to go, and so on until you are left with only duds who don't dare to find another job.
As for what system is in place it doesn't really matter much as a talented group will make it work. But if that system is encouraging any of the above disaster scenarios the project will suffer.
This is a quote from Elon Musk on what he thinks about "process":
"I don't believe in process. In fact, when I interview a potential employee and he or she says that 'it's all about the process,' I see that as a bad sign.
"The problem is that at a lot of big companies, process becomes a substitute for thinking. You're encouraged to behave like a little gear in a complex machine. Frankly, it allows you to keep people who aren't that smart, who aren't that creative."
This just about nails it for me.
This and no other is the root from which a tyrant springs; when first he appears as a protector - Plato (423 to 327 BC)
Corporate Policies requires that developers cannot have so called 'elevated rights' on a server. Any server, including test and development servers.
Well, that is, the developers have been granted local admin privileges for the development servers, but as a special exception to the corporate policies.
The daylight savings patch that has to be installed, requires an upgrade of the application on the server, with an uninstall and a fresh install.
With a full redeploy of the content and reconfiguring connections to ldap, databases, other servers and reconfiguring user autorizations linked to the content.
The developer documents the deployment procedure in an installation guide.
Next the the upgrade has to be deployd to the test server, but none of the developers have local admin rights for the test server.
So, resources from platform operations have to be claimed by the coordinator. For the installation and the finalization of the application upgrade.
The upgrade takes a little more than the standard 2 hours that have been reserved per week, but finally after a week a slot is available to do the part that has to be done that requires local admin rights for the test server, by someone from platform operations.
At this point, the system test has slipped by a week, on a monthly release cycle, that is a significant amount of time.
A couple of day later the upgrade is deployed to the acceptance server for user testing. Except that most of the users refuse to test the changes, because there are no new features. In their eyes it is purely a technical upgrade. Nevertheless a bug has been found, and it is declared blocking. It takes some days to resolved it. By now, due to all the previous delay, the issue has not been resolved in time to get the change in production.
The monthly release date slips. The next slot available is the next month, and the application gets finally released into production.
Essentially, it means that if something does not get tested beforehand, like a deployment procedure, it eventually gets tested in production.
That is the best way to test something, isn't it? A consequence of the Corporate Policies.
Absurd?
Now I am going to watch some South Park episodes. I like documentaries.