The Programmers Who Want To Get Rid of Software Estimates

An anonymous reader writes: This article has a look inside the #NoEstimates movement, which wants to rid the software world of time estimates for projects. Programmers argue that estimates are wrong too often and a waste of time. Other stakeholders believe they need those estimates to plan and to keep programmers accountable. Is there a middle ground? Quoting: "Software project estimates are too often wrong, and the more time we throw at making them, the more we steal from the real work of building software. Also: Managers have a habit of treating developers' back-of-the-envelope estimates as contractual deadlines, then freaking out when they're missed. And wait, there's more: Developers, terrified by that prospect, put more and more energy into obsessive trips down estimation rabbit-holes. Estimation becomes a form of "yak-shaving" — a ritual enacted to put off actual work."

Comments

Simple methodology

[Kjella]

Good managers get my best guess.
Bad managers get my worst case.
Horrible managers get my resignation.

Tag: WORKS4ME

Re:Simple methodology

[Penguinisto]

One would hope that a good manager would have enough practical and direct experience in writing software to at least come up with a half-decent estimate, no?

Most shops I've seen lately have the scrum masters spend a part of a planning session simply asking individual contributors "Here's a rough outline of the proposed project [...] now how long do you think doing that will take", and they come up with an estimate adjustment from there... most of the time, it's fairly close. PMs pad things a little of course, but the results tend to be fairly close.

YMMV of course... depends on who is posting the final estimates - is it devs, or is it the MBAs.

(If it's the latter, run like hell.)

Re:Simple methodology

[jellomizer]

Well a good manager, will work with the developer and give them clear deliverables, adequate milestone achievements even if these milestones may not look like much. Then we can give a good estimate.

A Bad manager will toss out a pie in the sky grand vision that does something. So the programmer will need to go back and rework idea over idea, going back and improving on the design.... So this will take a worse case scenario.

Re:Simple methodology

[lgw]

You can't really give a good estimate up front on most projects, because even if the requirements are well defined (and they rarely are) they will change.

That's all OK. That's the whole point of Agile, if you ignore the Agile BS that consultants sell. You can quickly do coarse-grained task breakdowns (say, to 2-week tasks) to get a reasonable estimate of the work as you currently understand it: that estimate is generally within a factor of 2, if the requirements don't change, so you can at least see if you have 1/3rd the staff you need or something way off.

Then you measure real progress against that first-take estimate. Usually by about 6 weeks in on a team-sized project, you'll have the real multiplier. "We thought the first set of tasks were 20 programmer-weeks, but really as we understood it it was 30", and multiplying that initial "6 months" estimate by that measured 1.5 multiplier is a damn good estimation method. And it only gets better as work goes on. I can reliably give fairly accurate estimates from project completion now, on Agile projects, by the time we're about 1/3rd of the way done (and that's so much better than some teams are used to that it seems like magic).

Selling that all properly to management is a different thing, but that's a whole job skill that any senior engineer just needs to have. Just like showing management the cost of each requirements change, as well as the skill to minimize that cost of change (which is the real point of Agile- the rest is hype).

Re:Simple methodology

[Anonymous Coward]

Yeah. Estimating the time to develop a simple web page is one thing. Estimating the time to design and engineer a new enterprise system is another. It seems that there are "managers" who think the first can be translated into the second. I'm sure Cheops asked his engineers to specify how long it would take to build his pyramid... The answer? "It depends!". Depends upon how many workers we can get, how far we have to go to get the stones, what the weather will be like...
 

Re:Simple methodology

[knightghost]

A well defined project can be estimated. Change Orders estimating needs to be done properly and BILLED - the cost of re-analyzing is a real cost and the business needs to see it. Once you get that across the number of change requests decreases dramatically.

Software engineering is Engineering... some of the costs are inverted, but otherwise it's the same project management as other engineering projects.

Re:Simple methodology

[sjames]

Then something changes and blows the estimates out of the water. But the MBA's think since only one line in the spec changed, the schedule should stay the same.

Re:Simple methodology

[Gr8Apes]

When I have one of those... I just say - well, if it should stay the same, why don't we just change that one line back. There's no difference according to you.

Re:Simple methodology

[Gr8Apes]

Agile is pretty much crap. You don't build a bridge that way, nor any other project, except maybe landscaping. If you have enough to sketch out a plan, you have enough to plan. You can't build something if you don't know what your target is.

Re:Simple methodology

[RabidReindeer]

Then something changes and blows the estimates out of the water. But the MBA's think since only one line in the spec changed, the schedule should stay the same.

More often in my experience the managers and the users refuse to believe the estimates and force new "more realistic" estimates to be submitted.

Because "It's Simple! All You Have To Do Is..."

Re:Simple methodology

[Zero__Kelvin]

"One would hope that a good manager would have enough practical and direct experience in writing software to at least come up with a half-decent estimate, no?"

No. That is the whole point, which you have seemed to miss. I'm the software engineer and even I can't come up with a reasonable estimate; why the hell would some manager several layers of indirection distant from the design be better at it?

Re:Simple methodology

[Anonymous Coward]

Exactly - that is why there was been a growing recognition that Agile isn't getting the job done. At my last company a bunch of Agile zealots came stampeding my way trying to say that we had to change all our development processes to be "agile" because, well, just because. So I went through a simple requirements exercise with them. The first and foremost requirement is that when sales/marketing sales that they want 100 features in the product and there is no way that we can doo all 100 features, we have to be able to prioritize them. One of the primary factors in prioritization is looking at the effort/reward ratio. Something that is little effort but has a big payoff is high priority. Something that is big effort with little payoff is low priority. Now, please tell me how Agile is going to tell me how much effort is going to be required to complete a particular project when the whole Agile approach is to start coding and figure it out as we go?

silence

OK, here is another requirement. We need to put together timelines because Engineering doesn't operate in a vacuum. We are part of a company with many parts all of which have to be coordinated. Sales needs to be able to set expectations with customers. Marketing needs to make plans for when and how to introduce features to the market place and make forward looking statements on upcoming features. Support needs to plan out how it is going to support upcoming releases, which requires logistics in terms of when to apply which resources to which tasks. So how does Engineering give people these timelines if we don't take the time up front to plan out what we are doing prior to banging on our keyboards?

silence

The big problem with Agile is that it was created by Engineers in a vacuum who failed to recognize their responsibilities to the rest of the Company. Couple that with an increasing number of studies that are showing that software quality is lower now and that it seems that Agile is part of the reason (basically, the idea that somehow the coordination of effort required to make a large project come together would magically happen "organically" is being increasingly debunked) and, well, clearly it is time to re-evaluate how Engineering needs to operate within a Company.

Re:Simple methodology

[blue9steel]

Of course they also don't usually decide that they'd rather have a skyscraper instead of a bridge when you're mid-project.

Re:Simple methodology

[BronsCon]

Depending on how far along in the project you are, it's reasonable to expect changing a single word in the spec to have a major impact on the project. Consider:

Paint my car black.

So you do all the prep work, car's primed with a dark primer, black paint is mixed and ready to go, then this change request comes in:

Paint my car white.

Well, now you've wasted the paint you just tinted black, and you can't paint white on top of dark primer (well, you can, but you need many more coats), so you've got to redo the prep work. That means waiting while the primer fully cures, so you can sand it off properly; otherwise, it'll gum your sandpaper. then re-prime, then you can paint. Assuming you don't see

Paint my car forest green.

in the meantime.

That was one word. Yes, one line matters.

Re:Simple methodology

[Maxo-Texas]

I had no particular problem with estimates.

At a minimum, you could break out easy "construction/recognized pattern" work from risky new stuff.

As far as managing programmers... it was humorous.

Few liked giving estimates. So they would say it couldn't be estimated.

So I would ask, will this project take 2 years... and they would say, "oh no- of course not" and after a bit, we'd get down to 6 months or 6 weeks or 6 hours...

So then we'd time box it to what could be done in a month and move any risky items up to the front so we could establish if a new technology wasn't going to work before we put a lot of work into the project.

Then, I recorded over/under for every project and found (over about 24 programmer data set) that programmers consistently overshot or undershot their estimates. So after a few projects, I had a pretty good idea of their deliverables.

Finally status reports and status meetings with function points and overall percentage delivered kept things on schedule or let us know well ahead of time there was a problem with the estimate/schedule.

Programmers were not my problem- executives were.

They...
a) pushed us to violate standards.
b) ordered overtime without ordering it. As in assign 80 hours work and then insist it be completed when everyone knew it couldn't be completed. Made worse by the fact the indian contractors said "I'll do my best" for "no- you are batshit crazy" and then things fell apart when the indians were unable to deliver. Of course, the indians were very good at delivering to the (crazy/incomplete) specifications on time. At least enough to be testable. I'm not sure if it is because they were contractor types or that they were indian- perhaps a bit of both. I learned in a contracting shop, you always say you can meet the estimate (to get assigned the work) instead of giving a realistic estimate. Then renegotiated it later when it wasn't going to make the schedule. If you didn't, then the three other people bidding on the work would get the work. Executives seemed to have zero memory for the fact that you delivered on time on estimate while the other people were usually late.
c) made everything priority 1a. they had no ability to prioritize as far as I could see. Which really just pushed prioritization down to me or the programmers.
d) cancelled projects without warning.

Re:Simple methodology

[ATMAvatar]

The compile step is building the bridge. The coding step is drawing the blueprints, which *does* involve iteration.

Re:Simple methodology

[Anonymous Coward]

Now, please tell me how Agile is going to tell me how much effort is going to be required to complete a particular project when the whole Agile approach is to start coding and figure it out as we go?

Sorry bro, but if you got silence in response to that question, those weren't "Agile zealots." They weren't even "Agile newbies." They were ignorant cunts.

Agile does NOT say "start coding and figure it out as we go." Agile says "Take some small subset of the functionality you've declared "the most important," estimate that, and deliver that."

Somebody has a list of 100 features? Great, estimate business value and level of effort - use story points, t-shirt sizes, etc. That alone gets you a pretty good way to prioritize your list - things that are "small-to-medium LOE, high value" go to the top of the list. Then you estimate those specific features, and start working on them - to start delivering business value as quickly as you can. Then as you go down your prioritized list, you continue refining and estimating the other features and pull them into your work queue as they reach the top of the "Effort x Most Valuable" ranking.

So how does Engineering give people these timelines if we don't take the time up front to plan out what we are doing prior to banging on our keyboards?

If you're a true Agile team, you're cross-functional, which means that proper representation from ALL of the organizations needed to deliver the product is present, embedded in your team. While devs are banging out designs and code, Testers are working on test plans and test automation given the specs that the devs produce, and Support people are building operations documentation and support matrices etc. - all in parallel, all coordinating with one another as the process goes. A team with proper representation from all functions SHOULD be able to provide necessary dates and timelines, because there is no "Dev team" just throwing dates over the wall to Support and Manufacturing.

Agile DOES answer these questions - if you're able to sit there and pretend that you've never heard the answers, then you're incredibly ignorant, or incredibly invested in somehow feeling like you've "discredited" Agile by sneering at it.

Re:Simple methodology

[presidenteloco]

My methodology:

If someone gives me their estimate for a software project or task, I double it and add 30.

If someone asks me for an estimate for a software project or task, I rough it out, then double it and add 30.

It's really amazing how much stress that avoids (oh, and it also does a passable job of converting Celsius to Fahrenheit.)

Re:Simple methodology

[Registered Coward v2]

A well defined project can be estimated. Change Orders estimating needs to be done properly and BILLED - the cost of re-analyzing is a real cost and the business needs to see it. Once you get that across the number of change requests decreases dramatically.

Software engineering is Engineering... some of the costs are inverted, but otherwise it's the same project management as other engineering projects.

Correct. Our rule was "Give away the initial project because we'll retire on the change orders..."

Re:Simple methodology

[Darinbob]

I've been programming for over thirty years, and I can't do estimates. At all.

Even when I think something is easy, it turns out to be hard if there's some snag I didn't foresee. If I think it's hard then I may find out it's not so bad after all because there are already tools to help put. Something that seems intellectually easy may take a long time because it involves a lot of tedious programming or detailed testing. Writing documentation takes me forever, I can not do that quickly at all unless someone wants a plain text file (which no one wants anymore). One big problem here is that I almost *never* program something that's been done a hundred times before, it's always something new, something that requires learning a new system, new hardware, something requiring reading through a lot of documents first, something with a huge amount of unknown variables, and so on. I also want to deliver something that works with minimal to no bug fixes required later, as opposed to shipping on time followed by a long sequence of updates.

Now certainly for some things I can say that I'll be done that night, because it's something really easy, like reverting a change, fixing up an obvious bug, and so on. But even then I occasionally hit a snag that's unpredicted.

The big problem is that the people who want the estimates don't understand this. And they come in all types. There's the person who thinks that a little bit of pressure makes people perform better (ya, you gave us someting to do in 3 months that needs 9 months). There's the person who thinks the estimates are accurate and golden and then sells the product before any work has begun. There's the person who thinks that your estimate means you will be working 100% of the time only on that single project and nothing else. There's the person who has a preconceived idea of how long it will take and considers that any longer estimate is "sandbagging". There are the people who will put me on two of the companies most important projects of all time, at the same time.

At one place, I gave an estimate of a simple task to be "two weeks after the main project finishes I will have the integration finished". So the sales rep wrote down a specific date based on that. The main project was late, but when that date arrived the sales guy comes up and asks me for my piece, which I had not started yet and could not start yet, then he panics because he's promised it to the customer (before even the product I am integrating with even exists). Ultimately I ended up with only two days to work on it.

I seriously wish that people would switch to a model where they announce a product after it is finished, and schedules are not based upon when the next conference or trade show is.

Re:Simple methodology

[BronsCon]

I have a project that was due last week that I haven't started on yet. I got the deposit check the day I handed over the SoW, but didn't get the signed SoW back until last week, 6 weeks later. As per the terms of the SoW, I'll reschedule it when I find time.

Most of the delays I encounter are caused by someone else; either the need to refactor someone else's shit code (that I wasn't allowed to review before providing an estimate, of course), delayed approval for the work, delayed access to resources, any number of external forces. Very rarely do I exceed my estimated *hours* for a project unless changes are requested (but then I'm not exceeding the estimate, either, since I make the client either agree to a new estimate or accept a refund of any portion of their deposit not already applied to the hours I've worked), but all too often I find myself completing projects well past their due date because some resource wasn't made available to me until after that date had lapsed.

Fortunately, I foresaw that unforeseen things would happen when drafting the boilerplate language of my SoW and covered all of those cases. I go over the entire SoW with my clients before starting a project and make sure they know what the triggers are for a re-quote, what will cause the project to be delayed, and under what circumstances they're entitled to a refund of any deposit they pay (e.g. if they back out of the project once I've started work, I'm deducting my hours from that). As a result of that, and perhaps a bit of luck, I haven't had any disputes over project scope, budget, or timeline, and the one client I did have back out of a project simply said they no longer had the budget (they were being sued) and told me to hold on to the remainder of their deposit as they'd be back to finish the project after they lawsuit ended, hinting that, even if that didn't happen, the small sum would make no difference going forward (of course, I'm sitting on that money for now, and if they fully back out of the project I'll insist that they either accept the refund or sign a document releasing the funds to me).

That was one thing that really pissed me off when I was working for someone else; I had no control over external interruptions or delays and it was usually the person interrupting and delaying me who was holding me accountable for all of them. I'm not out to scam anyone, but I always felt like I was when dealing with my previous employer.

Re:Simple methodology

[lgw]

The first and foremost requirement is that when sales/marketing sales that they want 100 features in the product and there is no way that we can doo all 100 features, we have to be able to prioritize them

Yes, that's exactly how all Agile development works. Did that not work out for you?

Now, please tell me how Agile is going to tell me how much effort is going to be required to complete a particular project when the whole Agile approach is to start coding and figure it out as we go?

That's not quite Agile - you need enough of a plan and a design to know WTF you're doing before you start, you just don't need an in-depth design of stuff you won't even touch for 6 months. But you know what the components are, and about how big each one will be, at least relative to each other, and you know where the dependencies are. So how do you know ho long it will take? You measure you're progress! When you know you're 20% done, and it took a month, then you know it's a 5-month project. I've been within 5% on estimates for large projects, even as they changed significantly over the course of the project we could be that accurate on updated estimates.

OK, here is another requirement. We need to put together timelines because Engineering doesn't operate in a vacuum. We are part of a company with many parts all of which have to be coordinated

Schedule, scope, resources, quality. One of these four must be give when estimates are wrong or changes creep in. You have to pick one, or it will be quality be default. Agile is just saying "OK, so scope is what gives".

I've been doing agile-style development since before that word was coined, and I've never failed to deliver the must-haves on time for a project. Really. Honestly goes a long way.

The big problem with Agile is that it was created by Engineers in a vacuum who failed to recognize their responsibilities to the rest of the Company

So, where'd you get your MBA from?

Re:Simple methodology

[Anonymous Coward]

I've been programming for over thirty years, and I can't do estimates. At all.

I've been programming for over thirty years, and I'm pretty good at estimates.

It comes from a practice that I learned at Microsoft. Before Microsoft, I did the usual "Oh, about two weeks" estimate for everything that I was shown.

Design your feature or product. Then figure out how you're going to build it. Then break down that build into tasks that are no less than half a day and no longer than three days. Add it all together and divide by the fraction of a week where you're actually going to be productive. At Microsoft, I was using 0.8 because I'd inevitably lose a day a week to 'stuff'.

Caveats? Sure. Unfamiliarity with the problem domain. Unfamiliarity with the development tools. Lack of control over one's own development time. Lack of access to development resources. And many more. If I'm working in an environment where I cannot use the above technique to get a good estimate of how long it will take to build the feature or product I'm responsible for, then I'm either the wrong engineer or I'm working for the wrong people.

Note that when you do the above, you will come up with a shockingly-long period of time. But when you present that number to your boss, you will have the data to show that are a result of a detailed understanding of your tasks. You'll have your ducks in a row. Your boss will love you because the two of you can talk intelligently about how the schedule can be adjusted by altering the tasks in it.

If your manager wants you to alter the time it takes to complete tasks, you're back to "working for the wrong people".

ignorant hypocrites

[Anonymous Coward]

so, estimating time is a waste of time, but complaining about estimating time is not?

GET BACK TO WORK, MONKEY.

Re:ignorant hypocrites

[gnupun]

so, estimating time is a waste of time, but complaining about estimating time is not?

No, you complain only once, whereas software estimation has to be done over and over for every task... for many years. Software coding/design is similar to solving a maze -- you just can't give an accurate estimate how long it will take to solve the maze. Estimates are good for repetitive or non-creative tasks that have been done for many years and you know exactly how long some task takes.

Estimation in the software world is a scam perpetuated by managers and management to get developers to work extra hard.

Re:ignorant hypocrites

[unrtst]

Holy crap! Are people actually buying into this BS?

Software coding/design is similar to solving a maze -- you just can't give an accurate estimate how long it will take to solve the maze.

WTF? Yes, you can give an ESTIMATE on how long it'll take to solve a maze. Go get a book of mazes; Do 10 of them; Time yourself for each one; You now have min, max, and average times for a maze of that size/complexity. Next time someone shows you a maze, you can make an educated guess about how large/complex it is, then give your best case, worst case, and normal/average estimates. Do the same for programming.

Estimates are good for repetitive or non-creative tasks...

They're also good for creative tasks. I went to college with a major in fine art and worked for years as a monument engraver (etching portraits/landscapes/etc on tombstones). If someone asked about how long it'd take to etc a 5x7 portrait, I'd have a VERY accurate estimate for them. Is that not creative enough for you? How about every single project for every single class for every year of art school? ALL of those had timelines, and every student became quite good at estimating how long each project would take so they could get them all done on time. And you could ask most of them (at least those with higher than a B average) how long it would take them to do a certain thing (ex. sculpt a bust in clay from a live model) and, if they were familiar with that medium, then they could give you a very good estimate.

Estimation in the software world is a scam perpetuated by managers and management to get developers to work extra hard.

If management is asking the devs for their estimate, then how in the hell is it management fault for any of those timelines? Let me put this another way... devs, BE SMARTER. You've been asked for estimates on more than one occasion. Most that are new to development will low ball (stating a best case estimate). If/When you do that, you're just setting yourself up to fail. The best outcome will be that you are on time, and every other outcome sucks for all involved.

If you're not good at estimating, just try, come up with your figure,then double or triple it. If you always come in under the estimate, then you manager may start adjusting your estimates... who cares? Let them. It's their fault if you don't meet their manipulated figure. However, if you're not giving a large enough estimate in order to get your work done, then it is YOUR FAULT if you fail to hit your own estimate!

All that said, there are cases where providing an estimate is unreasonable. On one end of the scale, if a manager is asking for very accurate down-to-the-hour estimates on vague but somewhat small tasks, then it's asking for too much (almost more work estimating doing the damn thing). On the other end of the scale, if it's some grand idea for a giant project with no plan yet, you'll be pulling figures from your ass just as much as he's pulling the project plan out his ass... but it is what it is. Just go big.

A lot of it is just about setting expectations. If you give them high estimates, then you're more likely to meet or exceed their expectations. They may dislike the figure at first, but when you come in under budget they will be very happy and forget all about how high the estimate started out. A high estimate is not "padding", it's setting expectations that you can meet (see the maze estimate above... use "max" and you'll be able to meet or beat your estimate every time, which is what all involved really want out of your estimate).

Re:ignorant hypocrites

[RabidReindeer]

The difference between software and mazes is that with mazes you are doing the same task every time.

I know, for example, that I can take cold iron and have it running most of what makes it a useful production server machine in about 4 hours. I'll sign my name in blood on it. Because I've done that same task over and over.

On the other hand, software is a creative work and creativity implies that you are NOT doing the same task every time and that at best you are guessing.

The problem is, everyone else is second-guessing, and they're guessing wrong. And they often will not accept a realistic estimate. They'll push for a "realistic" estimate, then blame you when you cannot meet it.

Actually developers guess wrong too - they usually think it's going to take half the time and work it really does (based on stats I've seen). But developers could simply allow for that by doubling what they think.

Except that in many cases, as I said, they cannot resist the pressure from outside.

Re:ignorant hypocrites

[naasking]

You now have min, max, and average times for a maze of that size/complexity

Estimating the size/complexity of a software project is exactly where all the error lies. The original quote was correct that it's like solving a maze, you just assumed they were talking about mazes on paper that you could estimate size and complexity at a glance. No, this is a full sized maze that you walk through and have no idea how deep and complex it is.

Re:ignorant hypocrites

[Darinbob]

I am absolutely an expert. Sometimes a lot of of my time is wasted by people needing me to help them out, delaying me from getting my own work done. And I can't estimate. Sometimes because I'm not good at telling people to go away and leave me alone, but usually because unexpected things happen, or I'm being shuffled around between two many projects and can't multitask well, or I'm doing something new I've never done before (but that's ok, people know I'm smart and assume I'll get it all figured out).

And yet, someone will hand me a coredump then ask me how long it will be until I know now to fix the bug is. That's ridiculous, they may as well hand me a sealed envelope and ask me how long to solve the math problem that's inside it.

Even though you're Anonymous Coward, I suspect you're really a pointy haired boss sneaking into a nerd site to spy on us.

Well someone has to do it

[Sowelu]

Business can't plan or talk to customers or have any strategy whatsoever without at least some estimate...that's just the real world. If devs don't give estimates, managers have to make estimates. If managers don't make estimates, business makes estimates. You want devs to do the estimating.

Re:Well someone has to do it

[omnichad]

WHY CAN'T WE HAVE BLANK CHECKS

I thought this was funny, but it probably isn't - especially now that I have typed in all lowercase words to get pass the yell filter.

Re:Well someone has to do it

[pjt33]

Without an understanding of programming, you can't reliably tell how similar a prior project is.

Re:Well someone has to do it

[Marc_Hawke]

An understanding of 'similar' is required though...and without knowledge of the programming there's no way you'll know that. When the programming is all 'magic,' everything is similar.

Re:Well someone has to do it

[penguinoid]

Business can't plan or talk to customers or have any strategy whatsoever without at least some estimate...that's just the real world. If devs don't give estimates, managers have to make estimates. If managers don't make estimates, business makes estimates. You want devs to do the estimating.

I suspect the problem is not so much the estimates, it's that a manager demands an estimate from their underlings, tells them they don't like that estimate and give them a quicker estimate, promises their customers certain completion by the quicker estimate, and then complains when the programmers can't finish on time or have buggy code. That's without the obligatory change in the project requirements halfway through. At some places the estimate is nothing more that bending over for the manager.

Obligatory Dilbert: http://dilbert.com/strip/2015-02-22

Re:Well someone has to do it

[boristdog]

IF managers listened to the developers.

My last major project:
Manager: What resources do you need for this project (Replacing a huge, entirely file-based factory product manufacturing system)
ME: At minimum, I need two more people, plus someone to take over most of my smaller duties. And a couple new DB servers.
Manager: How long will this take?
ME: Two years minimum, since we are replacing an entrenched 15 year old system.

Manager's report to director: He can do it by himself in 6 months.

3 months later:
Manager: Why isn't this finished? You started over a year ago!
ME: I started three months ago, you gave me nothing I asked for and you've given me several other projects since then that you said were higher priority.
Manager: Well stop wasting time and get to work! I told the director it would be done by now!
ME: You gave me nothing. At least let me have a CS temp.
Manager: There's a woman in finance who has a sister that might want the job. She knows Excel really well, she can probably learn databases and programming.

Aaaand I think we've all been there.

Fortunately the sister didn't want the job and I got a great college grad as a temp. I would not have finished the project (in two years) without his help, we hired him after a year, too.

Re:Well someone has to do it

[chipschap]

I was a technically literate manager, having done lots and lots of programming myself. My job was simple: run interference with the client so that my team stayed funded. The team was very happy to let me do that job, which required a lot of travel and unpleasant politics.

In turn I trusted the team. I asked them for realistic estimates to give the client. If the team thought they weren't going to make it on time, I asked them for a heads-up as far ahead as possible, and I would take the news/new estimate for the client. I did not criticize the team, either to them directly or to the client.

They knew they were asked to do their best but software being what it is, they were not held to preliminary estimates. (The only issue was with downright incompetence or negligence, which was very rare.)

It's interesting that I was considered a good manager by the staff and the client, but not by my own management, who said I wasn't hard enough on the staff. Well, sorry, I got results and I kept us funded.

So, was I a "useless manager"? I hope not. I didn't produce anything tangible, and that bothered me, but I hope I played a useful role.

Re:Well someone has to do it

[BronsCon]

He may have been able to write more maintainable, more stable, better tested, and altogether cleaner code, given what he asked. Possibly in half of his worst-case estimate of 2 years. In the long run, the crap that needed to be slapped together to get the project "done", in 4x the time he was allotted, will end up costing the company more than giving him what he asked for in order to do it right. You must be an MBA.

Re:Well someone has to do it

[Vorga]

I'm a manager. I stay in meeting so that you don't have to. I build relationships that can withstand political friction. I cover our screw-ups, into nice language. I convert tech bullshit, into manager bullshit, then into customer bullshit. I convert scope creep into money. I get the contracts signed so we get paid, so the bank gets their mortgage payments. Useless? No. We are all replaceable though.

Good method for improving

[phantomfive]

Estimating is a crucial skill for developers. If you don't have it, you'll make bad decisions. For example, answer the question, "should I use framework A, or should I write some code myself?" If you can't estimate how long it will take to use the framework and compare it to how long it will take to write the code yourself, then it is impossible to make a realistic decision. Similarly, "should we write the code using method A, or write the code using method B?" If you don't know how long they take then you can't make good decisions.

This is a trick I learned from agile (although I'm sure it's been around longer than that). Each time you make an estimate, pay attention to how accurate it was, then next time adjust and make a better estimate based on that. Estimating shouldn't take much of your time.....if it does, then you're doing it wrong.

These guys seem to be complaining that their managers aren't doing anything with the estimates. Which probably means they don't understand what the managers do (but it could also mean their managers are bad).

Re:Good method for improving

[phantomfive]

Good managers are almost as rare as honest politicians.

Then teach your manager to be better. Reject the corporate structure (it may shift many times while you work at a company, even though not much else changes. That is an indication it's not particularly real), accept that improvement can come from even the lowliest programmer, and then help your manager do better.

If you think that your manager is completely bad in every way, you're not going to be able to help him. You need to be honest, recognize his strengths, and help him overcome his weaknesses.

You'll probably also need to overcome some of your own weaknesses, unless you are perfect. I can tell you one way to not accomplish anything: start a twitter campaign telling the world how bad your boss is. That will not make your system better.

Re:Good method for improving

[Great Big Bird]

This is a very insightful post —the original poster was asinine to even suggest that we need to get rid of them. For "software engineering" to grow up, we need more accurate means of pricing projects — otherwise you have no business.

Parkinson's Law

[jrq]

Estimates are necessary, so that you can determine project cost and (sometimes) duration. However, most projects succumb to my amended Parkinson's Law which states "work contracts or expands so as to fill the time available for its completion" .

Predicting the future is hard

[Krishnoid]

Joel Spolsky has a take on this problem, called Evidence-Based Scheduling, which tracks past estimates against their deliveries, and uses that to improve future estimates.

Re:Predicting the future is hard

[Marginal Coward]

I've seen that sort of thing used fairly effectively. If you keep track of how long past projects took, it's fairly easy to estimate a new project based on an estimated ratio of how complex the new project is compared to the most similar past project(s). This takes practice and a bit of a knack to do well, but it doesn't take much time, and it's certainly better than nothing.

What doesn't seem to work well, in my experience, is breaking down a project into microscopic detail and individually estimating each detail, e.g, the "Microsoft Project" approach. I can understand the appeal of that sort of thing, but it's almost impossible to use a data-driven approach at the microscopic detail unless you use some sort of "Personal Software Process" tracking tool - which very few people want to actually do because it feels like you're instrumenting yourself as a lab rat in someone's twisted experiment.

In any event, having a plan and a schedule, though inevitably imperfect ones, helps motivate everybody and helps keep things on track. The more enlightened managers of the world will allow these things to be revised along the way as needed, as their imperfections get revealed.

Re:Predicting the future is hard

[Sklivvz]

I work for Joel Spolsky and we have a single estimate: 6-8 weeks.

Estimation is basically useless because it has the unspoken assumption of "perfect design", which is an invalid one: software development is not a repeated exercise, it does not have consistency.

Having a single estimate does this to your projects: much smaller things, just do them; much bigger things, they are too big so don't do them, things vaguely in that scale, do them iteratively but define what you are trying to achieve in writing so you know when to stop.

No project managers, product managers work at a way higher scale (decide which are the projects worth working on).

Re:Predicting the future is hard

[RabidReindeer]

Very often the original system was hacked out in a couple of days/nights by one or 2 heavily-medicated (caffeine, alcohol, whatever) people.

It more or less did the job even though it has one or 2 really awful (but infrequent) bugs and is difficult to maintain or expand.

Then one day someone decides it's time for the Second System (see Fred Brooks).

They put together a team and a schedule and - like as not - spend a lot of money on fashionable faddish tools and consultants and a lot of time coming up with bells and whistles (a/k/a the Second System Effect - see above).

65 days into the 90-day schedule, they realize that they don't have any actual working code, just an immense collection of stick-figure UML diagrams, panic and put all the programmers to work coding on 100-hour weeks.

Guess what the end result is? Hint: read the papers. Somewhere between 66 and 75 percent of all major software projects fail.

Re:Predicting the future is hard

[dcollins]

So is the system in the linked 2007 article no longer used, never was used...?

Depends on what "delivered" means

[mbeckman]

I've worked on a lot of software projects that delivered the original specified product on time. Sometimes the target changes, and the stakeholders need to be willing to give developers the extra time they request to meet the new objectives. Too often I hear, usually from upper managers, "We are still shipping on schedule. Tell the developers to work harder." Of course, that's not realistic, and the result is a predictable "failure to deliver." Alas, the developers get blamed, when it's really management's fault.

On the flip side we have so-called "agile" development teams who simply define a deliverable as whatever they've completed on delivery day. These developers rarely tell management until the 11th hour that what they're going to deliver is below spec. Agile development has its strengths, but this aspect is a giant weakness. The solution is not to eliminate schedules. It's to adapt them to changing conditions and be proactive about slippages.

My rule of software estimates

[sandytaru]

The amount of time it will take to complete a project is inversely proportional to the perceived difficulty of the project. This also applies to tasks and deliverables within the project. The project that you think will be easy and fast will be neither. The project that you think is going to be a tangled nightmare will turn out to have some surprising shortcuts and simplifications that make it not so bad.

Part of this is the stupid human trick factor - if we think something is going to be difficult, we approach it differently and with more caution. As a result we're more able to identify things to make the project go smoother. Conversely, if we think something is going to be easy, it's because we've only determined one path to approaching it, and if that path turns out to be non-viable, we have an immediate delay as we scramble to find other solutions that will work.

There's also the psychological factor at play for the customer - if we say something is going to be ghastly and difficult and will take us a long time ( because we think it will) but then turn around and deliver it ahead of schedule and under budget, we look good and feel good. This does not work if you say something will be ghastly and difficult but secretly think it will be easy and fast, however.

Re:Estimates are for planning - not penalizing

[phantomfive]

When estimates are accurate, projects get done on time. Then business trusts programmers, money is made, and life becomes better.

People do not predict well and it is not smart to use it against them.

You can improve. Most programming we do is boring, and similar to something we've done before. For things like that, you should be able to make your estimates reasonably accurate.

Sometimes projects have large unknowns, and it's impossible to know how long it will take. In that case you need to propagate that information up to the people who need it, either managers or salespeople or whatever.

First, get your story straight...

[QuietLagoon]

...Software project estimates are too often wrong, and the more time we throw at making them, the more we steal from the real work of building software...

...developers' back-of-the-envelope estimates...

From the above two quotes, it looks as if the author does not even know if too much or too little time is spent on estimates.

.
No wonder his managers are frustrating with the time estimates provided. The guy has not a clue.

It's research

[plopez]

Programming is basically research and development You are building something which has never been built before. As such how do you estimate something if you have nothing to compare it to?

Actually though I have a limited amount of experience using Function Point Analysis. Some of the aspects of it are old, it was created before web apps became pervasive, but that doesn't matter. The principles are the same and there are dimensionless constants you can use to tune the FPA model to your organization. Eventually you end up with an empirical model of your programming team, development environment, tool suite, management efficiency, complexity of the problem domain, people quitting or dying, etc. You need to avoid over fitting however and recalibrate from time-to-time.

Agile keeps story points and velocity which serve the same purpose as the dimensionless constants of FPA. Unfortunately managers try to translate them to mythical man hours. But if you keep velocity or function points, whether you are using Agile or not, you should be able to get some idea. You have to be a bit scientific about it.

That and 'T-Shirt Sizing' is about as good as it gets.

Not just software

[CastrTroy]

I always here that software projects are often late and over budget, but I don't think it's worse than any other industry. I've seen countless examples of construction projects that ran over budget and took longer than expected. Often the reasons for this are the same. Either the requirements changed halfway through, or the project was made more complicated than it needed to be to accomplish the task. There's a few bridges in my area that have been huge boondoggles in the past decade, and they all try to look impressive, where a much more conservative design would have be easier, cheaper, and faster to build, and still would have solved the transportation problem. But everybody wants a bridge that looks pretty.

Projects that deal with a small workload and don't have changing requirements are much more likely to stay on budget and on time. This is how things should be broken up. Build small pieces and deliver the pieces as they become complete. Don't set out to build an entire 5 year task as a single project.

Hmmm ....

[gstoddart]

On the one hand, it's pretty much impossible to do any project planning with no estimates.

On the other hand, some things are impossible to estimate until you do them.

Years ago I worked with a manager who kept repeating that bad estimates were a project risk and we should give good estimates. We kept telling him that an estimate is, by definition, based on incomplete knowledge and before you have done the work and that if he had a time machine we could give him better estimates.

If I knew exactly how long it would take, and what unforseen things I'd be running into ... it wouldn't be a frickin' estimate, now would it?

People treat estimates like you're expected to have perfect knowledge of the future, and then build their world around those estimates.

I have seen a tremendous amount of bullshit and stupidity from people who do not understand what an estimate is, and how it's done.

I don't think you can get rid of estimates entirely ... but management needs to stop being so stupid about how they interpret them.

If we could tell you for a fact exactly how long it would take, it wouldn't be a fucking estimate.

I rank how people do estimates right up there with how some PMs want you to track your time ... once had a PM say he wanted me to account for my time in 5 minute increments. And I told him in no uncertain terms that would mean 2 out of every 5 minutes would be spent documenting what I'd done the last two minutes, and there would be an additional 1 minute of lost time in each 5 minute increment doing to context switch back to what I was working, and that effectively 60% of my time would be wasted on his stupidity.

And then I told him to piss off.

Re:Hmmm ....

[david_bonn]

A couple of observations.

There is a hellacious difference between an estimate and a deadline. A lot of the problem is abusing estimates and quietly or unconsciously turning an estimate into a deadline. The process for producing the two is totally different, or at least should be.

One very good manager I had would never directly ask me for an estimate. He would ask me how long it would take to figure out how long it would take to code x. If you can make a reasonable estimate of the size and complexity of the problem you are trying to solve you can begin to make a reasonable estimate of how long it will take. Without that information you are pulling numbers out of your ass.

One other good manager I had emphasized to me that estimates were, well, estimates. It was as big a problem if I always overestimated as it would be if I always underestimates.

Selling ourselves down the river

[Anonymous Coward]

Research into procrastination has noted that people have much less concern about their future selves than their present selves and are willing to sell their future selves down the river for the sake of present ease. But when the present marches into the future, and we are confronted with the work that our past selves refused to do, we pay the price in unmet deadlines, all-nighters and general torment.

http://www.nytimes.com/2015/01...

The result becomes one of people providing estimates that allow them to screw off in the present and postpone the agony of late nights, weekends and looming deadlines for taking it easy today. This seems to always happen in our organization. A current project has had the hardware in place and ready for the teams to load their software on and configure the system for 6 months, but they waited until 3 weeks before the project is due to start and now they are changing the project design to make it easier on themselves in the present as well. Estimates = wasted time. Just do it.

"It's hard, so we won't do it"

[mveloso]

Part of being a software professional is understanding how long it takes for you to do things

If you don't know how, go read a book. It's not hard. The money that's used to pay you depends on your estimate being somewhat correct.

Imagine how upset you would be if you asked for your paycheck, and payroll said "we're not sure when we're going to pay you, but it should be sometime soon."

Re:"It's hard, so we won't do it"

[Gliscameria]

Thanks, I was beginning to think that software developers were completely dissociated from the rest of the world. Writing the software isn't the only part of the business. Marketing and sales are going to have to know about when things are done so they know when to have everything ready, and that's not even getting into the financial and executive parts of things. Not to mention you are basically saying that you have no idea how much the project is going to cost. Really, if you can't at least provide an estimate then it's either not your job to do so or you need to find a new job.

Re:"It's hard, so we won't do it"

[Anonymous Coward]

I don't know about you but I'm not making cookies. Sounds like you're not doing any design either...or implementation in the real world when bugs appear, refactoring is required, etc.

Re:"It's hard, so we won't do it"

[gstoddart]

and that's not even getting into the financial and executive parts of things

Yeah, and tell us, what is the track record of the financial and executive teams ability to prognosticate?

Yes, you need to be able to estimate to run the business.

But let's not pretend the average CEO, sales wanker, or marketing idiot has ANY better track record at making guesses about the future. In fact, in my experience, they're overly optimistic, not founded in anything real, and mostly pulled out of their ass of based on what Gartner tells them.

We can give you an estimate, but people have to understand that an estimate inherently carries uncertainty, and that they're equally inept over the long run of estimating the parts they're responsible for.

I've lost track of the times I've rolled by eyes when a CEO tells us what six months down the road will be ... and they have the ability waste far more money on fools errands and bad predictions.

We're not dissociated from reality ... we're the ones trying to explain reality to people who live in fantasy land.

But don't act for a minute like our estimates carry any more risk than those bullshit sales figures the idiots at the top are making.

Tilting at Windmills

[Cytotoxic]

I used to tilt at this very windmill myself. It took me a long time to realize that people really, really need to hear an answer to "how long?". When we were in startup mode a long project was a matter of a few weeks. Everything had to go into production immediately. So we got used to banging stuff out in small chunks and doing it as quickly as possible. Entire project timelines were completed in less time than it takes to draft a proper requirements document.

But as we grew in size, the new people were not happy with our development team. Even though they would ask for a new report at 10am and have it in production before they returned from lunch, they still felt like we were not responsive. It was one of those reports that began to help me understand the psychology.

There was a problem with a very complex Crystal Reports document one morning. The Director of the department called to let us know about it. I told him we were already working on it and would have it fixed as soon as possible. "When will it be fixed?" he asked. Well, I had no clue. We had only learned about the problem 10 minutes before and hadn't figured out what was broken. So I explained this to him and told him that we had this as our top priority and it would be fixed as quickly as possible. I certainly thought that should make him happy. Well, it didn't.

He was rather pissed that I wouldn't give him a timeline. The day before we had made some changes for him that took about 2 hours, but he was upset that we didn't let him know how long it would take. Being an engineering type, when I hear "how long will this take?", I hear a request for a certain degree of precision. The problem with short projects like this is that by the time you have enough information to give an honest estimate, you are pretty much done. Maybe you have 15 minutes or an hour to go, but nothing worth reporting to anyone. Well, after explaining my position a few times and just making him more angry I finally gave up and just gave him a made-up deadline of Thursday afternoon. He was perfectly happy. I had just spent 15 minutes trying to explain to him that we were working feverishly and would be done as soon as we could (which would likely be a couple of hours, but who knows) and he hated that answer. "We'll have it for you in 3 days!" made him very happy. Even though his production was impeded by the lack of this particular report. From a human psychology standpoint he would rather know that it will be done in 3 days, barring delays, than not know when it will be done and have it in two hours. I personally think that is a dumb way of doing things, but I am the outlier, not the director.

After that learning experience we began implementing a more comprehensive SDLC process and providing timelines for projects. Everyone was much happier with the development team after this. Even though their projects went into production much more slowly. They loved the perceived control that having a timeline gave them. We developers know that these things are basically fictional documents - just educated guesses really - but it provides real customer satisfaction, so we keep with it. In fact, we kept evolving this idea into more and more involvement from the business unit as we moved into Agile and SCRUM methodologies.

I would say that unless you are working in an organization of less than 25 people, providing timelines is an absolute requirement from a purely human standpoint. This comes from hard experience - even though I think that everything about a timeline is probably B.S. and all of the effort that goes into preparing one would have been better spent solving problems and building something useful. In the real world you can't ignore the psychological needs of the group.

Re:Tilting at Windmills

[Ryn]

It allows your boss to have a scapegoat. YOU are giving him the estimate, he's not having to come up with one. "They told me X, it wasn't ready in X, we made plans, I even added 10% padding and they still did not deliver".

Re:Tilting at Windmills

[Tom]

From a human psychology standpoint he would rather know that it will be done in 3 days, barring delays, than not know when it will be done and have it in two hours. I personally think that is a dumb way of doing things, but I am the outlier, not the director.

The psychological issue is that you don't know, but you have a hunch, you have some insight. You know it's probably going to be a few hours.

But for non-techies, all this stuff is a total blackbox. When you say "I don't know" they panic, because for them that means anything from a day to a month or maybe infinity. Uncertainty is a horrible psychological state and people try to avoid it. It's an instinct. When you don't know if that shadow is a monkey or a lion, it's better to panic just in case.

By saying "three days", you give him certainty. Now he knows the shadow isn't a lion.

Re:Is this really a problem unique to devs??

[duck_rifted]

If non-trivial problems could be solved within a guaranteed time frame, then you'd have already have a doorstep delivery date for your starship. Good management can make sure that core features are delivered on a schedule and help ensure that the schedule is not actually impossible to meet. If the schedule is not lenient enough, then the best management in the world can not guarantee delivery while also providing a guarantee for stability, security, and performance. Were this a simple matter, then the service of software development would not be worth so much.

You have three choices. You can give me enough time to finish the project correctly and receive an awesome result, but you will pay for that extra development time. You can give me too little time to deliver a quality result, but you will pay me for patches. You can give me too little development time and opt out of paying for updates, but you will pay for it when there are problems. Choose well.

There's a very old, very over-used saying about technology that people must be forgetting. It comes with three possible qualities: good, fast, and cheap. You may choose two.

Re:Is this really a problem unique to devs??

[bughunter]

No, it's a very common problem in engineering in general, and not unique to software. But the reaction "let's eliminate estimates" appears to be.

As an engineering manager, I learned the hard way many times how estimates turn into deadlines. Your estimate is reported to the manager's manager and so on up the line, and someone uses it in planning their shit.

Your estimate, in which you did not build any schedule margin, then becomes an item in the critical path of someone else's plan, someone who didn't build in any margin either, or —worse— who was pressured to make a completely fictional "plan" which is really just a backwards-calculated paper justification to "prove" that a job could be completed in an impossibly short period of time by assuming nine women can make a baby in one month and things like shipping, reproduction, and quality assurance take place in zero time. This "plan" makes upper management happy. Temporarily.

You, leader of a small team that is working merrily away, accomplishing real work and solving the occasional unexpected problem (OEM pinouts were wrong, widget zeta delayed in shipping, amplifier stage behaving like oscillator, etc.), are asked for a status update. Because of your unexpected problems, your estimated completion date is now two weeks later than your previous estimate.

Now the middle manager, who knew he wasn't going to meet the "plan" he was forced to develop, now has someone to place the blame on. He knows he's going to be in the path of a metric fuckton of shit, but he's placed himself uphill of you.

It's clear even in TFS that the real problem isn't estimation, it's poor program management, lack of requirements management, and often also marketing-driven decision-making.

In other words, the same old shit.

Re:Estimates are for planning - not penalizing

[Opportunist]

If you penalize me for going over time and budget, my next estimate is 30 eons and 200 trillion USD.

Watch me come in well inside the budget.

Get rid of time estimates?

[Yunzil]

Peter Molyneux is that you?

Wow!

[endus]

Engineers think project managers and deadlines are a waste of time and a pain in the ass, while project managers think they are essential. Now that's what I call news! Whodathunkit!

This is business. Management wants to quantify everything to manage resources, manage spend, control cost, maximize profit, etc. It makes perfect sense at the same time that it doesn't really jive with how engineering works a lot of time. One thing for developers to keep in mind, though, is that *doing* something is never as important as *telling people* about how you did it. Metrics mean way more to the people who sign your paycheck than the code you write does and you should design your metrics accordingly.

The other component is PMs themselves. How many really good PMs have you worked with in your life? Grand total of 1 for me. Most PMs are people who don't really understand technology and have created a whole system of super-important metadata to "add value" to the process. When it's done properly a PM can help a lot, but mostly its just blustering and wasting everyone's time. These people want to protect their jobs, and their jobs are defined by timelines and metrics.

Re:Without estimates you can't budget...

[JWSmythe]

Lets see... What would they say? This is the one-sided conversation, since it doesn't matter what you say anyways.

"Ok, we can accept that estimate."

"Ya, ya, ya, whatever."

"We'll have that information to you by the start of the project."

"The information isn't ready yet, we'll have that by the time you need it."

"I thought we had that to you already. We'll have to check with the information source."

"The PMs have some changes."

"Here's the information, but there are some small changes."

"No, those are small changes, they won't impact the timeline."

"No, you can't have more time, we already made commitments."

"The PMs have some changes."

"What do you mean you won't have it in on schedule? You agreed with the initial estimate."

"You're going to stay here until it's done, I don't care how long it takes."

"I don't care that you've been in the office 30 hours straight, this is your fault."

"We're hiring an off-shore company to help you with the project. Get them up to speed."

"The PMs have some changes."

"Since we have the off-shore team, we need to cut your department back."

"I read an article saying Java is the future. Redo it in Java."

"What do you mean we're waiting on the off-shore company?"

"We fired the off-shore company. You're good, you can get it done in time."

"Ok, hire more people into your department, but we're only offering half the salary, and no more bodies."

"Why is this project so far behind? Don't you know what you're doing?"

"The PMs have these changes."

"Why aren't you done? We're weeks from the deadline!"

"You didn't meet the deadline. Don't you know deadlines are firm. We have commitments."

"I don't want excuses, I want results."

"You and your idiot team are fired. Get out of my building."

[2 months later]

"We need you to come back and finish the project. We need it by next Monday, that should be plenty of time."

"Here's all the new specs. They should be easy to do."

"What do you mean total rewrite, it's only a few chances. You are an idiot. Get out."

[1 month later]

"We need you to come back and finish the project. We need it by" {click}

"We need you to come back and finish the project. We need it by" {click}

"We need you to come back and finish the project. We need it by" {click}

"Why do you keep hanging up on me?" {click}

After which managers toss the "bad" estimates

[msobkow]

My experience has been that management comes to the developers for estimates. They provide those estimates to the end users. The end users bitch, whine, and complain that they need it to be done in half that time.

Management then comes back to the dev team and tells them they've agreed to get the project done in half the time that was estimated.

Then both management and the user community bitch when their "estimates/targets" aren't met, and who is blamed for the issue?

The developers.

The developers always are to blame for computer problems, never the bad specs, the conflicting specs, the unknown variables, the use of "new technology" that some vendor flim-flammed onto the department/team, or anything or anyone else.

Screw 'em. Now that I'm retired, I'll never have to give anything more than the most vague ballpark estimate of how long it will take me to do something ever again. Instead, on my pet project, I just bullet point some of the things I intend to work on next -- and even that is subject to change. The lack of stress and the freedom to live my life according to my own whims and needs has proven an invaluable source of improvement in my "quality of life."

What a shame I've never encountered a job that would let you do that.