Why Crunch Mode Doesn't Work

so sue mee writes "There's a bottom-line reason most industries gave up crunch mode over 75 years ago: It's the single most expensive way there is to get the work done. When used long-term, Crunch Mode slows development and creates more bugs when compared with 40-hour weeks. Evan Robinson has an article at the International Game Developer's Association site talking about the harsh realities of crunch time, and why the gaming industry should stop using it." From the article: "It is intuitively obvious that a worker who produces one widget per hour during an eight-hour day can produce somewhere between eight and 16 widgets during a 16-hour day. As we've seen, that's the essential logic behind Crunch Mode's otherwise inexplicable popularity. But worker productivity is largely dependent upon recent history."

Comments

Obvious

[mind21_98]

In my opinion, if you need crunch time to finish a software product, you've done something wrong. It's much better to encourage a culture of efficiency and program stability than to make up for it by forcing it down people's throats close to the ship date. *shrug*

Re:Obvious

[Anonymous Brave Guy]

To be more precise, if the development staff need crunch time to finish a software product, then chances are that the management did something wrong.

Re:Obvious

[badasscat]

In my opinion, if you need crunch time to finish a software product, you've done something wrong.

You're not quite seeing it the way the publishers see it - it's not crunch time as in "holy crap, the deadline's coming up, we'd better work overtime!" It's more like a standard schedule. It's actually built in to the project timelines. It's not a surprise. You know that four or more times a year you will be working 80 hours a week, and management knows they can create milestones based on that schedule.

The way this came about most likely was accidental... you often hear stories from days of yore about things like the original Pac-Man for the Atari 2600 being developed in 6 weeks because Atari realized they had the license but had no product for Christmas of that year. The problem is, as even this article points out, crunch time does work in short bursts. The publishers learned this fact, and came to rely on it through a series of these happy accidents, where workers who were otherwise excited about what they were doing were asked to put in extra time on projects to make up for mistakes... and they did it successfully.

Once you start to make it less of an anomaly and more of an everyday thing, though, that's when productivity starts to drop and discontent starts to rise. Management doesn't really see it this way, though; they only do the math and figure that more hours equals more productivity at the same salary level. Obviously, this is poor management, because not only does it not hold true after a certain period of time, but it ignores other inevitable problems, like the incredibly high turnover rate that results... which costs a huge amount of money. (Recruiting a new worker for a full-time, non-management, white-collar position costs approximately $80,000, last I read, including lost productivity during the replacement period, training, new benefits, the actual search and interview process, and other miscellaneous HR costs.)

The game industry is young, as are most of the managers and even CEO's involved. (When I worked for a game publisher, my boss - the CEO of the company - was younger than I was, in his late 20's.) They simply do not have any real management skills or training. They are wasting huge amounts of money without even realizing it, in fact believing they are doing the opposite. They think they have stumbled upon some magic formula for business that nobody else has ever thought of - simply drive your workers as if they're slaves! They don't know that everybody else has already tried this and figured out it doesn't work.

Eventually, as the industry matures, this will likely change... though by how much is anyone's guess, as it's a culture at this point. But already you're seeing quite a bit of consolidation as poorly-managed companies get merged into larger, better-managed companies - or simply go out of business. But even heavyweights like EA obviously have their problems, and once the growth spurt we've seen over the past decade or so subsides, they will have to deal with their management issues too. (If they're smart, they'll do it now, before it's too late.)

Re:Obvious

[cratermoon]

Quite true. And one of the mistakes management makes is hiring people who can't or won't estimate correctly. On prefectly reasonably well-run projects I've been on, things don't get done because some hot-shot lone coder tells management he (and it's always been guys in my experience) can do in half a day what will actually take 3 days from start to completion of integration and testing. Sometimes they just plain don't actually know how long things take. Often, the only portion of the task they think of as taking time is typing time. These wishful thinkers typically type in the change they think is right and check it in and call it done.

If this sounds more like the coder's fault, let me assure you it's not. Miss an estimate once or twice, OK. Nobody is perfect, you learn and move on. Consistently fail to estimate correctly and management needs to take notice. Either pull that person out of the critical path (so missed deadlines don't hurt so bad) or have someone else who has shown better estimating ability size the task and use that number.

If, after 12-18 months of work, management is still letting someone who makes committments he can't keep blow the project's schedule, that's pretty much professional malpractice in any real profession.

Re:Obvious

[cratermoon]

Right. I left out an important point. Estimating is hard. Estimating something you haven't done before is nearly impossible. Estimating something neither you nor anyone else has done is completely impossible.

When management asks for that kind of an estimate, the only reasonable, professional, response is, "I don't know, how long can you give me to research the problem to determine the scope?". Management needs to accept that, and be prepared to respond if the estimate comes back "too long".

In games development, (and plenty of other fields) management doesn't accept that answer. The deadline is set at next Thanksgiving. It's impossible for them grasp the concept that some things can't be estimated.

Management's "solution"-- just push as hard as possible to get what they want, and punish the failures. Reality intrudes regardless, games get delayed or features dropped. Sane people would plan and prioritize features. If task X can't get done in time T, either that feature gets dropped or another, lower priority, feature is dropped to give time to finish task X. There is no "work twice as many hours so you can get both done".

Similarly for unanticipated stuff that comes. Change happens. A perfectly reasonable estimate that can be agreed on by multiple programmers can turn out to be wrong when something else comes up. It's no use railing against fickle customers or glitchy hardware and libraries. But that's another essay.

Re:Obvious

[OverflowingBitBucket]

What I want to know is if there are any big game companies that have actually done away with crunch time. .... With so many reasons to trash crunch time, why haven't we seen this happen??

Huge and steady supply of warm bodies and marketing-driven deadlines being prioritised over software stability. It is cheaper to work star-eyed programmers into the ground and discard them at the end when the overall quality isn't a priority. It's disgusting.

SSDD

[BrookHarty]

My boss tried that on me again last week, just put in a couple more hours, take on another task, work smarter not harder...

WTF are these people thinking? I'm working a few more hours on the last damn project you gave me, i'm already working smarter since you downsized our company every year for the last 6 years. Take on more work?!

Then he has the nerve to say, if your working more than 50 hours, we can get you time off. Ahem, 60 is the norm there bucko. Tells us he wants us there 8-5 while we are also working maintenance and weekends. Ya, thats gonna happen. Last I checked with HR im salary, you cant make me clock in and out.

Crunch time seems to be the norm. Either your working mega hours, or you are in a quiet time before something breaks. Like Sys-admins are like fire fighters, you automate as much as you can, and when something does break, you work your ass off.

Trying to work as a corporate whore, I mean sys-admin and try to balance a personal life isnt working out so well. Having to deal with PHB's who think computers are magic fairy dust and can make anything happen is slowing killing my soul.. Then come home to a wife who says I'm not spending enough time with the kids, ARGH!!!

So tempted to switch my job for a differnet crunch time. Flipping burgers during rush hour.

American Beauty is a great movie, to say fuck it and go be happy again.

I'd join a union, but all these ass-hats want work and burn out, and of course they do burn out. Happy hour can only keep you going for so long....

Re:SSDD

[Elwood P Dowd]

Look for other work with better hours.

Don't stay there poisoning yourself.

When you find work with better hours, tell your boss you want better hours or you're taking someone else's job offer. If you can't do that, ask your boss to consider reducing your hours in exchange for a pay cut.

My stepmom did that, and it worked to everyone's benefit. She asked for a 10% raise and a 20% reduction in her salary in exchange for a four day work week. Would your boss like to reduce the amount he spends on salary?

I'm a gamedev

[Anonymous Coward]

And I was at work until 2am last night. I wish I knew a way around crunch time... but with marketing, disk pressings, public betas, christmas, and all that fun stuff, it seems impossible to avoid.

If anyone has any good ideas I'll pass them onto my manager :)

Yeah, right

[c0ldfusi0n]

Tell that to EA Spouce, she knows.

Working longer just doesn't produce more

[WouldIPutMYRealNameO]

One of the best places I worked was a place where the boss understood that happy workers are productive workers. And workers that are required to work longers hours simply don't get more work done.
This guy kept us happy with relatively cheap methods - decent coffee, free biscuits/cookies and taking us out to lunch/dinner on a regular basis.
Even under stress times he told us to leave for the day. Interestingly this made people want to stay later and work harder.

At other places I have worked there has been an expectation of "we're near deadline, work an extra few hours every night" - for me this doesn't work. I get less done in more time, I end up sitting watching the clock or reading Slashdot, and resenting staying at work.

The solution to getting things done on time is simple
1) Hire smart people who get along with each other
2) Don't push them, let them work hard for 8 hours and then go home
3) Don't choose arbitary dates for shipping
4) Don't let features creap into the spec.

But managers don't seem to understand this.

Why there's a crunch mode

[Brandybuck]

There's two main reasons why there's a crunch mode. The first is because someone somewhere managed to do it sucessfully. Management isn't stupid enough to cut development time in half, but shaving a day or two off of a three month schedule isn't that big of a deal. So everytime you manage to get your project out in time they shave another couple of days off the next project's schedule. And even if you manage to avoid that, there's always another team somewhere that did manage to succeed with an insane schedule, "so why can't you?"

The second reason is that the schedule you've estimated and the schedule the market demands live in two different universes. Management isn't stupid, they KNOW it's costing them more to make you do crunches. But the market says they need a product out in three months and not six. So you're given an insane schedule.

Re:Why there's a crunch mode

[vega80]

Nope, there is almost always one reason for crunch mode: Feature creep. On every project I worked on, schedules were made for design doc from the publisher client. These schedules are generally good, even allowing for some slack. Invariably, as the shipping date approaches, some similar game will be released, and the client will say, we need to have that feature or this feature. Generally, this happens around E3. The developer management can say, "no, that's not in our contract" and perhaps not get another contract from the publisher again, or bite the bullet, and the team will have to crunch to put what the client requested into the product, or at least as much as they agree to.

Re:Why there's a crunch mode

[rossifer]

Management isn't stupid enough to cut development time in half, but shaving a day or two off of a three month schedule isn't that big of a deal.

We had been working for 15 months of a 24 month project when the newly hired marketing team finally presented a complete product spec (the previous marketing team had been fired because they couldn't produce a sane product spec and we spent our time running around in circles). We did several estimates on the spec the provided and came up with a range of estimates for a nominal schedule of 22-24 months, +/- 25%. We also figured that we could reuse a good sized hunk of what we'd already written and guessed that that would save us about 6 months, leaving us with 16-18 months of work to complete.

But the original schedule had us finishing in 9 months (remember, we were already 15 months into this "two year" project).

They chose not to alter the schedule. Or substantially alter the spec. i.e., management was stupid enough to "cut the schedule in half". Hilarity ensues.

We ship two months late, and what we ship sucks. Most of the internal data-management frameworks were left half-baked so that developers could spend more time working on screens and reports, which means that even minor changes are painful; performance is pathetic; the UI abuses the user in several ways; and it has errors in data management that can corrupt customer financial data.

But you can't teach management a damned thing about how to write software. They're the ones in charge, so they're the ones who have to tell us how to do our jobs best. Right?

Regards,
Ross

This is silly

[BlightThePower]

I particularly object to the "what management wants" paragraph. Unfortuantely I detect a coder's tendency to try to over-rationalise the world here. Their analysis does not provide the "essential logic behind Crunch Mode's otherwise inexplicable popularity". I don't believe the cruch is what management wants at all, the problem is simply poor planning. All they want is the software to specification by the deadline. If you can do this without the crunch then obviously this is a Good Thing, but if you can't, thats business.

The cruch is a response to a problem (that may be flawed) but its not the real problem. This is somewhat different from the issues that people like Abbe and Ford were discussing which was the simple problem of extracting sustained and predictable productivity from their workforces.

The difficulty is that the work processes surrounding the writing software appear to be still relatively poorly specified, which is why there are many methodologies -- which attempt to produce sustained and predictable patterns of productivity -- but no silver bullet as yet. A hint to this is that of course Ford was in the vanguard of people who went out of their way, at considerable expense, to enforce a well-specified process behind their output. He had to have that in place before his adjustments to working hours made any sense; the author's analysis of Ford's management style misses this vital aspect out.

Re:This is silly

[BlightThePower]

The specifics of how Ford managed people are not important in this case since all we are debating are ludicrous work hours...Why should the gaming industry be any different?

I've just told you why the gaming industry is different, because software development in general is a poorly specified process unlike industrial processes which are incredibly accurately specified; thus people can't plan accurately just how long a widget is going to take to produce (This is actually stated in the article but they don't seem to draw any connection between special difficulties in the measurement of IT productivity and the special nature of IT working hours; I'm saying they are closely related). As a businessman this causes you to enter into a world of trouble because your core process is unpredictable. This is worse for gaming than most programming because they are a retail business, you don't earn money for games that haven't shipped yet. Put it this way, if you say a project will take 40 hours of productivity to produce and you give your workers a 40 hour week, but half way through you discover your project has changed and you need 50 hours of productivity, then you've missed your deadline.

The problem for software development is that this potential for change is constant, its as true in the first week as it is in the last week. Furthermore, you don't really know what form of work the change will engender, it may be donkeywork, it made need great creativity to come up with new concepts, it may be tracking a bug. A huge amount of money is spent to make sure industrial processes don't change in either their nature or their duration. Ford was the leader in this and when he'd managed to make things so predictable then he could measure productivity accurately and thus could adjust working hours safe in the knowledge of what the outcome would be. He spent 12 years experimenting on this, the gaming industry hasn't been at its current strength for anything like 12 years. Making people work crazy hours even though their productivity will vary widely as a result is a response to this uncertainty and planning difficulty. Your EAian 60 hours gives you considerable slack. Some 60 hour periods will be easier on the workforce than other 60 hour periods though. The only thing that makes the gaming industry different is that they seem to be able to hire people prepared to do this for whatever reason.

Crunch works. Death march does not.

[LordZardoz]

In controlled doses, Crunch works. It is perhaps even necessary. Shit happens, and you have to meet a critical deadline, so you work late for a few days in a row.

Death marches dont work. They just result in angry workers and an increased rate of errors.

And yes, I am a game developer, and I know all too well what I am talking about.

END COMMUNICATION

Still common, but not intentional...

[Taulin]

While 'in the books' the design of a project should be done far before release is near, even if you are doing iterative development. However, many MANY companies still like to design as they go. It is not the developers fault most of the time since many companies create software for clients who know nothing about making software and add change requests, or sneak changes in by claiming the developers mis-interpreted the specs they gave them. Added to this are ambitious managers/developers who want to change a feature at the last second after seeing that their own original design is crap even though we told them from the beginning. All of this is what makes crunch mode. Changed specs with a static deadline.

Re:Still common, but not intentional...

[bskin]

'Designing as you go' is the opposite of design. The whole point of designing is to plan out what you're going to do ahead of time. It's a shame more software companies don't seem to realize this.

Re:Still common, but not intentional...

[cratermoon]

Yes, if you design as you go, you get something like a beehive, a termite nest, or an anthill. And we know how how badly those meet the needs of their builders. Oh wait...

Hey, if 24 hours a day isn't enough time...

[nganju]

Then work nights.

Crunch mode does not work in the Software industry

[Syncdata]

Crunch mode only works when it isn't de rigeur.
If a dev team knows they are gonna get stuck on crunch time beforehand regardless, they're gonna put off the crucial issues.

The video game industry is sick. From the devs to the phone monkeys, most companies prefer mandatory OT to sensible deadlines. As far as phone monkeys go, I know EA at least has a 6 month policy for employees. Right when the employee really starts to rock the party, they're cut loose.

Yes, I was a phone monkey in the day. EA could have cut half their support staff if they would have gotten rid of their 6 month policy, and started paying benefits. They would also reap the rewards of a competant workforce.

I hate to say it, but it's going to take unionization, and an industry wide strike to make them see the light. I'm no fan of unions, but this garbage needs to end, for everyones sake. The consumer, the company, and the employee

different metabolisms

[epine]

I have an odd metabolism where my body prefers long nights and long days (my cycle can range between 26 and 40 hours). If I'm well rested, I rarely witness a loss of productivity until 22 hours of wakefulness. Rarely at my best first thing in the day. I'm still gaining speed twelve hours later. For some reason, I go down like a rock after 28 hours of wakefulness and this has always been true. I've gone from near normal to "legally drunk" in the space of twenty minutes. I have far more problems with my body not being designed to sit in a chair for that length of time than I have with my mind falling apart. Unlike most people, I rarely allow myself to operate in a sleep-deprived state. Everything that article says about extended wakefulness is suspect because the studies didn't differentiate "rested" from "well rested" relative to how much sleep the body really wants as opposed to cultural norms (most people think that eight hours is luxury, and in the Army they expect to function on six hours routinely). There's plenty of research that shows that up to ten hours is needed to achieve the "well rested" state. Measure those people for task decay. You'll get entirely different numbers.

Crunch mode is inevitable...

[mutterc]

... because the market demands it.

Nobody will pay enough for software that isn't done half-assed, so you go with half the resource you need and slave-drive them to get it out when the customer wants.

If you push back at the customer, and say "you can have it in twice the time or at twice the cost", then they will just buy from someone else who accepts the constraints.

Also, if your company is publically-traded or has more than a few investors, their profits must continually grow at a continually growing rate. You can't just keep raising the cost of your sofware, so you need to keep the staffing levels low, so they spend all their time on death marches.

Clueful management can't save you... ours knows the stuff we're doing is a bad idea, but it's either that or go out of business now. At least if they hold on for a while then they might be able to cash-out.

If anyone believes there are software development jobs, anywhere in the world, not held to these constraints, let me know... I will sleep with whomever I have to to get one.

Re:Crunch mode is inevitable...

[startled]

"Crunch mode is inevitable because the market demands it."

It does? Funny, we must be in different markets. The market I'm in sometimes demands a superior product, or a lower price, or a quicker delivery. Those requirements are generally fulfilled by maximally efficient production: more quality and quantity produced for less time and money.

Now, if the market you're in somehow demands products completed well beyond schedule, or way over cost, or ridden with bugs, that's a curious challenge to current economic theory. You should probably get someone to study it.

This problem didn't start from nothing...

[OverDrive33]

No one can seem to figure out where this "crunch time" for programmers came from.
Frankly I'm surprised no other slashdotter's pointed it out: Programmers want to be 'leet'.
I remember starting to learn how to program about 9 years ago, I used to brag to friends about how I stayed up all night coding - a few people were even impressed by the fact I hadn't slept in 48 hours and made this cool little game.

Now we're all getting older. Young programmers (IMO) will tell any employer "Oh yeah, I can code for days on end, I'm a leet code haxor". And even if a young programmer tells a potential employer that he can't work 50+ hours a week, the next young programmer that comes for his interview will.
I have a feeling upper management of these game companies know this and exploit it to their own ends.

ALL programmers have to wise up and realize that their best code comes not after staying awake for 50 hours while running on 4 cans of Jolt cola, but after a relaxing sleep (it's a fact!).

Re:This problem didn't start from nothing...

[chthon]

I already found out when I was 21 or 22 (I was still in school then), that it was better for me to go to sleep early, rather than stay up late for a programming problem.

When I woke up fresh, I usually had the solution of the problem the evening before, and it was much easier to spot problems in already implemented code.