How Fast is Your Turnaround Time?

petrus.burdigala writes "I work for a mid-sized commercial software company (~20 Mloc) and we are frequently challenged by our supervisors to get fixes around the clock. Overall, we manage to get a 'bullet-proof' patch in about 4-5 weeks (from coding->QA->Build/Packaging->shipment), which I consider not so bad. But the other day, we got an urgent request from our support team to come up with a decent fix in 48 hours. I think they're a tiny bit unrealistic. So I wanted to get feedback from my peers: are we doing that bad? It takes months for other software vendors to issue zero-day exploit fixes, are our customers being unreasonable?"

Comments

how long is a piece of string

[LiquidCoooled]

It depends upon the nature of the problem and the competency of the developers.
If you know enough of the code tree you can tell when first reproducing and examining the failure whether it is a one off mistake or a larger procedural fault.

Single instance stupid errors (doh! moments) can be rectified and put through testing fairly quickly, however if your initial examination uncovered a larger problem then obviously the process will take longer (if at all - consider workarounds).

If the original dev/test team has been replaced over time this becomes a more difficult issue and every bug must go through complete verification simply because the extent or ramifications of the code modification will not be known.

In some instances we have had fixes out of the door the same day an issue was noticed, in others months go by before a final fix is put in place.

Can't compare

[Sloppy]

It depends on what you're maintaining and how complicated it is. I've gotten fixes out in 2 or 3 minutes. That doesn't mean I'm fast and you're slow, though. "How fast is your turnaround?" is like "how long does it take to write a computer program?" It's hopelessly vague.

Parent is right.

[iknownuttin]

It may just be me but I think that's why they are called "customers"

Sometimes, customers are unreasonable and if they are, they should be treated with respect and the problem explained to them. Yes, they may be incredulous, but if you hold your ground (if they're being unreasonable), treat them with respect, they will come around.

The fact that the parent was moderated down just shows me that the arrogance, contempt, and stupidity in corporate America is alive and well - especially in IT.

It depends...

[gillbates]

48 hours is tad bit tight. However, I've turned things around in a similar amount of time.

But, the old adage is true: you get what you pay for:

• Granted, 48 hours is tight, but possible if you know the root cause, *and* the customer is willing to forego the usual QA process before delivery. It doesn't mean that you don't do QA, but rather, that you do it later and patch the patch, if necessary. In most corporations, this means that if the customer doesn't complain, QA doesn't get done for these "extra-special" releases.
• Four to five weeks for 20Mloc is probably about average. As a general rule, a good team will average about 1 week per department the fix has to go through: field team -> engineering (fix) -> department review -> QA review -> ship. However, in some organizations, particularly the smaller ones, defects can get turned around in 1 to 2 weeks, especially if the customer works directly with the engineer/developer, and the developer is authorized to make releases to the customer. Be aware that your customers may have dealt with such organizations in the past.
• 20Mloc is not really that big, provided that the project and code was well-organized from the beginning and the original developers are still on staff. But, if this is not the case, or you have a large developer base, where no one is actually an expert on the systems, or subsystems, you can add about a 50% overhead to your turnaround time. If the original developers are no longer there, add 100%.
• If you don't know the root cause of the problem, you can't promise anything, and you need to inform the customer of this - you simply can't make any guarantees because you don't know the scope of the problem. Once the root cause has been identified, a 48 hour deadline is still tight, but is long enough to allow the key developer to build and do some rudimentary testing of the fix. Should the customer choose to accept it at this point (and you *must* make the point that it is their choice), they must be willing to forego the normal QA process, and should sign a statement of understanding to that effect.

When faced with unreasonable deadlines in the past, I usually voice my opinion once, and just do the best I can. Your higher-ups are probably already quite stressed at this point, and adding stress to the situation doesn't do anything for your career or theirs. Rather, if you make the point that you're doing the impossible, you might just have a little bit more bargaining power when it comes time for raises.

But on the flip side of the coin, if management doesn't learn, and you find yourself constantly asked to do the impossible, you might want to consider employment elsewhere...

Re:Web based

[Gregb05]

*15 minutes.
It's bad enough that they directly state they're not really testing patches with a 15 minute turnaround, but the fact that they're making mistakes that can be fixed in 15 minutes speaks loudly as well.

It's about risk

[PIPBoy3000]

I work for a large healthcare organization and typically have very fast turn-around times (bugs often get squished within an hour). For clinical applications and other core applications, though, we're much more methodical and careful.

I often explain to the user that I can push changes out immediately, but it introduces certain risks. I then detail the risks they may face, and that if they say to go ahead anyway, at least they'll be aware of what might happen.