Environment Variables - Dev/Test/Production?

Woody asks: "It's common knowledge that the development environment should be the same as the test environment which should mimic the production environment whenever possible. I'm currently working on a project where we develop on Dell, test on IBM, and the production system is up in the air. For embedded systems the differences can be running on a VM versus running on the actual hardware. What I want to know is what kind of problems have Slashdot readers been faced with because of different environments being used for different phase of their projects, and how have they been overcome?"

Comments

Depends on the discipline of the developers

[BigLinuxGuy]

I've worked on several similar projects and one of the items that bit us was when the developers made assumptions based characteristics of the development platform that were somewhat radically different than the production platform. Assumptions around the size of data (not just word size, but buffer sizes for some text retrieved from "standard" libraries) caused a number of problems on several Java projects I've worked on (not to mention the ones in other languages). Data size seems to be one of the more commonly overlooked items in design and implementation, so I'd urge you to make sure that you've done your due diligence to ensure that you don't find those after the fact.

Another area is performance. For reasons I have yet to understand, there seems to be a prevalent myth that performance can be bolted on after the fact (the "make it work, then make it work fast" mindset). The truth of the matter is that performance has to be engineered in from the beginning or you simply spend a lot of time and money rewriting code that should never have been written in the first place. Sadly, educational institutions don't appear to place any emphasis on actual performance or teach the principles of performance tuning as part of their curricula.

Re:Depends on the discipline of the developers

[FroMan]

Here I would disagree. Performance should rarely even be a consideration until the product works.

As such, this does not mean to use braindead implementations, but worry about a working product first.

The first setp to a serious project is to work through the design. This means looking at the interfaces that the software will provide. Whether those are UI or API. Those are your targets that your users will work with.

Then you will need to have test cases for the interfaces that you have agreed on. These test cases should validate the accuracy of the system. Accuracy is key, far more than performance.

Finally, work on implementation. The implementation of the project is the most fluid of the rest of the system. You do not want to be changing APIs or UIs after they have been agreed upon unless absolutely necessary.

Performance, while not exactly an after thought should only be worried about once the problem is known to exist. Often more/better hardware can be thrown at the situation, if not now, 6 months down the road. If there is a problem with the algorithm, then you can change the implementation without affecting the interface.

Re:Depends on the discipline of the developers

[dubl-u]

For reasons I have yet to understand, there seems to be a prevalent myth that performance can be bolted on after the fact (the "make it work, then make it work fast" mindset). The truth of the matter is that performance has to be engineered in from the beginning or you simply spend a lot of time and money rewriting code that should never have been written in the first place.

Myth, eh? Personally, I do no "bolting". The steps I happily follow are

• make it work
• make it right
• make it fast

The notion here is that you get something basic working. Then you refactor your design to be the right one for where you ended up: a simple, clean design. Then you do performance testing, discover what the real bottlenecks are, and find ways to get the speed with minimal design distortion here. The you go back to step 1 and add more functionality.

There are a couple of benefits to doing the optimization last. One is that a clean design is relatively easy to optimize. But the big one is that by waiting until you have actual performance data, you get to spend your optimization time on the small number of actual bottlenecks, rather than the very large number of potential bottlenecks. That in turn means that you don't have a lot of premature optimization code of unknown value cluttering up your code base and retarding your progress.

Of course, this is not a license to be completely stupid. Before you start building, you should have a rough plausible architecture in mind. If there are substantial performance risks at the core of a project's architecture, it's worth spending a day or two hacking together an experiment to see the if your basic theories are sound.

As Knuth says, "We should forget about small efficiencies, say about 97% of the time: PrematureOptimization is the root of all evil."

Re:Depends on the discipline of the developers

[SunFan]

Performance should rarely even be a consideration until the product works. ...until the prototype works. The final product has to perform well. Otherwise, people will find trivial excuses to say it sucks and needs to be replaced, even if, on a whole, it is a decent product.

Re:Depends on the discipline of the developers

[Godeke]

So, which "performance" are we optimizing for? Memory footprint? Disk access? CPU utilization? Network utilization?

It turns out you rarely know which will bite your face off until you get a representative data set running. If you made the wrong choice, you probably made things *worse* than if you had opted for working code that could be re-factored from "easy to read" to "mostly easy to read and performs where it counts".

When you are working on server based solutions that will be hit hard, all of those could be your bottleneck: or none.