Environment Variables - Dev/Test/Production?

Woody asks: "It's common knowledge that the development environment should be the same as the test environment which should mimic the production environment whenever possible. I'm currently working on a project where we develop on Dell, test on IBM, and the production system is up in the air. For embedded systems the differences can be running on a VM versus running on the actual hardware. What I want to know is what kind of problems have Slashdot readers been faced with because of different environments being used for different phase of their projects, and how have they been overcome?"

Comments

Depends on the discipline of the developers

[BigLinuxGuy]

I've worked on several similar projects and one of the items that bit us was when the developers made assumptions based characteristics of the development platform that were somewhat radically different than the production platform. Assumptions around the size of data (not just word size, but buffer sizes for some text retrieved from "standard" libraries) caused a number of problems on several Java projects I've worked on (not to mention the ones in other languages). Data size seems to be one of the more commonly overlooked items in design and implementation, so I'd urge you to make sure that you've done your due diligence to ensure that you don't find those after the fact.

Another area is performance. For reasons I have yet to understand, there seems to be a prevalent myth that performance can be bolted on after the fact (the "make it work, then make it work fast" mindset). The truth of the matter is that performance has to be engineered in from the beginning or you simply spend a lot of time and money rewriting code that should never have been written in the first place. Sadly, educational institutions don't appear to place any emphasis on actual performance or teach the principles of performance tuning as part of their curricula.

Re:Depends on the discipline of the developers

[GoofyBoy]

>there seems to be a prevalent myth that performance can be bolted on after the fact (the "make it work, then make it work fast" mindset).

Its not a just myth; its a damned lie that hurts all of society as a whole.

Re:Depends on the discipline of the developers

[FroMan]

Here I would disagree. Performance should rarely even be a consideration until the product works.

As such, this does not mean to use braindead implementations, but worry about a working product first.

The first setp to a serious project is to work through the design. This means looking at the interfaces that the software will provide. Whether those are UI or API. Those are your targets that your users will work with.

Then you will need to have test cases for the interfaces that you have agreed on. These test cases should validate the accuracy of the system. Accuracy is key, far more than performance.

Finally, work on implementation. The implementation of the project is the most fluid of the rest of the system. You do not want to be changing APIs or UIs after they have been agreed upon unless absolutely necessary.

Performance, while not exactly an after thought should only be worried about once the problem is known to exist. Often more/better hardware can be thrown at the situation, if not now, 6 months down the road. If there is a problem with the algorithm, then you can change the implementation without affecting the interface.

Re:Depends on the discipline of the developers

[dubl-u]

For reasons I have yet to understand, there seems to be a prevalent myth that performance can be bolted on after the fact (the "make it work, then make it work fast" mindset). The truth of the matter is that performance has to be engineered in from the beginning or you simply spend a lot of time and money rewriting code that should never have been written in the first place.

Myth, eh? Personally, I do no "bolting". The steps I happily follow are

• make it work
• make it right
• make it fast

The notion here is that you get something basic working. Then you refactor your design to be the right one for where you ended up: a simple, clean design. Then you do performance testing, discover what the real bottlenecks are, and find ways to get the speed with minimal design distortion here. The you go back to step 1 and add more functionality.

There are a couple of benefits to doing the optimization last. One is that a clean design is relatively easy to optimize. But the big one is that by waiting until you have actual performance data, you get to spend your optimization time on the small number of actual bottlenecks, rather than the very large number of potential bottlenecks. That in turn means that you don't have a lot of premature optimization code of unknown value cluttering up your code base and retarding your progress.

Of course, this is not a license to be completely stupid. Before you start building, you should have a rough plausible architecture in mind. If there are substantial performance risks at the core of a project's architecture, it's worth spending a day or two hacking together an experiment to see the if your basic theories are sound.

As Knuth says, "We should forget about small efficiencies, say about 97% of the time: PrematureOptimization is the root of all evil."

Re:Depends on the discipline of the developers

[SunFan]

Performance should rarely even be a consideration until the product works. ...until the prototype works. The final product has to perform well. Otherwise, people will find trivial excuses to say it sucks and needs to be replaced, even if, on a whole, it is a decent product.

Re:Depends on the discipline of the developers

[Godeke]

So, which "performance" are we optimizing for? Memory footprint? Disk access? CPU utilization? Network utilization?

It turns out you rarely know which will bite your face off until you get a representative data set running. If you made the wrong choice, you probably made things *worse* than if you had opted for working code that could be re-factored from "easy to read" to "mostly easy to read and performs where it counts".

When you are working on server based solutions that will be hit hard, all of those could be your bottleneck: or none.

Re:Apropos

[SpaceLifeForm]

That is all good and fine, however, you didn't address the issue of environment variables, which your testing/QA people are typically afraid to change (or conversely, they do change and create a mess).

The best approach is to create an environment variable that defines the development environment, example DEVENV=DEV, DEVENV=TEST, DEVENV=IT, DEVENV=QA, DEVENV=PROD (or not needed for production at all), and then elsewhere (controlled and basically kept hidden from the testers or users), other environment variables are set based upon the value of DEVENV. Examples of these environment variables would be your PATH variables, ORACLE_SID, etc.

Then, the final problem is educating the testers what DEVENV means, and more importantly, why that one has to be correct and that they should not mess with any other environment variables.

If the testers can't understand that, you need smarter testers.

Continous integration

[maxmg]

We are developing J2EE applications using a continous integration server (currently anthill open source, but others are available). Ant is used for building, testing and deploying.
Now we have a number of environment-specific settings, for example database connection details, etc.
All environment-specific stuff goes into .properties files which are included conditionally by the ant script (based on a single environment variable or ant parameter). All of those properties files live in a directory conf/<environment name>, where environment name is either a developer's name, or "test", "production", "staging", etc. Each night, new deployment packages for each of the different deployment targets (test, prod, etc.) are built and made available through anthill. Some of those targets are also automatically deployed for the testing team every night, so the latest features are always available to be tested somewhere the next day.
Every successful build is tagged in CVS with an autoincrementing build number. When we have identified a release candidate, it is as simple as instructing anthill to (re)build a deployment bundle for a particular target with a specific build number. That deployment bundle (usually a .ear or .war) is then simply dropped into the production environment - remember that all the environment-specific settings are already included in that particular bundle. The benefit of this is that all environmental settings are maintained in the main source repository, the downside being that different packages exist for the different targets, but in practice that has not proved to cause any problems.
An additional benefit is that each environment's individual settings (including development machines) is always available to all developers for comparison and troubleshooting.
I guess the lesson learned is this:

• Automate your build!
• Extend your build system to include the environmental configuration
• Automatically build separate targets for different environments