Managing Environment Specific Config Files?

byrnereese asks: "It seems that every organization I have worked for has been plagued with the same problem: 'how do you effectively manage multiple environment configurations for the various development, staging and production environments that exist.' For example, each application at our company requires a different set of database credentials for each environment we deploy it to. For a long time we have maintained different config files for each environment and checked them into our SCCS; but that has proved incredibly error prone because a different build is required for each environment. Each company I have worked for utilizes a different systemology to make config file management a little easier, but none of them have stumbled upon anything that is any less error prone, or any easier to manage than any other solution to choose from. So I am curious, how does your company manage environment specific application configuration? And what does your change management process for these configurations look like?"

Comments

Re:IANAP

[stefanlasiewski]

Rather then logging in as different users, I would just setup a set of shell scripts which contain a PROJECT variable.

The shell scripts could be shared by different users, so each user could set their environment to one of the four phase-project environments.

Script1: PROJECT=beta-foo
Script2: PROJECT=alpha-foo

That way, any of your developers could simply set their $PROJECT , and YOU as the administrator, don't need to clean up when they fscked up the shared user account.

Requirements and other issues

[phamlen]

I've encountered this issue in a number of places, and have only been satisfied with homegrown systems. Here are some of the issues we've tackled (but entire books have been written on this topic.)

Requirements

First, here's my short list of requirements of a build/config system:

1. All config files must be under version control. Given a release of software, you must be able to find the appropriate config files for that release.
2. Config files MAY change when the software doesn't - this should still count as a release (ie, a release is not just 'when the developers have new code.')
3. Config files may vary depending on the environment (ie, stage, development, production, backup production.)
4. In an environment where you may have to rebuild machines (ie, WebServer1 just bit the toilet), you need to be able to build a new machine with the correct config file. Thus, your "build routine" for new machines needs to hook into the system.
5. You need some kind of script that can check the validity of a machine's configuration - (eg, run 'select 'x' from dual' against the database, read/write to the appropriate directories, check that you can connect to server X on port Y, etc.)

Some solutions:

Generally we've "rolled our own" because the good config packages cost a lot of money and still eat up a lot of resources to maintain them.

The Redhat Package approach We build a package of all appropriate config files - one for each type of machine in each environment. The machine contains a "well-known file" which indicates which kind of machine and what type of environment it's in. The package checks to ensure that it's being installed on the appropriate type of machine.

Advantages:

• the wrong config files can't get onto a machine.
• You can use rpm commands to check if config files are installed.

Disadvantages:

• It's a pain in the neck to write the build script.
• There are a lot of tricky issues around how to write the Redhat package, ensure the deployment works, etc.

Centralized Perl 'Config Server'

In this approach, you write a centralized Perl server that keeps track of all config files. If you write it correctly, it can even be 'heirarchical' (ie, there are default config files, then webserver config files, and Webserver1 inherits all default config files and all webserver config files.)

Each machine then asks the Perl server for its config files.

The perl server ensures that config files are checked into CVS

Advantages:

• One centralized location for config files.
• the ability to determine which machines have which config files

Disadvantages

• Someone has to write the Perl server- it took us about 3 months, but once it was done, it worked great.