Slashdot Mirror
A Proper Environment for Web Development?
umdenken wonders: "I'd like to know how others on Slashdot do their server-side web programming. We have dozens of Perl CGI scripts, and are currently doing development by editing these production scripts in place on the web server (!). Our sysadmins have finally installed an SVN client on the server (Solaris), and have offered to create a new virtual host that we can use as the development server. What are some of the practices you use for organizing this kind of set up?"
Even with a virtual host, you'll still be hitting the production hardware. Sounds like an ideal use for virtualisation here - mimic as much of the production environment as possible (OS versions, web server versions, application container versions etc.) and have a go with that.
Subversion is definitely a stride forward - well worth using and getting used to, it's good that you have a client there. You should be able to fix your config scripts such they they recognise the environment (prod, dev, uat) and can be deployed directly from a tag in svn. A tag of course, not the trunk. Given the constraints it sounds like you're up against, I would definitely be looking to virtualise at least three environments - one dev, one system test, one UAT. You may have multiple dev virtual machines depending on your needs.
Cheers,
Ian
I hope you're not doing anything important. Although chances are if you're using Perl CGI it's probably not.
Most web development environments I've been exposed to have a development, UAT (User Acceptance Testing) and production environment. Alternatively your development environment can be local and you can stick a "system testing" environment in between dev and UAT. Your UAT environment should mirror production, and before you apply any changes should have whatever code is currently in production.
You do your development, and if it's being done locally you integrate your changes with everyone else's in system test and do your automated testing and so forth. Once the developers/testers are happy a release is packaged up and deployed to UAT. You should probably run your automated tests again here for a sanity check.
The end users (business or whoever) do their testing in UAT, and if they're happy (and this is important) you take the same package you applied to UAT and apply it in production.
In some environments the developers aren't the same people with access or rights to apply changes in produciton, so you've got different groups performing different roles, but you get the idea.
Disclaimer: this is just my experiences of corporate web development, your mileage may vary, but I believe this sort of setup is pretty common (with differences here and there).
I'm sure this sort of problem is trivially "obvious" if you've been in the IT industry for a decade or so, but if you're not already in this sort of environment and someone sort of dumps the problem in your lap, what are you supposed to do? How to construct and manage a source control system for a decent-sized environment isn't usually the sort of thing you will find with your average B.S. in computer science. I certainly know that I wouldn't know how to go about doing much more than the basics (give me svn, don't edit things straight from production, umm, two machines to work on would be nice, and... uhh... I dunno, what next?). It's the sort of thing that I'd expect to learn about either on the job (using one that someone else with more IT knowledge set up for me), or from some sort of technical articles (which?), or both. Perhaps someone can recommend an online guide on how to structure a sophisticated system for source code, system configurations, and such... or maybe some books...
The World Wide Web is dying. Soon, we shall have only the Internet.
I'll tell you how our entire setup works, and hope it helps:
...
A. Development Box
B. Test Server
C. Production Server
A. Development Box
Every developer has Apache (or IIS respectively) and PHP/SQL on his box. People without experience can just install one of the premade packages that exist (like XAMPP or whatever its name was). This setup is isolated from the outside and responds only to 127.0.0.1 and the virtual domains. Each virtual host in Apache is a separate project.
Server Side Developers work in Eclipse PHP IDE with SubClipse, designers/client side work in Dreamweaver/Photoshop with SVN4DW & TortoiseSVN.
B. Test Server
This is used for few purposes: devs can checkout a revision and run it there on a "real" server to test, QA (well we have no dedicated QA.. it's a small team) can test on this server too.
If everything is ok it goes to...:
C. Production Server
That's it, it works really well though, everyone has his own server that can run files right of his PC, and this helps a lot in quick development. Showing to clients is as easy as checking out a revision on the test server.
This is Unix/OSX-centric, and I've been doing this for years and showing others the way:
.....
.. by the way, use daemontools to supervise your daemons, anything else like crappy pidfile-based startup scripts are unpredictable). Ideally someone OTHER THAN YOURSELF should be able to mindlessly follow some simple instructions to accomplish this step. Design your procedures with this in mind.
.. OR when you get a SECOND client who will run an older version of t
* My code always has a standard layout (bin, conf, src, lib, and so on). No exceptions, because you never know when that little script will become a big app (this happens to me at least once a year).
* Use good coding practices: unit tests, continuous integration, whatever
* The code is checked into CVS/Subversion/Darcs, whatever. Use branches and so forth intelligently (dev on the trunk, release branches which are bugfix only, whatever). Make it "obvious" where the latest stable code always lives, so that someone besides yourself can deploy it.
* I have a script which will deploy the app to any server with rsync (excluding CVS, config, test, and dev files). There is also a flag that will "pull" the files from the server, in case of an emergency fix that was done directly on the server.
* There's a "config" directory with all system-dependent configs. No passwords or other stuff is hard-wired into the app.
* As others have described, you have your various "dev" machines, a "staging" server (identical to production but non-critical), and a "production" server. NEVER WORK ON THE PRODUCTION SERVER EXCEPT IN EMERGENCIES. Also, resist the temptation to install extra stuff on the staging server like some do (MRTG or Nagios or whatever). The staging server must be identical to the production in all respects.
If you do this properly, all you have to do to work is the following:
1) go to your laptop or other dev environment and check out the code.
2) review the REQUIREMENTS file for any packages you might need to install
3) adjust the app config files appropriately
3) code, test, checkin, code, test, checkin,
Then when you're ready to update the code on the server, first sync to the staging server and test as needed (user tests, whatever). Virtual hosts can work this way too (I've done it this way, for instance "foo.com" is the app and "dev.foo.com" is the staging version, on the same machine).
Once everything is working, push to the production servers (my script will also restart anything that's needed
I also agree with the poster above who said it's good to have different OS or environments on your dev machines, to catch any hidden assumptions. I dev on Linux and OSX and push to FreeBSD usually. Use conditional code as appropriate, and sparingly.
Ideally, you can take a blank server, install the OS, install the REQUIREMENTs, push the app, config the app, run a setup script, and go. No undocumented requirements. No weird "procedures". Just "push button install".
You should also get into the habit of making apps "learn" as much about their environment as possible. For instance, in my Ruby, Perl, PHP code, I use the __FILE__ variable or equivalent to determine the install dir, that way I don't have to configure it. A common library sets up all the necessary paths based on that.
Write your code to be flexible and backwards-compatible. For instance, if you need to move some data files around or change database fields, write code that detects the old version and does the update at startup time. A little extra work, but oh-so-automatic.
Once you get this working, you'll never want to work any other way. Being able to check out and deploy your code ANYWHERE in just a few steps is a very powerful feeling. Heck, just being able to check out in a different directory on your dev machine is useful. Having separate release branches are awesome when the client reports a bug but you've already started the big changes for 2.0