Slashdot Mirror

There are many ways to do the things you describe. I personally make extensive use of Puppet.

This is a great solution for your configuration files, but note (directly) your code. This is where your distribution's packaging system comes in.

Build packages of your code for your OS package manager (be it RPM, portage, apt, whatever... it's usually not that difficult). Give the packages version numbers based on svn revision, if you need that granularity. Create an automated mechanism to build your package and insert it into a local repository.

Tell puppet to ensure that your 'dev' environment is always using the latest package. Tell puppet to ensure that your production and test environments are running whichever specific version they're supposed to be running.

A downside of puppet is that it's a 'pull' based system, by default every 30 minutes. For most situations, this is adequate - but not all. You might also investigate Func as, at the very least, a convenient way to tell a group of notes to phone back home to puppet on demand.

I do mine with ssh, bash and git, for the moment. I'm looking at moving to something like puppet for system configuration, though. I've also heard good things about cobbler for initial provisioning, but it's mainly aimed at an RHEL environment and that's not what we're using.

The 2009 Utah Open Source Conference had several good presentations on infrastructure automation. See, in particular, Phil Windley's slides on puppet and cobbler (hopefully audio and maybe video will be available soon).

Puppet.

http://reductivelabs.com/trac/puppet/

Well large managed networks is two miles away in the distance on the scale of things Linux is awesome at. Active Directory, Exchange, Terminal Services... Windows really does have a very impressive offering in this area, while Linux stays behind the scenes and rarely faces the user.

For one, OSX != Linux. But that's irrelevant to my post. I use puppet as does Google. Works a treat.

You set up configs for the machine groups you want, it will change files, install applications, run upgrades etc. in an idempotent fashion.

With puppet of course.

Puppet, LDAP, and your own apt-server should do . You hardware should be bought for the OS you run, not vice-versa. I have no solution for your "run Office X" correctly, as I don't understand the problem.

http://reductivelabs.com/products/ That will be $4000 in consulting fees, thanks.

The key is not to look for "ubuntu and centos solutions" (which sounds more like something typically asked a consultant), but to apply your supposedly broad system administration skills and adapt the already existing administration tools to your environment.

Typically done with small snippets of shell script building on your environments build- and packaging tools, and so on, but more mature things do exist. I mention two:

arusha (ark) http://ark.sourceforge.net/
puppet http://reductivelabs.com/products/puppet/

Puppet is a tool made to do exactly what you're asking for by abstracting the specific operating system into a generic interface. It might be worth checking out. Also there's a newcomer called chef. And then there's the oldies like cfengine.

Puppet only supports Ubuntu as a side effect of supporting Debian. If the poster is going to use Ubuntu they should review these warnings when considering Puppet.

http://reductivelabs.com/trac/puppet/wiki/PuppetUbuntu

I'm more of a puppet fan, myself: http://reductivelabs.com/products/products/puppet

Actually you should look at a newer generation of config management than cfengine. Try Puppet: http://reductivelabs.com/products/puppet/

You are looking at it from a system security perspective, not "IT Policies" perspective.

Most "I.T. Policies" are stupid and written by control freaks with no managerial sense.

force all connections through a proxy server for web filtering

The ridiculousness of web filtering aside, this is easily accomplished by pre-made config files in /etc/skel.

pass down 802.1x keys, force people to use a certain network printer,

Again, /etc/skel or something like Puppet works fine here.

He needs to be able to disallow solitare,

Oooh, this is by far my favorite, that's why I saved it for last. If you're to the point where you're seriously considering disabling solitaire, this reveals a number of things about the organization:

1) The I.T. staff and/or managers are unapologetic control freaks and perhaps even proud of it.
2) You don't trust your employees to actually be productive on their own.
3) Your hiring standards are probably pretty low.
4) You have unrealistic expectations of employee efficiency.
5) Morale must really be in the toilet already.
6) It's solitaire for fuck's sake, possibly the most boring game ever devised. If your employees are playing it instead of whatever they should be doing, that means they have no motivation to work, which means management should be the ones to get their lunchtime games taken away, not the employees.

Use puppet to enforce configuration: http://reductivelabs.com/products/puppet/

...I usually start with documenting the configuration on a Wiki (or a file in a Subversion repository somewhere) and then shift things over to Puppet when I get the chance. The nice thing about the latter is that you know all the setup specifications are correct since that's what's actually being applied to the servers.

Documentation of system provisioning is just one small part of the question you're asking, but hopefully that helps.

...I'm not sure how accurate that is. In my experience S3 and EC2 enable small companies to do things they might not otherwise hassle with.

The article also says "The glory days of the UNIX system administrator and the Java programmer are dead and buried". Really? From what I've seen, good Unix sysadmins are in high demand - whether the servers are in your colo rack or in a RackSpace facility, you still need someone to mind the farm and twiddle the Puppet manifests. Not sure about Java programmers, but demand for Ruby (especially Rails) programmers is quite high.

Two of the best open source projects that I first learned about and utilized for "real work" in 2008 (though I don't know that they count as "victories"):

Puppet, the system administration automation system. (Like cfengine, but way smarter and easier)

CodeIgniter, the PHP web application framework that doesn't box you into its idea of a web framework

The fact that we are even posting about this, ALL OVER THIS WEB, this IBM/Ubuntu deal, is AMAZING. IBM is no company to laugh about. After all they have been in the computing business well before Microsoft or most any if any at all IT company has. IBM may be expessive, but no more so that MS, and IBM has massive support resources and obviously a warm feeling to sooth corp chair's and CEO and CIO's about the longevity of deploying a Linux Desktop.

The saving will be MASSIVE. Here is why.

1) Open Office FREE (savings of untold amounts)

2) Web servers, ftp, app servers, centralize management tools... all of this is open source. puppetD from http://reductivelabs.com/

3) free mail, Postfix, Dovecot and MySQL (no more exchange licenses)

4) Free Dev tools for ISVs, Eclipse, KDev and GTK tool kits. NO MORE MS Visual Studio License Frees!

5) Free firewalls, NO MORE viruses, and malware worries.

6) Class A security, like tls, ssl3, SSH and GPG GPG, the list goes on and on. 7) AND BEST OF ALL.. NO MORE WINDOWS Licenses! Use Ubuntu (Desktop) and Redhat/Ubuntu for servers... guess what? ALL FREE

If corps are worried about retraining or certs, many MANY MANY Open Source companies have such training, REDHAT is one, OpenLogic another... and anyone with 1/2 a brain in the use of Open Source can get FREE support on the mailing lists.

My God, I run a few .coms and make some money, all running Linux. I refer to online open comminuty for help from time to time, as well as, offer consulting FREE in return for the support they gave me.

3)

When you say 'old' it depends what you want to run on them.
As a developer I use a whole range of systems, and I don't throw old machines away, I use them for testing.

• My main desktop is a Quad core AMD Phenom with 8G of memory
• Sitting next to that are two AMD Athlon machines, each with 4G of memory

I also have

• Four Pentium III machines with 512M of memory
• Two AMD K6 machines with 256M of memory
• Four Pentium I (yes one) machines with 64M of memory

A Pentium III machine with 512M of memory is quite capable of running a fairly complex website. I use them to test websites developed using the Drupal content management system. If your website won't load and run in 512M of memory - you are probably doing something wrong.

In the past I have used Xen VMs, but at the time I found it tricky to setup (from what I have seen it has improved a lot since then, so this may be a better option now). For setting up a simple test system, it worked out easier to fire up one of the old machines and run the tests on that.
If I need to setup a test system that other members of our team can access, I use rented VMs from one of the cloud providers, FlexiScale, SliceHost or Amazon EC2.

One thing I would recommend is that you never configure a machine by hand. Everything should be automatic, using shell scripts or equivalent to setup the machines. Everything, including the scripts for installing packages and configuring the system should be in source control.

To setup a new set of tests, I start by writing a shell script that will install and configure all the components needed to run the tests. It will take a while to create the first few scripts, but you will gradually build up a library of functions that you can re-use. Someone else has already mentioned using Puppet and Cobbler to achieve the same thing. Unfortunately they weren't around when I started doing this. I haven't used either of them yet, but I hope to experiment with them fairly soon.

Whichever system you use, automating the install and configuration will save you a huge amount of time in the long run. Using my library of configuration scripts, I can setup and configure a new test system in a matter of minutes. The configuration scripts are designed to be portable, so I can use the same tools on one of my local test machines, or on an external VM hosted by a cloud provider.

As to what I use the Pentium I machines for - stress testing. I write Java web services for a UK eScience project, processing large (Tbyte) data sets. One of the things I need to check is the webservice should never try to load the entire dataset into memory. It should process the data bit at a time, and free up resources as soon as it has finished with them. As a stress test, I deploy a webservice on one of the tiny 64M machines, and then run multiple clients on the bigger more capable machines to hammer it into the ground, repeatedly, day after day for a week. If my webservice can process Gbyte data sets on a Pentium I machine that only has 64M of memory - without grinding to a halt. Then I can be fairly confident that when the same webservice is deployed on a multi core machine with Gbytes of memory it will probably be able to cope with the kind of load our scientists intend to throw at it.

Summary : Keep the old machines and learn how to setup, configure and use them as test machines. In the process you will encounter many of the problems that your developers and sys admins have to cope with on a daily basis, and you will be much better placed to be able t

If you do go the VM route, consider using puppet to define your machine configurations. That'll make it much easier to build out more than one of each machine.

Along the same lines, if you don't want to run a bunch of machines or VMs yourself you could spin up new EC2 instances as needed, point them to your Puppet server, and have them built out - consistently and correctly - in short order. Good times.

or something similar

I can recommend Puppet.

puppet http://reductivelabs.com/projects/puppet/

Puppet could be the answer to the multiple system thing. It can handle different systems running different packaging systems quite well and update them all according to generic directions. For instance you could have it install a HTTP server on all Linux machines and on RHES it uses Apache and on SLES it installs Lighttpd. Puppet is completely configurable and (fairly) easy to use.

From the sounds of it you're leaning toward a system administration position. Managing desktops, servers, helpdesk, etc -- a perfectly reasonable career path for some folk. That said, only if you're comfortable making 50-60k for the rest of your life can you can get away without any programming.

What you're going to find out is that in today's day in age, system administration is all about automation -- and automation is all about scripting. This holds true for any platform: Windows, Linux, OSX, etc. Even if you're horrific with traditional languages (C/C++/Java), you're never going to advance in a sysadmin position unless you become proficient in a scripting language like Python or Ruby.

My recommendation: learn Python. Of the major scripting languages Python is arguably the easiest to grok. Once you understand simple procedural scripting you can move into OO-scripting, and from there you can begin to utilize the wealth of the sysadmin modules/frameworks Python has to offer. After that I'd probably recommend you check out Ruby and the excellent Puppet project http://www.reductivelabs.com/projects/puppet/.

Either way, if you're not going to learn a scripting language, don't expect anything beyond entry-level work for the rest of your IT career.

I am putting together puppet to manage our Debian boxes. It works with Linux and Mac OSX (according to the site, we don't have any Mac OSX machines for me to test the reality)

Using something like subversion or any other version control system for such a task just leads to Yet Another Homebrew Administration System, that will probably lead your successors to tears and insanity. Use tools already there, and that are pertinent to the job.

Problem for your situation is that it has no Windows or Novell support as of yet, but recently work on Windows at least seems to have started. And if your first priority is mainly config file management: that part should be fairly trivial.

These days it's just a few commands:

cd /usr/src
wget https://reductivelabs.com/downloads/facter/facter- 1.3.7.tgz
wget https://reductivelabs.com/downloads/puppet/puppet- 0.22.2.tgz
tar xzvf fac*
tar xzvf pup*
cd fac*
ruby install.rb
cd ../pup*
ruby install.rb

and then finally:

sudo puppetd --test --waitforcert 15

and then either have the puppet server autocert the new machine or do it by hand (for the paranoid)...

So, about 5 minutes, I'd think.

Check out puppet, it's really a great (albeit a bit new) project: http://puppet.reductivelabs.com/