Cross-Distro Remote Package Administration?

tobiasly writes "I administer several Ubuntu desktops and numerous CentOS servers. One of the biggest headaches is keeping them up-to-date with each distro's latest bugfix and security patches. I currently have to log in to each system, run the appropriate apt-get or yum command to list available updates, determine which ones I need, then run the appropriate install commands. I'd love to have a distro-independent equivalent of the Red Hat Network where I could do all of this remotely using a web-based interface. PackageKit seems to have solved some of the issues regarding cross-distro package maintenance, but their FAQ explicitly states that remote administration is not a goal of their project. Has anyone put together such a system?"

Comments

Tools exist

[PeterBrett]

1. Create a local package repository for each distro.
2. Set apt/yum to point at only the local repository.
3. Create a cron job on each box to automatically update daily.
4. When you want to push a package update out to all boxes, copy it from the public repository to the local one.
5. Profit!

Re:Tools exist

[Jurily]

When you want to push a package update out to all boxes, copy it from the public repository to the local one.

Assuming of course all boxes have the same version of the OS, the same packages installed, etc.

I suggest tentakel, and that OP could have found it in 2 minutes with Google. I did.

http://www.google.co.uk/search?q=multiple+linux+remote+administration The first hit mentions it.

Re:Tools exist

[value_added]

Assuming of course all boxes have the same version of the OS, the same packages installed, etc.

And segregating things on the system that hosts the public repository is impossible?

I don't think any of this is exactly rocket science. On my home LAN where I use FreeBSD, for example, I have a motley collection of hardware ranging from Soekris boxes to Opterons. Everything gets built on a central build server and distributed automagically from there using a setup similar to what's suggested the OP. Not a single box has the same collection of userland software installed, while certain boxes do get their own custom world/kernel. None of this really requires more effort or involvement on my part than some careful thought beforehand.

One of the nice advantages of a centralised setup is that it accommodates a clean way of testing things beforehand. Rolling out the latest but broken version of "foo" to multiple systems is something to be avoided.

Webmin

[trendzetter]

I recommend Webmin which 100% FOSS. I have found it reliable, flexible and feature-rich.

clusterssh

[circlingthesun]

allows you to ssh into multiple machines and execute the same command on all of them from one terminal window. So if you set up a shell script that detects a host's distro and then execute the relevant update command you should be sorted.

You don't want it

[mcrbids]

I admin several busy CentOS servers for my company. You don't probably want a fully web-based application:

1) what happens when some RPM goes awry to borken your server(s)? Yes, it's pretty rare, but it DOES happen. In my case, I WANT to do them one by one in asc order of importance so that if anything is borked, it's most likely to be my least important systems!

2) How secure is it? You are effectively granting root privs to a website - not always a good idea. (rarely, never)

Me? I have a web doohickey to let me know when updates are available. Cron job runs nightly to yum and a pattern match identifies whether or not updates are needed, to show on my homepage. So it doesn't DO the update, butit makes it ez to see has been done.

Re:You don't want it

[galorin]

Depending on how uniform your servers are, keep one version of CentOS and one version of Ubuntu running in a VM, and have these notify you when updates are available. When updates are available, test against these VMs, and do the local repository thing suggested by another person here. Do one system at a time to make sure something doesn't kill everything at once.

Web based apps with admin privs are fine as long as they're only accessable via the intranet, strongly passworded, and no one else knows they're there. If you need to do remotely, VPN in to the site, and SSH into each box. You're an Administrtor, start administratorizing. Some things just shouldn't be automated.

Puppet or CFEngine + Version Control

[hax0r_this]

Look into Puppet or CFEngine (we use CFEngine but am considering switching to Puppet eventually). They're both extremely flexible management tools that will trivially handle package management, but you can use them to accomplish almost any management task you can imagine, with the ability to manage or edit any file you want, running shell scripts, etc.

The work flow goes something like this:
1. Identify packages that need update (have a cron job run on every box to email you packages that need updating, or just security updates, however you want to do it)
2. Update the desired versions in your local checkout of your cfengine/puppet files (the syntax isn't easily described here, but its very simple to learn).
3. Commit/push (note that this is the easy way to have multiple administrators) your changes. Optionally have a post commit hook to update a "master files" location, or just do the version control directly in that location.
4. Every box has an (hourly? Whatever you like) cron job to update against your master files location. At this time (with splay so you don't hammer your network) each cfengine/puppet client connects to the master server, updates any packages, configs, etc, runs any scripts you associated with those updates, then emails (or for extra credit build your own webapp) you the results.

Re:Remote admin of a UNIX box?

[Nursie]

Uh, right. Like putting ssh commands into a script?

ssh user@host aptitude update

Set up key based login and you don't even have to type passwords. By the sounds of it he needs to pay some attention to each individual machine anyway, as he has multiple distros and wants to determine which patches he needs for each box.

Re:Can You Script?

[dns_server]

The corporate product is http://www.canonical.com/projects/landscapeLandscape

yum-updatesd is meant for that

[MrMr]

1) yum -e whateveryoudontneed
2) chkconfig yum-updatesd on
3) Make sure do_update = yes, download_deps = yes, etc are set in yum-updatesd.conf
4) /etc/init.d/yum-updatesd start
This makes your yum system self-updating.

Great!

[Frogbert]

I for one look forward to the rational, well thought out, debate on the various pros and cons of linux distributions and their package managers, that this story will become.

Re:Tools exist: we do it this way.

[nick_urbanik]

I work in a large ISP, and this is the way we manage updates for the various Linux platforms we use. Quite simple, really. You can build tools that help: diff between the downloaded updates and what you have in your own repository, and mail you the ones that you are not using. I find lwn.net's security pages useful in keeping track of what security updates matter to us.

Re:Remote admin of a UNIX box?

[supernova_hq]

Sorry to reply to my own post, but circlingthesun actually posted the name of it below!

clusterssh

Re:Remote admin of a UNIX box?

[BruceCage]

Set up key based login and you don't even have to type passwords.

Since you basically need root access to do updates this definitely poses a security hazard as when your client is compromised there is direct access to the server. Then again, an attacker could always use a keylogger to capture the password anyways.

If you even attempt to do this I'd setup a different user account specifically for the process of updating and limit the rights accordingly and then I'd restrict the commands that can be executed (you can do this per key).

There may actually be better ways but I'm not a very experienced sysadmin. Most experience I have is from managing a single web server and my local desktop obviously. Be sure to correct me (in a friendly manner) if I'm wrong.

Then again, if you do this from the same machine as your normal account is located on you'll still have the same issues in case of a compromised client. Probably just best to limit every single account to just that what is specifically needed and setup proper host based intrusion detection (OSSEC?) to be notified when something goes wrong. This stuff is hard...

Re:Remote admin of a UNIX box?

[walt-sjc]

It's called "dssh". Google is your "search" friend (we will ignore the evil side of Google at the moment... :-)