Linux Patch Management

Ravi writes "Any system or network administrator will know the importance of applying patches to the various softwares running on their servers be it the numerous bug fixes or vulnerability checks. Now when you are maintaining just a single machine, this is really a simple affair of downloading the patches and applying them on your machine. But what happens when you are managing multiple servers and hundreds of client machines? How do you keep all these machines under your control up to date with the latest bug fixes? Obviously, it is a waste of time and bandwidth to individually download all the patches and security fixes for each machine. This is where this book named "Linux Patch Management - Keeping Linux systems up to date" authored by Michael Jang gains significance. This book released under the Bruce Perens' open source series aims to address the topic of patch management in detail." Read the rest of Ravi's review Linux Patch Management - Keeping Linux Systems Up To Date author Michael Jang pages 270 publisher Prentice Hall rating 8 reviewer Ravi ISBN 0-13-236675-4 summary This book offers Linux professionals start-to-finish solutions, and examples for every environment, from single computers to enterprise-class networks.

The book is divided into seven detailed chapters, each covering a specific topic related to patch management. In the first chapter, the author starts the narration by giving an introduction to the basic patch concepts, the various distribution specific tools available for the user including Red Hat up2date agent, SUSE YaST online update, Debian apt-get and also community based sources like those in Fedora. What I found interesting was instead of just listing the various avenues that the user has regarding patching his system, the author goes the extra mile to stress the need for maintaining a local patch management server and also the need to support multiple repositories on it.

The second chapter deals exclusively with patch management on Red Hat and Fedora based Linux machines. Here the author walks the readers through creating a local Fedora repository. Maintaining a repository locally is not about just downloading all the packages to a directory on your local machine and hosting that directory on the network. You have to deal with a lot of issues here, like the hardware requirements, the kind of partition arrangement to make, what space to allocate to each partition, whether you need a proxy server and more. In this chapter, the author throws light on all these aspects in the process of creating the repositories. I really liked the section where the author describes in detail the steps needed to configure a Red Hat network proxy server.

The third chapter of this book namely SUSE's Update Systems and rsync mirrors describes in detail how one can manage patches with YaST. What is up2date for Red Hat is YaST for SuSE. And around 34 pages have been exclusively allocated for explaining each and every aspect of updating SuSE Linux using various methods like YaST Online Update and using rsync to configure a YaST patch management mirror for your LAN. But the highlight of this chapter is the explanation of Novell's unique way of managing the life cycle of Linux systems which goes by the name ZENworks Linux Management (ZLM). Even though the author does not go into the details of ZLM, he gives a fair idea about this new topic including accomplishing such basic tasks as installing the ZLM server, configuring the web interface, adding clients ... so on and so forth.

Ask any Debian user what he feels is the most important and useful feature of this OS, then in 90 percent of the cases, you will get the answer that it is Debian's contribution to a superior package management. The fourth chapter takes an in depth look into the working of apt. Usually a Debian user is exposed to just a few of the apt tools. In this chapter though, the author explains all the tools bundled with apt which makes this chapter a ready reference for any person managing Debian based system(s).

If the fourth chapter concentrated on apt for Debian systems, the next chapter explores how the same apt package management utility could be used to maintain Red Hat based Linux distributions.

One of the biggest complaints of users of Red Hat based Linux distributions a few years back was a lack of a robust package management tool in the same league as apt. To address this need, a group of developers created an alternative called YUM. The last two chapters of this book explores how one can use YUM to keep the system upto date as well as hosting ones own YUM repository on the LAN.

Each chapter of the book explores a particular tool to achieve patch management in Linux and the author gives in depth explanation of the usage of the tool. All Linux users irrespective of which Linux distribution they use will find this book very useful to host their own local repositories because the author covers all distribution specific tools in this book. The book is peppered with lots of examples and walk throughs which makes this book an all in one reference on the subject of Linux patch management."

Michael Jang has specialized in networks and operating systems. He has written books on four Linux certifications and one of them on RHCE is very popular among students attempting to get Red Hat certified. He also holds a number of certifications such as RHCE, SAIR Linux Certified Professional, CompTIA Linux+ Professional and MCP.

You can purchase Linux Patch Management - Keeping Linux Systems Up To Date from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Update: 02/07 14:52 GMT by J : Book rating changed from an intended 4 (of 5) stars to Slashdot-normalized 8 (of 10), by Ravi's request.

Comments

Patches using RPM

[IMightB]

What I want to know is how to issue patches via RPM rather than distributing the whole app again. Wether using some sort of binary diff, or just packaging the changed files. And how to manage things like this with the RPM database. I know that SuSE has got "patch" rpm's but I can't find any info as to how these are created, or how they are viewed/managed by the rpm DB.

Anyone?

Re:Patches using RPM

[jacksonj04]

And yet people like you complain about those who don't keep their PC patched?

You have two choices:

Make things easy for some who can't remember some cryptic command to download source, compile, install, patch, re-patch, re-re-patch, change the config, find it was the wrong config, hunt for the config, change the config, find they have to re-compile, re-compile, re-install, re-re-re-patch and finally use.

Stop whining.

Re:Patches using RPM

[Subrafta]

I've used RPM to patch and update / change configurations on a number of in-house applications. We've got several hundred Red Hat systems of various vintages installed on customer networks. Basically I make the "patch" RPM dependant on the original RPM, then use the %pre and %post areas of the spec file to ID the target system, update configurations, start / stop services, and move updated files into place. It's not a perfect system and I only use it to do automated, broad-based configuration changes, not to scrimp on bandwidth. You have to track just about everything yourself and deal with regression testing. An RPM to update named.conf, originally installed by the bind-9.3.1-14_FC4 RPM, might be named MyUpdate-bind-1.0-0. AutoRPM (some of these systems pre-date up2date, yum, etc. and I don't want two software management systems) handles noticing that new or updated RPMs are available.

Re:Patches using RPM

[Peter H.S.]

What I want to know is how to issue patches via RPM rather than distributing the whole app again.

I will try to answer why this probably won't happen for at foreseeable future, and why it probably not is a good idea.

The only advantage that a binary patch system have over distributing the whole rpm package is that it saves bandwith.
A major disadvantage of such a system is that it creates twice the overhead, since most of the work that a Linux distributer have with patching its software, is the (regression) testing. So now the Linux distributer has to track _and_ test two kinds of updates; binary diff packages, and whole packages. They can't skimp testing one of the two types, since that would almost certainly mean, that a trivial error borks the untested package, that then would hose thousends of machines. And if the distro skimps distributing the whole packages, well, then types like me would start to whine about how much is sucks to keep track of "package" +"hotfix_1" +"hotfix_2" +"hotfix_3" instead of just getting "updatedpackage".
The package management systems would also have to be reworked, since they now have to keep detailed track of packages and updates, and the exact order of which to apply these updates. (when I was working with MS Windows servers years ago it was not uncommon that Windosupdate would loose track of updates and installed software, so that old software would overwrite new security patches)

In short, a binary diff patch system would mean a lot of work, for a negliable gain

Way back when I started with Linux, I also thought that it was a good idea just to distribute binary diff updates, since that was what I was used to, and because it somehow seems wastefull distribute a whole package.
I changed my mind when I actually started to manage some Linux servers.

--
Regards
Peter H.S.

ZENworks Linux Management

[ezs]

This is a nice review of 'patching Linux' - and it's a subject close to my heart. Usual disclaimer - ZLM is partly my product and baby. One thing that the review clearly describes - there's a lot of choice out there. From Red Hat Network; to Novell update; to YaST Online Update - and there there is yum, apt etc etc etc. One of the cool things that ZENworks Linux Management brings to the table is the ability to integrate multiple sources of patches - RHN, YOU, Novell, roll-your-own, apt - and bring them into a central release server and control what goes where. For those that are too small or don't want to shell out for ZENworks - remember there is also the fully open source Open Carpet product - http://opencarpet.org/

yawn: in linux, it's called a package, not a patch

[totro2]

Old school commercial Unices like Solaris, HPUX, and AIX have "patches". Modern linux systems have "packages". Anyone who doesn't deal with a modern, automagical package management system like apt or yum is usually slogging through the mud unnecessarily. By updating a package, you get your patches. Most Linux users should never have to patch source code from tarballs, like the kernel or other software. This book may be useful for those few exceptions, however.

I wrote a book on Linux Patch Management

[wawannem]

Chapter 1:
apt-get update ; apt-get upgrade

Re:I wrote a book on Linux Patch Management

[Dogers]

For one machine, yeah, no problem.

For 10 machines? 50? 100? 500? No thanks.

Re:I wrote a book on Linux Patch Management

[El_Muerte_TDS]

apt-cron

although that only works when the patch doesn't need human attention

Re:I wrote a book on Linux Patch Management

[sholden]

Obviously you do testing on the test machines and only push the updates to your apt repository after they have been tested, at which point the production machines auto update with them.

You don't point the production machines at the distro's repository, but non-retardation is an assumed and hence these bits aren't usually made explicit.

How about a useful link?

[Dogers]

Such as to the actual open source series?
http://www.phptr.com/promotions/promotion.asp?prom o=1484&redir=1&rl=1

This book will be there as a PDF in a few months, or you can buy it in dead tree format now.

Other books are also linked there.

Book summary

[MirrororriM]

1. Set up your own package server on the LAN - this means the package server will download from the internet. So you basically have ONE machine downloading from the net - the rest of the machines are done internally.

2. Next, set up your sources.list file to point only to that server.

3. 17 8 * * * root apt-get update; apt-get upgrade

4. ???

5. Profit!!!

apt-proxy

[Douglas Simmons]

For those of you running Debian networks with a lot of boxes, you can use apt-proxy to apt-update/upgrade and patch all the machines through one download.

Gentoo Linux anyone?

[wolverine1999]

What about Gentoo's system where you use emerge and emerge sync...
Did it get a mention, or not?

Re:Gentoo Linux anyone?

[YouCanCallMeAl]

The author was testing emerge, but as it's still compiling, he had no results by the publication deadline.

Package management includes testing.

[khasim]

Then you'd have to trust that the distro doesn't self destruct by patches breaking your vital (read mission critical here) services.

No trust allowed.

Before anything goes into production, it goes into test.

YOU are the one responsible if a package breaks a production server.

You can still set a cron job to auto-magically download and install the apps, but you'd point it to your own repository where you put only the packages that have passed your testing.

The more "mission critical" something is, the less you want to automate ANY process that changes ANYTHING on the OS or apps.

For our critical database server, I come in on the weekend and hand apply every patch. And that is AFTER those same patches have been applied to the test server.

Authors Wanted

[Bruce Perens]

We're looking for more authors. All books in the series are placed under the Open Publication License (commercial use permitted, it's a real Open Source license) and made available in source and unencrypted PDF three months after they get to bookstores. Paper copies sell through about as much as other books outside of the series. We get good placement in brick-and-mortar bookstores like Borders and Barnes and Noble. But IMO the biggest benefit to authors is that once sales die down your book won't be locked away while you don't have any rights - a common headache that technical book authors have. Open Source books are living books.

Interested? Write to Mark_Taub at Prenhall.com and say you're interested in being in the Perens series.

Bruce

Two to three steps needed.

[jd]

Step 1 is only required if some patches are optional AND either have to be applied in a certain order or can't be applied together at all. This step involved using the %pre phase of the RPM to roll back any changes that will clash with the patch that you want to install.

Step 2 has two parts. Files that simply overwrite existing files can be installed with no further change. There probably wouldn't be too many examples of those. The other step is to install patch files into a patch archive directory.

Step 3 has one mandatory part and one of two optional parts. The first part is to apply the binary patch(es). You're always going to need that. The second part is to re-insert patches rolled out by the %pre phase that can be rolled back in again at this point. This is only meaningful if there is a %pre phase. Finally, if there isn't a %pre phase, you want to clean up the patch archive directory.

You now have a binary patch RPM.

Re:I'd patch your book on Linux Patch Management

[wawannem]

Really, you are taking me too seriously.

My post was simply meant to make light of someone's attempt to write a book on a topic that seems trivial to me. Although my original comment was quite simple in nature, I was meaning to point to a versatile set of tools. IIRC, debian and the APT tools were developed because of Ian Murdoch's need to keep the Pixar render cluster up to date. Any 'debian in the datacenter' SysAdmin can tell you that the entire suite of APT tools is very handy. RedHat's recent attempt with RHN is nice as well, but from an evolution standpoint is still a bit behind APT, and Gentoo's Portage is nice as well, but, APT still has a bit of a head start on many of these tools.

One thing you do not mention is simply setting up your own repository. Depending on the size of your installation, this could be quite beneficial. I worked one job that required a 25ish-node datacenter with consistent installations of Linux. We set up our own repository using the packages we needed, and then left it up to the QE dep't to test new packages as they were released and they gave us the word when packages were ready to be pushed to our repository. Worked out quite nice and only required that we have a custom sources.list file. It was quite easy to maintain a uniform installation of Apache, and when I left, revisions of our application were being pushed with our APT repository.

Re:yawn: in linux, it's called a package, not a pa

[ahodgson]

You could run Gentoo, and get to do both ...

An alternative approach: Don't patch, use rsync

[bit01]

I like making all files on all machines on a LAN, excluding network addressing, electronic licensing and logs, bit-for-bit identical. Doing so massively reduces management overhead and improves management control.

I've managed networks of several hundred machines this way and it works well. I checksum all files and directories on all machines on a regular basis and if anything's different in time or space I find out why and make sure it doesn't happen again. I've found dozens of very obscure and troublesome software and hardware bugs this way, have very good uptime and I can concentrate on making sure the master machines are well configured rather than waste time trying to put out fires all over the network all the time. If individual machine classes need to have different configurations I partition those differences out and manage them separately

Distributing patch packages is error prone. By working at the file level it's easy to be confident everything is okay. You can also often distribute and back out "patches" (just a list of files to be rsync'ed) in the background very quickly at short notice with minimal impact on users.

---

Keep your options open!