Google Gets Slack with Software Updates

An anonymous reader writes "While Google's open source project titled 'Slack' was released over a year ago, last week's Australian Unix Users Group Conference marks the first time that Google has ever discussed the system in public. Corporate systems administrator Michael Still helped to illuminate a little bit about how Google uses Slack and how their network of computers fits together. From the article: '"Slack is a source deployment system and it's the way we install applications on servers," Still said, adding Slack is based around a centralized configuration repository which is then deployed onto selected machines in a "pull" method. Each of the "worker" machines asks for its new configuration regularly or when a manual command is run.'"

Comments

Name infringement?

[From A Far Away Land]

How closely is Google Slack, tied to Slackware? If it's not, why did they choose that name? The Slashdot icon and URL suggests this story has to do with Linux.

Re:Name infringement?

[brunascle]

i was thinking the same thing, but i dont think it's tied. from the project page:

slack is a configuration management system designed to appeal to lazy admins (like me).

Re:Name infringement?

[Noryungi]

You mean... Google uses Slackware Linux? AH! I knew it! :-)

More seriously, re-read TFA summary: Google Slack is a configuration manager, and not necessarily Slackware-Linux-related. OTOH, The fact that the Google engineers used that adjective may be an hommage to the oldest surviving Linux distribution.

"Slack" is, after all, a perfectly valid ajective in the English language. Even though it has deep religious overtones for the Subgeniuses out there... This being said, Slackware is still, for me one of the best Linux distribution out there.

Re:Name infringement?

[AslanTheMentat]

"Slack" isn't a trademark first of all (AFAIK anyhow), but it may very well be something of an inside joke with some of the SubGeniues that Google has working for them...

from the wikipedia article:

The central belief in the Church is the pursuit of Slack, which generally stands for the sense of freedom, independence, and original thinking that comes when you achieve your personal goals. The Church states that we are all born with Original Slack, but that Slack has been stolen from us by a worldwide conspiracy of normal people, or "pinks". The Church encourages originality and frowns on actions seen as pinkness, which happens when one bows down to authority and the accepted limits of society. Popular Church phrases supporting these goals are "The SubGenius Must Have Slack" and "Fuck 'Em If They Can't Take A Joke."
The Linux distribution Slackware is named for Slack.

Fnord!

Re:Name infringement?

[nsanders]

It's an acronym

S.L.A.C.K. - Sysadmins' Lazy Auto-Configuration Kit

Here's the source code

[Benley]

Before anybody says "Hey! Where's the source?!", let me just provide a link right now:

http://www.sundell.net/~alan/projects/slack/

Do me a favor and don't destroy sundell's server, or he's likely to hurt me :-P

Re:FP

[nmb3000]

Google is all over the news these days. They should go on vacation.

s/News/Slashdot is a little more accurate :)

Maybe Google bought OSTG when nobody was looking. Might help explain why Taco's omelet is a little heavy on the Google sauce lately.

Makes sense

[frosty_tsm]

Google has what, 700,000 servers? I would imagine that along the way they would have found existing solutions inadequate. Now they are making a version of their tool available other developers.

Re:Makes sense

[charlesnw]

1) The number of servers being used at any given time is pretty transparent. Do you think the average Google engineer has a way to do an actual count? Google's implementation of GFS (http://labs.google.com/papers/gfs.html) indicates that a user doesn't really have an idea of what server their using at what time... or if they're even using the same servers they were the day before. The actual filesystem is distributed and mirrored.

Well I imagine that accounting would have an accurate idea. And I was interviewing for a data center position. These people work with servers on a regular basis.

2) With the kind of growth that Google is known for (building of new datacenters, new offices, etc. all the time) the number is probably rapidly growing. From the time the engineer heard that unmber to the time when he leaked it to you to the day you're posting this, who knows where it has gone.

Very true. This was less then a year ago. But your right they do grow at a fair clip.

3) For what were you interviewing? Production? Corporate IT? Engineering? QA? What servers were you talking about? Do you think these are all sitting in the same place being used for the same things?

Corporate IT. I am not so stupid as to presume all the servers sit in place doing one thing. I work in a very large envrionment with multiple data centers and know that server counts are meaningless when you talk about things like virtualization etc.

Re:Makes sense

[Benley]

You (and most others here) are largely missing the point. Slack is not a replacement for apt or rpm. It's more comparable to something like cfengine, although it's only vaguely in the same universe because Slack is so much simpler. Here's what goes in a slack role:

• a preinstall script
• a set of files (config files, little scripts, etc)
• a "fixfiles" script for setting permissions on the files before they're moved into place
• a postinstall script

That's it. You don't tar it up, you don't make a package, you don't have to learn a special language to describe your changes, etc. Nothing fancy. I know those four items sound similar to what you can do with a package management system, and they are, but that's not what this is for. You don't put binaries (for example) in a Slack role - slack is for higher-level stuff like your customized config files. You *could* roll custom packages for your config files and whatnot, but that's a lot of work (relatively speaking).

Say I've got a slack role for my web server. In the preinstall script, I'd make sure the right packages are installed, so I'd do something like "apt-get install apache2". Then in the files section, I'd have etc/apache2/httpd.conf along with whatever other config files I use for my installation. In the postinstall script I could check to see if I've just installed new config files and restart apache if I have.

That's all there is to it - it's purposely extremely simple, to make it easier than just copying files from an existing server to a new one when you set one up. Laziness usually wins out (a law of human nature, not a statement about google sysadmins :-P), so if config management is too much work, it either has to be forced upon everybody or it won't be used. Additionally, since you don't do anything with the files and scripts you put in a slack role, you can trivially put your slack repository in Subversion or CVS or your revision control du jour, so getting people to use Slack has the nice side effect of also getting them to use revision control for their config files!

And yes, I use it on my personal server with two admins :-P

Sounds similar to our setup

[nsanders]

We run Ubuntu in my department and ended up building an in-house Apt repository/svn/rsync system to maintain all our machines. We also use custom scripts that monitor NFS shares to emergency push operations. Obviously our down side is that an entire .deb package must be rebuilt for each change, but it's nice to see Google's method isn't out of this world after all.

Coral Cache

[dch24]

I doubt it will get too slashdotted, but just in case, this is the link with the downloads coming from coral cache:

Index of /~alan/projects/slack

[TXT] COPYING 13-Oct-2006 00:09 1k
[TXT] ChangeLog 13-Oct-2006 00:09 7k
[ ] slack-0.13.1.tar.gz 08-Jan-2005 20:01 28k
[ ] slack-0.13.2.tar.gz 09-Feb-2005 11:27 28k
[ ] slack-0.14.0.tar.gz 13-Oct-2006 00:09 47k

Short Description:

slack is a configuration management system designed to appeal to lazy
admins (like me). It's an evolution from the usual "put files in some
central directory" that is faily common practice. It's descended from an
earlier system I also wrote, called "subsets", and uses a multi-stage
rsync to fix some of the problems I had there.

Basically, it's a glorified wrapper around rsync.

License:
See the file COPYING.

Getting slack:
http://www.sundell.net/~alan/projects/slack/

Documentation:
Not much, but there's some in doc/

Reporting problems:
Send an email to <sundell (at gmail.com)>. Probably want to put
"slack" in the subject and be patient for replies. :)

$Id: README,v 1.5 2006/09/25 21:35:22 alan Exp $

Re:Confusing title?

[ardran]

FWIW, the name is an acronym: SLACK - Sysadmins' Lazy Auto-Configuration Kit

I knew it!

[Rachel Lucid]

The SubGenius must have slack.
Google makes Slack.
Ergo,
Google is the corporate incarnation of 'Bob'!

... What? Someone had to say it!

Re:FP

[sammy baby]

Might help explain why Taco's omelet is a little heavy on the Google sauce lately.

Thanks. All the bleach in the world isn't going to help me with that mental image.

Not new

[HavokDevNull]

Sounds very much like CFEngine http://www.cfengine.org/ with subversion?

Kinda similar to APT

[Tarlus]

What they're describing there is like Debian's apt-get, or BSD's portage (also incorporated in Gentoo). Each of those technologies I listed are most commonly used to install packages from a common central repository over the internet (per distro). But, each of them can be configured to retrieve and install packages from a localized server. It's actually a very handy thing to have in large-scale networks because instead of installing and updating packages on each machine over the internet from a remote repository, you just update one machine from the internet and let all the rest of them update from it, thus cutting way back in internet usage and greatly reducing the time needed for the local machines to download and install updates.

Sounds like Slack is a simplified version of all of that.

Re:Kinda similar to APT

[nsanders]

Some what. We use APT here in our department. One of the big changes from SLACK is that it doesn't require you to compile packages like RPMs or DEBs. We have a couple of very large applications that take up 1-3GB and it takes a very long time to rebuild the packages. Slack negates such a need.

Re:Kinda similar to APT

[apachetoolbox]

For the record *BSD did NOT create the gentoo portage system.

FreeBSD = Ports Collection
Gentoo = Portage

Re:So its part of the Active Directory for Unix

[Zombie Ryushu]

You mean OpenLDAP, Samba, Kerberos, Bind.
I can give you one better.

I use Mandriva Linux as my Domain Controllers and workstations.

With urpmi's parallel operation with SSH support is a Godsend. See my Secure Shell server is GSSAPI enabled (Kerberos.) Because of the fact every machine is authorized by LDAP, and authenticated by Kerberos, I can do this:

urpmi samba-server --auto --parallel dcs

This will install Samba on all Domain Controllers

urpmi gnumeric --auto --parallel all

This will install gnumeric on every machine in my Domain.

urpmi.update -a
urpmi --auto --auto-select --parallel all

This will update every machine I have in my Domain while resolving dependancies. There are problems with doing it this way. The big one is, under AD, updates can be pushed to offline machines. For this to work, all machiess must be online.

Also this does not update the urpmi catalogue synthesis.

radmind

[Anonymous Coward]

As soon as I read this I immediately though of radmind, which, by the vague descriptions seems to do exactly what is going on above. I encourage everyone to take a look!

It may be an homage, true

[mister_jpeg]

Or maybe the googadmins have a sense of humor.

Naming an automated package management/software update system after Slackware? That's cute.

BZZZT!

[SanityInAnarchy]

What they're describing there is like Debian's apt-get, or BSD's portage (also incorporated in Gentoo).

Gentoo is not Ports. Ports is not Gentoo. Gentoo is inspired by BSD's ports, which is why the Gentoo package manager is named portage. But there are huge differences.

Last I checked, Ports was primarily a distribution system. Portage is a full-fledged package manager that happens to use source packages, and happens to have a file called "make.conf".

Each of those technologies I listed are most commonly used to install packages from a common central repository over the internet (per distro). But, each of them can be configured to retrieve and install packages from a localized server.

True, and they mention this. Gentoo's probably the closest, but their main motivation for rolling their own is to be able to go into a filesystem and tweak it, instead of having to tweak, build an RPM (or Deb, or whatever), download that to the target machine, etc etc.

It's also simultaneously a like cfengine, which is used to manage configurations. That is, it's not just for packages, but also for app configurations. That is, if your webserver goes down, hard, you can bring a new one up in an hour, 100% automated, and it will be configured the exact same way as the old one. If you need to add a new webserver to the cluster, same process. Want to change the config on a webserver? Test it on a local machine, then put the new httpd.conf on the Slack server, and watch all the frontend machines download it.

Personally, I think they should've at least looked into cfengine, but APT alone is not enough. Saying APT could do it is like saying you can write a webserver in assembly -- yes, you can, but why would you want to?

Re:BZZZT!

[drsmithy]

Personally, I think they should've at least looked into cfengine [...]

Maybe they did ? I looked into cfengine for automating our server configurations and my head asploded.

Then I experimenting with it for a few weeks, but I could never quite grok how it was supposed to work and always had the feeling that "this is as much work as managing all the systems manually anyway". It seems extremely capable, but it's very difficult to use if you want to do anything that's more trivial than pushing (or pulling) a bunch of files (for which some rsync fiddling is _vastly_ easier).

Mod Parent Up; Not a troll

[Anonymous Coward]

The parent is clearly not a troll, folks.

The distribution tarball is 28k. This is a tiny little one-man project that is little more than exactly what the parent said. There are 90234580928345 other dumb little rsync wrappers on sourceforge, many of which are MORE functional. Why focus on this one?

I'm not the only one who is getting sick and tired of everyone pulling out their e-penis at the slightest mention of Google and jerking vigorously. Nor am I the only one sick of half of all slashdot articles being nothing but spew from the Google spin machine.

Re:So its part of the Active Directory for Unix

[0racle]

No, its not possible. It's true (and I know) that OpenLDAP, Kerberos, Bind and a Network File system will give you a some of the functionality of the AD, if all you want is SSO and someway to centralize Automount settings and Printers. However doing DFS is not simple, pushing updates and revoking them is not as straight forward or robust, as the other poster pointed out and there is nothing like Group Policy.

This project seems to just provide another way to push applications to systems, presumably with the same limitation as any other, excepting maybe for Zen, where all your systems must be from the same vendor. It's from Google though so it must be gold.

CFengine

[Pegasus]

How exactly does this compare to CFengine? From the short slack description it would seem like cfengine is a much more mature solution ...

how is that different/better than...

[oohshiny]

There are a bunch of widely used systems like this: rdist, cfengine, fai, ... In what way is "Slack" supposed to be better? Or is this simply a case of NIH?