Does launchd Beat cron?

Blamemyparents writes "For those who aren't Appleheads, you may not have heard that with Tiger, Apple swapped out old Unix standby cron for their own creation, launchd (Apple mentions it on their OS X page and has the man page for it up as well). Seems like it's a bit nerdy, but this changes a LOT about how *nix systems have done things for many years. Launchd is Apple's replacement for quite a few utilities, including launching and quitting quite a few different things, and getting info from the system and other running processes. This page from Ars Technica talks a bit more in depth about it. Apple has open sourced the thing, and is apparently hoping all the unix kids will take a look."

Comments

Flexibility

[Anonymous Coward]

But can it be piped?

Not a cron replacement, a init replacement

[Anthony Liguori]

Simply referring to launchd as a cron replacement is a major understatement. launchd runs as the init process and according to TFA was primarily made to replace the /etc/rc.d scripts during startup.

This is somewhat understandable for something like OS X. Doing something simple like displaying a GUI detailing startup is terribly difficult with /etc/rc.d scripts (considering the length of pause an fsck would take--users would certainly assume there system hung during boot).

I'm not sure launchd is something you'd want in 99% of Linux installs but if you're looking for a end-to-end user-oriented desktop I can see how a technology like this is necessary.

I'm not sure Apple Gets It though. Why in the world would they use XML configs? Gesh.

Re:Not a cron replacement, a init replacement

[Anonymous Coward]

No, you don't get it. Why in hell would you want to write yet another config file parser and use your own awkward config format when you could use standard XML and that very same parser that's already loaded to system RAM as a shared lib?

Re:Not a cron replacement, a init replacement

[Senjutsu]

I'm not sure Apple Gets It though. Why in the world would they use XML configs?

To make it easy to edit the files both by hand and via utilities which can be written very easily by leveraging existing XML parsing libraries?

Re:Not a cron replacement, a init replacement

[gstoddart]

Because XML configuration files are self-validating.

As opposed to all of those depressed, self-loathing DOC files out there?

Re:Not a cron replacement, a init replacement

[As Seen On TV]

Is there any reason why a daemon is necessary to improve startup time?

The launchd service isn't a daemon. It's a replacement for init. And yes, it speeds things up considerably. Previously, every startup script required its own instance of a shell to be fired off. Now that middleman is removed. And it made things considerably faster and simpler.

But why XML over say something like INI or a much simplier format?

Because XML is self-validating, like I said. The contents of the file can be checked for consistency before being used for anything.

It also quickly becomes unreadable.

You don know that we're talking about property lists here, right? It's an XML-based format, but it's not totally free-form XML. You can't just plug in any valid XML and expect launchd to handle it. If the file doesn't validate against the property list schema, launchd will just ignore it.

Besides, we've been using property lists everywhere for five years now. Nobody's complained yet. ;-)

Re:Not a cron replacement, a init replacement

[JQuick]

The launchd source code is a collection of programs which span several related areas. In the broadest terms the contents of the distribution cover init, SystemStarter, and launchd.

Most people talking about launchd in the context of parallel system startup are actually talking about SystemStarter. Also note that the plist files which drive SystemStarter can be in one of two forms. The most common of these is the NeXT style textual plist format which is very human friendly and is not XML-like in structure or appearance.

As a replacement for aither BSD style or System 5 style rc scripts, SystemStarter is a definite improvement, both in terms of ease of use, speed, and reliability. The launchd subsystem uses XML formatted property lists. I am unsure whether NeXT style plists are supported for these.

Regardless, you were dismissive of launchd claiming that it was mere init replacement. You thought you were speaking about launchd but sound like you had only heard about SystemStarter.

launchd itself it a super-server which reads process definitions. These definitions can describe processes to be run using a set of predefined keys. Some examples of key value pairs are: Program/ (path to executable), or ProgramArguments/ (The array of strings to pass as arguments to the program). The interesting thing about launchd is that the predefined key/value space is so rich, and so well considered.

The keys span several related domains which Unix users associate with a number of different tools and data sources. Here are a list of some interesting areas covered.

like sudo or vixie cron the user and group under which to launch the program can be specified using any of { UserName, UID, GroupName, GID }

WorkingDirectory causes a chdir(2) before executing the job.
RootDirectory specieis a chroot(2) for the child process.

StdOutPath and StdErrPath can specify where to redirect output of the program.

OnDemand specifies whether to run only as needed or ensure that a server process should always be available, thus respawned.

Labels and Groups provide a way to refer to processes with greater specificity or to group related processes together for reporting or action using launchctl.

A set of time specific keywords cover a superset of at and cron time specification.

Soft and hard resource limits for the job can be tuned using keys that provide acces to all ulimit functionality (file descriptor file size, memory footprint, core size, etc.. Similarly the nice value can be specified to constrain CPU usage, and the *PriorityIO keys specify what priority the kernel should use for scheduling disk IO for this task.

The Socket Key defines an optional dictionary defining IPC behaviors controlling combinations of open/bind/connect/listen. (think inetd and xinetd). Beyond this you can also specify other things here like "Bonjour" to register this service with mDNSResponder. If a SockPathName is specified attempts to open a named unix domain socket will cause this program to launch.

If these are not enough, a launchd plist can start/stop a program when a watched file changes. When passed the path to a directory, the program can be started when the program is not running and the directory is not empty. Think about this one... A robust queue processing program might be simply:
#!/bin/sh
lockfile $SOMELOCKFILE
for file in *
do
process "$file"
done

WorkingDirectory and a watched directory path, along with whatever CPU/Disk IO/Memory/user/group are all handled by launchd. Similarly launchd ensures that the process runs when there is work to do, and is never launched unless there is work to do.

Finally, if the predefined keys and actions are insufficient you can call a small number of stubs implemented in Tcl to set either one shot timers or interval timers at which point the scripts is re-interpreted. You can pass messages to syslogd, start or stop jobs. A form of system introspection is also pr

Re:Not a cron replacement, a init replacement

[As Seen On TV]

There is no access layer that surrounds a plain text file.

In fact, there is. With Core Foundation, the process of unserializing a property list is atomic. The bytes are read from disk and validated. If and only if the file validates, the data structure is populated and returned by reference, all in one function call.

So yeah, property lists are self-validating.

You have to be DAMN sure that the file is easy to understand and edit by an admin, even if he only has "ed" over a serial connection available.

You mean like the universal, thoroughly documented and self-validating property list format, for example?

I'd much rather work with: TAG:VALUE

Great ... right up to the point where "TAG" has a colon in it, or whatever your delimiter. Or where "TAG" or "VALUE" needs to be in Japanese. Or where "VALUE" needs to be a container like a CFArray or a CFDictionary.

Or, for that matter, where "VALUE" needs to be of a specific type. If you're stuck with "TAG:VALUE" then you're kinda screwed when it comes to explicit type-checking, aren't you?

I can confidently write a program to validate and parse inittabs, but I don't know how to that with XML

You aren't expected to. Core Foundation does it for you. There's one function call to serialize a data structure and one function call to un-serialize it.

XML files, are difficult to read, difficult to edit, difficult to edit with programs, and poorly-defined.

Come on, now. Isn't that kind of a stupid statement? XML files are spectacularly easy to read, especially compared to the random mish-mash of configuration-file formats available on Unix. Editing is trivial, of course, because it's just text. More importantly, editing is trivial wherever you are, because the files are UTF-8 and not just ASCII. And calling a document that includes a machine-readable DTD for automatic validation inside it "poorly defined" borders on the farcical.

Some of us have been complaining since the NeXT days. :\

That's funny. Because there were no XML property lists in NEXTSTEP. They were added for Mac OS X.

Makes booting DAMN fast

[the_2nd_coming]

the machine boots to a desktop (auto log in and all) in less than 15 seconds.

This sentence no verb?

[Anonymous Coward]

I have nothing to contribute to this discussion, except for my opinion on how ambiguous headlines are. For example, "Does launched beat cron?" has the following possible meanings (yes, I see it's launchd, but listen):
1) Does as a verb, launchd as n, beat as v, cron as n. That is, a question whether or not the thing "launchd" beats the thing "cron".
2) Does as a verb, launched as adj, Beat as n, cron as v. That is, a question whether or not thing Beat, which has been launched, crons. (Well, "croon" is a verb, so maybe "cron" is too).
3) Does as a noun, launched as v, beat as adj, cron as n. That is, a statement that female deer have launched "Beat cron" -- cron that has been beaten, perhaps.
Any others?

Also check out Solaris SMF

[Wesley Felter]

Sun came up with (at first glance) a similar thing called Service Management Facility in Solaris 10.

As long as it doesn't break the fundamentals...

[yorick]

Apple is doing a good thing here is shaking up the administrative and setup side of the UNIX world. Most of their work was actually NeXT's, but their "Application as directory" concept is wonderful in a lot of ways, and their reorganizing of the filesystem so that it makes more sense to users is probably a good move in the long run.

What makes UNIX superior for management and power-users is the ability to do everything you need to from command-line tools and options, and the fact that the storage for configuration is in understandable, alterable files. If those are still there...if I can still run my Apple from the shell--I'm a happy man.

The only exception to this would be cases where a vendor deliberately makes a deviation simply to introduce incompatibilities. I don't see that as the case here.

Re:Submitter is confused

[As Seen On TV]

The launchd service replaces (takes a deep breath) init, rc, the init.d and rc.d scripts, SystemStarter, inetd and xinetd, atd, crond, and watchdog.

Basically any task the needs to run on its own, rather than being manually started by a human being, can be handled by launchd.

Re:From the apple page:

[netsrek]

So nothing like cron what-so-ever. Another great piece of Timothy's fact checking.

Before you jump the gun:

netstrek@computer: ~ $ cat /etc/crontab
# The periodic and atrun jobs have moved to launchd jobs
# See /System/Library/LaunchDaemons
#
# minute hour mday month wday who command

So yes, it is a cron replacement.

Re:Ahh yes.

[Cid Highwind]

You move away from human readability/editability and into the realm where only a machine can fathom the format.

You're looking at this from the perspective of someone who understands and remembers the differences in a dozen config file formats. Most people don't.

Do comments in my .whateverrc start with a hash mark? A semicolon? A double slash? C-style /* ... */ delimiters? XML-style ? Are you even allowed to have comments? Can you use nested parentheses? Are angle brackets a symbol for redirecting i/o or do they delimit XML tags? Does a pipe chain the output of cone command to the input of another, or is it a logical or operator? Replacing that mess with a single standard format will make things *more* human readable IMHO, and it will certainly make it easier to have a single integrated pointy-clicky interface for changing all those configuration options.

Re:Less Compatible?

[revscat]

Isn't half the attraction to OS X for geeks how its not that much different from Linux or BSD from the console?

I'm sure some do. But in the aggregate most users will balance something like this against the benefits gained. There are those who are also experimental enough who will try this, and if they find it to be a better implementation will then port it over to other *nix variants, leading to a new "standard".

rc.d, cron, etc. may be the way other unices do things right now, but that in and of itself is not justification enough to be married to such mechanisms in perpetuity. Sometimes it's beneficial to *cough* you know, "think different."

*sigh*

[cbiffle]

This thread, so far, seems to be filled mostly with "Ah! Someone's doing something differently than UNIX has for 30 years! My knee is jerking!" So I feel I should respond.

launchd is neat. It's not simply a different way of doing the same things, it lets you do different things.

Like automatically evaluating dependencies between daemons, starting them in the right order, and running them in parallel if needed. FreeBSD's the only other OSS system I've ever seen do this; Gentoo does the dependencies but not the parallel startup. (Which is annoying while it's, say, trying to get an address from a nonexistent DHCP server.) Long story short, it dramatically reduces boot time, while eliminating dependency hacks like runlevels and numbered scripts. (Not that BSD had them.)

For those of you who posted without reading the manpage (or administering an OSX system), it also lets you do init-style startup tasks on a per-user basis. You can configure it to start daemons and other processes on the behalf of users as the log in, and shut them down -- gracefully, not by TERM; KILL -- when the user logs out.

Anyone who's ever dealt with the myriad of hacks to get ssh-agent in place will understand why this is good.

There's a lot of resentment these days toward anyone who does something that's perceived as "not the UNIX way." Change is the only way to innovation, people; perhaps the UNIX way is broader than you thought?

Re:*sigh*

[enthused i swear]

Gentoo will do parallel startup, though it is not set to by default:

Modify your /etc/conf.d/rc file thusly:

# Set to "yes" if you want the rc system to try and start services
# in parallel for slight speed improvement.

RC_PARALLEL_STARTUP="yes"

Re:From the apple page:

[De Lemming]

Parent did not state it was no replacement for cron, but it is "nothing like cron what-so-ever." It's actually a replacement for "init, rc, the init.d and rc.d scripts, SystemStarter, inetd and xinetd, atd, crond, and watchdog" (copy/paste from another post, this is all explained thoroughly in the Ars Technica article).

It's about time. Linux needs something similar

[Nelson]

Cron and init and rc.X work fine, I'm not a hater. However I do think it's time to move forward. I like what Sun is doing with SMF and this is Apple's cut at it.

Basically, cron needs higher resolution, that's my only beef with it. It also seems like to do anything facny with cron you end up writing a program to do it and it's not that uncommon to do that.

The startup scripts need an all together different kind of overhaul. I've been working on Linux appliances for over 6 years now, without exception, it seems like someone ends up writing some kind of "health script" that is kicked off by cron every minute or a few or is a daemon in it's own right that watches for things to crash or not be running and then it restarts them. I've seen it in a production set-top box based on Linux where we essentially wrote our own init and had it treat some processes special and 5 different software appliances. Fact of life is software crashes from time to time. These scripts then do something else, like ping the gateway or something stupid to check if the network went down before they do what it is that they do. To make matters worse, they are always written in Perl by some guy who doesn't know Bourne shell to write a good startup script in the first place; that's the part that chaps me. Rather than contaminate the system with all this extra shit we should just have an easily extensible and configurable system process starter and monitor and it shouldn't require scripting to do anything advanced.

• Use XML for configuration, it's easy to parse, it's here to stay and I think it's a little bit better than the columns of a crontab because it's more verbose.
• Have variable restart behavior, restart, restart and restart dependancies, reboot, just let processes die, etc..
• Have dependancies (start the database before the shit that inserts data in to it, even though it shouldn't matter, it does) and start them in the right order.
• Kill dependencies when I kill a process
• Smart killing, if something doesn't gracefully go down, 9 it down.
• Have some kind of health monitoring call back, at a time I specify, execute a program that will check to see if a process is okay.
• Once all this is done, maybe support something like soft cycling, kill everything down to single user mode and then bring it all back.
• Variable dependency notification, if I restart my database allow me to do nothing to dependant processes, restart them, or HUP them, or something I define.
• Full featured logging.
• At some configurable interval, sniff out all of the XML configs in this "next gen startup proc" config directory, like every 5 minutes just check for changes and if needed, start something new up.
• Allow for user to be specified in the config
• Allow for end users to have thier own daemons defined by files in their home directories.
• Then finally, allow for cron like behavior and run tranient tasks with all the flexibility of cron.
• Proc specific ulimits would be nice also...
• And some form of runlevel logic still.
• And startup and kill in parallel if allowed by the depends...
• And it should have a template for common daemon type things I should just provide a startup command, arguments, a user to run it as, and a run level.

There are probably some things I forgot, it's really not that much, I could see bangin' this out in Python or java in not that much code. My thinking is that instead of checking for a PID file or grepping through ps outout to provide "status" you query a running process and it tells you your proc is up and running. Yeah yeah, I know, shit shouldn't crash and whatever. I've just seen such a shitty job of this stuff being done over and over and even on fairly stock installs of major linux distributions I've seen service

Re:how can it automatically know dependencies?

[sean23007]

The programmer or administrator is responsible for putting that information in the XML file. That's why they're using XML. It allows for more data to be available. Where it (init) used to basically be a list of programs to launch, it (launchd) is now a system whereby a collection of programs are launched. It is much more flexible, and since it can launch programs concurrently, much faster as well.

Re:Wait, wait, wait...

[Paradise Pete]

If you add all the minutes wasted over all the people who use the OS over its life, that's a hell of a lot of man-hours.

Yeah, no kidding. I must boot at least once every few months or so, so pretty soon that's going to add up to...uh, a couple of minutes.

Re:Submitter is confused

[As Seen On TV]

The "get toasted" expression wasn't my choice of words. I was quoting. I would have said, instead, "what happens if, say, you fat-finger inittab while editing it."

The problem with traditional Unix configuration files is that they're not self-validating. A single typographical error can very easily result in an unbootable system. XML-based property lists reduce that possibility. They don't eliminate it entirely, of course; nothing could. But they reduce it.

They also open up the door for us to make the launch system self-repairing. Check out the changes to NSUserDefaults in Tiger to see what I mean there.

Re:Open Source? Where's the source?

[Eridius]

It's part of Darwin. Try this page: http://www.opensource.apple.com/darwinsource/10.4/

Re:XML plist configs are easy

[cakoose]

If you can't figure out an xml plist config, then you are not smart enough to be in the shell.

1. It's not about figuring it out. Just because you can figure something out doesn't make it optimal or even non-crap.

You scared people clearly don't understand plist files and how they represent records and lists.

2. The grandparent post was talking about XML in general and how it's great for configuration data. That is what I was taking issue with. You can build record/list support on top of XML, but it doesn't fit in naturally.

3. I still think the property list XML format is crap. Property lists are decent; they're not perfect, but their extreme simplicity makes them very useful. The ASCII format is great, so why would you want an XML format?

Does having an XML format lets you leverage XML tools? It might have if property list XML worked with XML instead of simply beating it into submission. Apple created a completely opaque layer on top of XML instead of integrating directly.

{
Dogs = (
. {
. . Name = "Scooby Doo"
. . Age = 43
. . Colors = (Brown, Black)
. }
)
}

A natural XML equivalent would be a little more verbose:

<Config>
<Dogs>
. <Dog>
. . <Name>Scooby Doo</Name>
. . <Age>43</Age>
. . <Colors>
. . . <Color>Black</Color>
. . . <Color>Brown</Color>
. . </Colors>
. </Dog>
</Dogs>
</Config>

Using Apple's XML format, you'll get:

<dict>
<key>Dogs</key>
<array>
. <dict>
. . <key>Name</key>
. . <string>Scooby Doo</string>
. . <key>Age</key>
. . <integer>43</integer>
. . <key>Colors</key>
. . <array>
. . . <string>Brown</string>
. . . <string>Black</string>
. . </array>
. </dict>
</array>
</dict>

The result: something that's more verbose and harder to read than even a natural XML format.

What's worse is that you can't leverage DTD/Schema as much as you could with a natural XML format.

• You can't validate semantic structure. (You can ensure that the file is a valid property list XML file, but you can't make sure that it contains launchd configuration info).
• DTD/Schema sensitive editors still wont help you out with semantic structure. It can auto-complete "key", "dict", etc. but it can't auto complete "Name", "Age", or "Color".

Re:As Seen On TV obviously needs attention ("we"??

[jkh]

Well, since you mention it.. If you're reading this, ASOTV, please drop me an email. Nobody's going to eat you, but we'd like to have a discrete word... Thanks.