Slashdot Mirror
Replacing the Aging Init Procedure on Linux
SmellsLikeTeenGarlic writes "Seth Nickell (of Storage and Gnome HIG fame) has started a new project which aims to replace the aging Init system on Linux. OSNews has more details on the project, directly from Seth. The new Python-based approach will make booting faster and it will talk to the D-BUS daemon, freedesktop.org's leading project. And speaking of freedesktop.org, it is important to mention the release of HAL 0.1, an implementation of a hardware abstraction layer for KDE, XFce and Gnome, based on a proposal by freedesktop.org's founder Havoc Pennington and being implemented by David Zeuthen. It is innovative projects like Storage, SystemServices and HAL that can bring the kind of integration to the underlying system that current X11 desktop environments lack."
The current init system is actually fine in my opinion, but if someone comes up with better system that can boot my Linux system as fast as my XP system boots, I'm game.
I like python a lot, but why make it a requirement for init ? Just means more stuff has to be installed fort he default system to work. I prefer to sue the base shell.
SystemServices is not at all tied to Gnome. It will probably not require much more than the kernel and Python. His goal is partly to make a nice set of APIs callable from a desktop like Gnome to ease with management and error reporting. This project is not tightly integrated with Gnome just because someone from Gnome has started it.
Developers: We can use your help.
Back in my day, young whippersnappers didn't show disrespect to their
elders by talking about replacing them with fancy-pants code. We were
shell scripts and we liked it.
Not Bash scripts either, real life ksh scripts. You hippies and your
bash shell.. why I outta.. ouch, hurt my arm waving it so.
Anywho.. back in the day when we were happy little startup systems
we didn't have to worry about snakes. Python? Who ever heard of that
back then? We has ksh and csh and WE LIKED IT.
Damn granola-crunching kids and their need to improve what don't need
improving.. Stay in school, don't become pot-junkies and enjoy the
blessed rc that nature intended.
It started with all this Rock and/or Roll that you ingrates listen to.
Back in the 70's we rc scripts were appreciated. Then along comes this
electric gee-tar thing getting so popular and that did it.
Dag nabbit... where's my crontab..
where was I? Oh yeah..
Back in my day, young whippersnappers didn't show disrespect to their
elders by talking about replacing them with fancy-pants code. We were
shell scripts and we liked it.
.
.
.
It's "faster", because it's "python based"?!
The standard Linux init system is based on sysvinit and is slow precisely _because_ it is interpreted (it's basically a ton of shell scripts). The other reason why it's so slow is because glibc is slow and the init system starts several hundred processes during the init process. Just log in on a freshly restarted Linux system and type "echo $$" in a shell prompt to see how many programs were run before you logged in. On my minit based notebook, the number is below 20. On my minit based server, it's still below 30.
minit takes less than one second to initialize the whole server system, on an aging 466 MHz Celeron box, right from the point where the kernel starts init up to the login prompt. And the server does file sharing, cvs serving, rsync serving, runs a mail server and sshd.
In fact, because minit does not even depend on glibc, minit can probably initialize a small system in less time than it takes to even load python and glibc on this init system.
Fast and python based, give me a break. And the freedesktop people should keep their bloat to themselves, if you ask me. With the notable exception of KDE, all the gui systems on Linux have gotten progressively slower and more bloated over the years. KDE has also become slower, but less drastically, so it can be excused IMHO. But Gtk? Give me a break! Even starting the gnome theme engine takes 5 seconds on my 2 GHz Athlon XP!
A replacement for the current init system, while necessary, should have fewer, if possible, dependencies than the current init, not more. Unices are being deployed across more and more diverse kinds of systems, and dependencies on python and d-bus, both of which projects I support in themselves, are not going to be welcome in the init of the majority of unix systems today, especially in servers or embedded systems.
As far as "most" sysadmins not understanding run levels, he's out of his mind. Maybe he doesn't get it, but it's a long standing thing that works well. In fact, it works SO well, that Linux adopted it from System V after using the older monolithic rc scripts for a long while.
7 November 2006: The day Americans realized corruption and incompetence weren't addressing 11 September 2001
The relevant one here is the management of services -- far easier than anything similar I've ever dealt with in the Unix world. I realize that 1) this proposed replacement goes beyond what the Gentoo system manages and 2) real sysadmins running real servers aren't going to leave the existing system any time soon, but for desktop users today, it's a great advance that doesn't get the credit it deserves.
What I'm listening to now on Pandora...
His approach isn't very *nix-ish, it is very Windows-ish. It assumes you have a GUI. It relies on a large complex framework of interfaces. It assumes you have Python scripting. This may work very well for many desktop distros, but it can't become some great unifying thing like he wants, since he chose dependencies that are not the least common denominator. I believe that his goals can all be achieved with minor changes to the existing init system, rather than a megalithic rewrite.
This approach sounds like he is trying to push some specific technologies he is interested in, and so he decided a new init system that uses them would be a nice PR way to push them.
So I skimmed through the D-BUS spec and, as I expected, they are simply reinventing CORBA.
When will Open Source developers figure out that just because the OS community didn't come up with a technology, doesn't mean it has to be re-written with fewer features?
I gaurantee that whatever aspects of CORBA the D-BUS developers found unnacceptable - complexity, overhead - will be reintroduced into D-BUS by the time it reaches maturity. That's just how these things go - someone decides that Standard X is "cool, but too complicated", and then five years later they realize that their solution has become just as complicated as Standard X because, lo and behold, all that complexity was there for a reason. Real-world solutions never stay simple, because real-world problems aren't simple.
--
CPAN rules. - Guido van Rossum
If someone comes up with a whizbang new boot system. great. Just make sure that if something goes wrong, or something needs to be changed, it's:
I don't want some horrific equivalent of the Windows Registry lurking in the background. There should be no mysteries about what gets started and when. I'm not a Windows guru, so maybe this stuff is easy to determine in XP or Server 2003, but I've always found plain ol' text files to be much easier to deal with than fancy-dancy databases. Or at least compile the databases from plain ol' text files.
The only init replacement i've seen that seem to have some sense is: http://www.atnf.csiro.au/people/rgooch/linux/boot- scripts/index.html
elFarto
I think Seth's idea is a good one. Of course, there are some things to refine: the dependency shouldn't be external (e.g. SystemService knowing the dependancy tree) but dynamic (e.g. GDM sees that its config requires network login, so it asks SystemService to start network), etc.
But overall rethinking the init is a good thing. Even just opening the debate is a very good thing. The mess of shell scripts is more a giant hack than a well-thought bootstrap system.
As an old school programner let me say I do not hate Python. I just do not know it. So far I have found noting about Python that makes me want to take the time to learn it.
I do tell people just learning to program that it sounds like a good first language.
For internal programming that requires a GUI I tend to use Java. Netbeans is just too good of an IDDE to give up.
For non-gui internal programs I tend to use Perl. Why perl over Python? I know Perl.
For programs that end up in the hands of our customers I use C++.
I really doubt that Python sucks. I have seen too many really good programs written in it.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
Apparently freedesktop.org has devolved from a desktop standards initiative to a home of pointless wheel-reinvention. Here's a list of the projects listed above, followed by their existing, more mature counterparts:
Init Replacements: simpleinit, minit, jinit, runit, daemontools, serel. Progeny also has their own system based on Gooch's need/provide architecture.
D-BUS: CORBA
HAL: Discover
--
CPAN rules. - Guido van Rossum
Oh, you mean until you find another OS that can "fake" boot as fast as XP...
XP has some serious issues in some cases with the way it "optimizes" the boot order. It appears quick, but in a corporate environment that can lead to weird timing problems. That's probably why MS left/added a feature to disallow logins until XP was really booted.
.sigs are for post^Hers.
Hmmm... Windows XP gives me a desktop (somewhat) more quickly than Gentoo does, but I have to wait for about 90 seconds to actually do anything or run a program.
It's a tortoise-and-hare thing; Windows (and lately Gnome) give you eyecandy early on at the cost of having to wait with what looks like a perfectly functional desktop while the rest of your base loads. That's why I use Windowmaker. Once I get to GDM it's rare that it takes more than 5 seconds to have a desktop with nothing stalling on me. YMMV, and this is fairly OT anyways since once I'm at GDM, init's not automatically loading stuff anymore.
All's true that is mistrusted
There already is a better init system: simpleinit. I've worked with it on embedded systems, and it rocks.
It's dependency-based, so you only start up the services you need. Read all about it.
Dependancies are a big improvement over runlevels, although you could implement runlevels on top of it.
ETLinux replaced init several years ago with an entirely scripted version. Different scripting engine (Tcl instead of Python) and for different reasons (keeping an embedded Linux kernel tiny.)
... added Tcl equivalents for some common syscalls and services: sys_dup, sys_exec, sys_fork, sys_kill, sys_nice, sys_pipe, sys_reboot, sys_sync, sys_wait, sys_chmod, sys_umask, sys_mknod, inp, inw, outp, outw, setleds, mount, umount, uudecode, uuencode, time, ifconfig, route, udp. The introduction of these commands allowed the rewrite of the initialization scripts in pure Tcl, avoiding the inclusion of large binary programs.
Making this happen was that the ETLinux folks
More on ETLinux architecture here.
Oww, gawd!
Not Emacs, vi!
SystemServices: Hardly wheel re-invention. If you read the linked-to article you'll find that none of the listed "init replacements" address any of my four major goals. They addressed their own goals, which are nice ones I'm sure, but not things that the desktop needs. In fact, the only thing they really add over init that's interesting to me is a dependency system + parallelization. And RH's internal work on init scripts has suggested this results in only small improvements (neighborhood of 30% I believe) in boot time... which is better but not dramatic. Dependency systems are pretty trivial to implement and a dime a dozen, in any case.
DBus: yeah, its a lot like CORBA. Its even more like KDE's DCOP. In fact, its a lot like DCOP. You could even think of it as a major iteration of DCOP. So look, GNOME has been using CORBA for IPC for several years and its still something people avoid using whenever possible. KDE used CORBA for a while, and even with the comparatively nice CORBA/C++ bindings found KDE devs avoided it when possible. CORBA is a freaking PITA to use for lightweight desktop-style stuff. DCOP was the solution (its a good solution, GNOME should have done this ages ago): make it easy enough to use that developers will actually readily communicate over DCOP. Communication protocols have no inherent value on their own. They acquire value when there's things to talk to. Developers won't use the API unless its simple. You can write very simple comm layers for KDE and GNOME around DBUS. Even if we GNOME folk wanted to use CORBA (we don't), KDE wouldn't, and a requirement for DBus being truly useful is that KDE+GNOME have to be willing to use it. End of story.
HAL: I'm not really familiar with discover, so I'm not going to shoot my mouth off (much *grin*). From ransacking the web page you linked to, it doesn't look like discover really supports the sort of "central daemon with notification signals" model that we need to provide good hardware support on the desktop side. Without that... its sort of useless to us. It looks a lot like Kudzu, which is a good thing (and trust me, Havoc who proposed HAL in the first place knows about Kudzu, and probably discover) but it simply isn't what those of us who are in the ditches writing this desktop code need.
Moral of the story: a superficially similar "solution" does not necessarily address the issues that we as desktop developers face. We propose these things because we have concrete problems to solve. Sometimes the problems are not obvious until you try to do something and end up butting into them. We're lazy people, just like anyone, and we don't like :-)
Some of you may not have written your own dists, so here is a breif overview of why this is the stupidest thing I have heard in a very long time.
1) Bootloader straps a kernel
2) Kernel mounts a filesystem ro, and loads init from it
3) init forks a single shell
4) that shell launches everything found in the inittab
5) that shell then procedes to run your init scripts
From that shell what you do is up to you.
In single user {runlevel S/s} you stop at #4, and start from that shell.
Any other runlevel will be started from that shell. It doesn't matter if your scripts are written in perl, csh, sh, php, tcl or python.
It doesn't even matter if you write them in a language that you can compile for extra speed.
99% of your time is in loading the interpreter, and waiting on the daemons themselves.
sendmail is extremely slow on loadup, so are other apps. Try setting dhcp, and having it time out.
or RedHat's retarded Kudzu tool.
Calling those from a faster interpreter, doesn't make them load or run any faster. It never will.
If you think runlevels aren't important... Try patching a running system sometime.
-- Sir Ace
This is what is great about Open Source - people can innovate, create new systems, and if they prove to be good, they can be widely deployed. That's how progress happens, and the heart of a developer rejoices on seeing things like this. In the end, superior technology will triumph, and seeing that he is using Python, he is already well on his way :-).
And as for all of you sticking to the old stuff, because it's good enough: are you sure you are not just getting old? The time I start to whine about progress ("the old way was good enough") will be a sad day indeed. Or perhaps this is the difference between sysadmin-types and programmer-type: sysadmins like to stick to old shell script based system because it's uniquity, while developers see the opportunities of new technologies and have a certain inborn respect for technological superiority.
Linux will evolve, live with it. If the old system is indeed better, it will be used indefinitely, but unless we try something different, we will never know. Having some distros doing the thing in an alternative way is a good way to hash this out.
I for one welcome our new freedesktop.org overlords. I'm really liking the direction that Havoc Pennington and other Gnome-related people are taking as far as desktop things go. We need more dynamic & motivated people like them on powerful positions.
Save your wrists today - switch to Dvorak
Ok, so I'm not the best person to say this because anyone who looks for my publishing credits will immediately realize that I'm a "Perl guy" and thus, I must hate Python (I'm not really a Perl guy, and I don't hate Python, but that's beside the point). I'll go on anyway because this is important, and folks who don't do sysadmin for a living may not have had to think about this.
init itself is so lightweight and small that it rarely, if ever, fails. This is a good thing, since it's init's job to start a ton of very heavy-weight services.
That said, I see the logic in bloating init with some of the features that are almost always implemented in a distribution built around it. For example, it would be nice to see init perform some service tracking such that it could be told directly to kill a service, and it could do so.
Keep in mind that every time you increase the size of init, you remove a class of systems that can now no longer use it because of it's footprint. This matters a lot for some kinds of embeded systems that have just enough brains (that is, RAM and installed libraries/software) that it makes sense to have init today.
You certainly could not achieve this minimal-growth by re-coding init in Python, Java, Perl, Lisp or any other high-level language. That's not a slam against high-level languages, it's a simple fact of life that their flexibility comes with costs.
As for the shell-script init-scripts, I certainly feel that all of that should be moved out of init's domain. Each application should have a control program (like apachectl) which knows how to start it, stop it, get status, reload configs, etc. That program can be written in C for speed; a high-level, general purpose language for ease of maintenance; or even in shell. But, the point is that that should not be a constraint of init.
init, might well provide a library and/or command-line tools to make writing those controlers easier and more modular, but I don't think there should be any REQUIREMENT that your program know anything more than the calling conventions of an "init controler". The more constraints you heap on, the less software is going to ship ready to integrate with your init system, and that way lies far too much integration work to create a workable OS (and thus MORE variants between distributions, not less).
Once you've done all of that, THEN you can think about the high-level glue so that things like a desktop integrate better.
Whoa an AC with some insight. Windows 2000 had a single-threaded boot, kind of like Linux. Load things one by one and in order. Win XP came out with a multithreaded boot process, which brings the system up faster. However it only appears to be ready sooner, it takes the same amount of time to get into a usable state.
Can I get an eye poke?
Dog House Forum
As someone else noted (in an article with a link done as text, not as an , and with an extra blank in it), Mac OS X has a possibly-interesting scheme for starting and stopping system services. Here's the section on System Initialization in the Mac OS X online documentation; in particular, look at the Startup Items stuff.
It's a dependency-based scheme, like at least some of the other proposed system services mechanisms; see the section on adding your own startup items.
Note also that Boring Old Init isn't itself changed; /etc/rc runs SystemStarter as its last act, and that starts up the system services. SystemStarter can also be used as a command to start, stop, or restart services, so this isn't just a system startup mechanism.
NetBSD 1.5 and later have a new rc-based system for controlling services; it's also dependency-based. FreeBSD 5.x has rcNG, which is derived from the NetBSD scheme.
(Note that all of those systems have a BSD-style init, and thus don't have run levels.)
Those schemes don't address one of Seth's complaints, however - according to his description in his blog, he doesn't like having shell script wrappers doing the configuration, he wants it done in the service's process itself:
(I express no opinion on this one way or the other; if you like or don't like the idea, send bouquets or brickbats to him, not me.)
Note also that it appears that he is not designing something just for desktops - in particular, he says:
Now, whether the kernel should start ServiceManager directly, or whether it should continue to run init and have ServiceManager started from an rc file, is another issue; I'm not sure that there's a compelling argument to eliminate init other than to discourage people from continuing to use the current rc script scheme (which might be Seth's motivation for running ServiceManager as process 1 - he might want to discourage rc script tweaking).
Perhaps one of the reasons why people thought of ServiceManager as being purely for desktop systems is that they thought that, as D-BUS is somewhat associated with freedesktop.