Slashdot Mirror
Booting Linux Faster
krony writes "IBM's DeveloperWorks explains how to decrease boot times for your Linux box. The concept is to load system services in parallel when possible. Most surprising to me is the use of 'make' to handle dependencies between services." The example system shown is able to cut its boot time in half, but the article stresses the effectiveness can vary widly from machine to machine.
I guess someone has a use for this, or they wouldn't have spent the time working on it. But I don't see it.
I never noticed Linux taking very long to load, and even if it did I doubt I would care very much, as reboots are so rare anyway.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
This sounds like an awesome reason to have multiple cpus. Maybe I missed it in the article (I did read it rather quickly), but it didn't look like it was mentioned.
Even on an HT-enabled P4 this would be cool. Although the I/O would be the limiting factor in the process startup speed, letting multiple proceses start up at once would allow the cpu to switch to others while I/O is being services, much like make -j(# of cpus+1).
There are only 10 kinds of people in this world... those who understand binary and those who don't
It's just human nature. Douglas Adams wrote in "Last Chance to See" that he would glady spend an hour working on a way to save himself ten minutes on the computer.
http://www.rootstrikers.org/
to reboot without rebooting, such that uptime remains the same but kernel upgrades can take place?
I remember reading about it somewhere, but it was skimpy on details, sufficing to say that it was a "bad idea".
You can't judge a book by the way it wears its hair.
This was three years or more ago, but I remember one of the PPC Linux developers "converted" all his system boot scripts in init.d to compiled C.
.no "glory". Fixing this would be like someone fixing fdisk... no one wants to touch the damn stuff...
Boot times went from about 2 minutes, to 35 seconds.
(It took "so long" because it was an old PPC 601 60MHz or something like that).
Distributions such as Mandrake and Gentoo claim they go the extra mile for "performance". I've wondered why neither has cleaned up their boot process.
You wouldn't think Bash is slow from interactive use, but it really it. Piggyback on that speed problem that too many "functions" (OK, *commands*) are standalone executables... greate sub-process, collect result, destroy, rinse repeat.
This is pretty interesting stuff, and I applaud this guys efforts. INIT script achitecture is pretty thankless stuff..
If the second reason is your main reason to boot quickly, you'll probably want to start X too every time you boot. So waiting to start X until a user says X should be started is no option. Your other suggestions are spot on.
If you'd like to take away the last, big, bottleneck, it would be a good idea to start X in parallel with the other, independant, services. This is exactly what's described in the insightful IBM article. Hooray!
DNA is the ultimate spaghetti code.
Bah. Mac OS X's done this since Jaguar.
s /sanchez/sanchez_html/). Then SystemStarter makes a dependency graph and starts them up in parallel whenever possible (http://developer.apple.com/documentation/MacOSX/C onceptual/SystemOverview/BootingLogin/chapter_4_se ction_2.html).
The big question is "how do you specify dependencies?" The article uses makefiles. In Mac OS X, each startup item has a properties file (associative array) that names the item and specifies all the items that it depends on (http://www.usenix.org/events/bsdcon02/full_paper
Not having looked at the code, it seems to me that make would only handle half the problem: booting. Shutdown is the other half; the dependencies would be reversed. For example:
For boot you would tell make this:
sshd: network
rpcd: network
But for shutdown you need to tell it this:
network: sshd rpcd
Ideally one set of input data should take care of both cases.
This isn't make's intended use (it was designed for programming), so it's a bit suprising to see it used this way at first.
That said, it does make perfect sense.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
> Really? That's an odd statement. How surprising that they choose to use an open-source software application that is designed to compactly represent dependencies for representing dependencies.
Actually, I also found it surprising, and I think I know "make" pretty well. The thing about make is that in 95% of cases almost all of the rules correspond to an actual target file that should be generated or not based on presence and timestamp. There are exceptions, like the usual "all" rule that's called a phony rule since it generates no file. (And make sure you have a ".PHONY: all" line right before it or "touch all" will break your build.) It's usually just there for the dependencies on a bunch of real targets, so you don't have to type "make this && make that && make ...".
Parts of make that they're not using here:
- logic for checking if a real target is up-to-date
- rules for creating specific targets from generic ones, like the
.c.o target
- variable substitutions
- a lot of other things...look at the man/info pages; modern versions of make have a lot of functionality that makes no sense here
And they are using:When I say the syntax doesn't make sense here, I mean (in addition to the usual make complaints) that it's all in one file. Distributors (notably RedHat in particular) have been very serious about separating out stuff into .d directories so that packages don't need to touch each others' files.
So, I think make is the wrong tool for the job here, at least in the long term. A simple tool with separate files for each service would be a win. I don't think the author of the article really cares about that (it's just a little tip for intermediate users), but if a distribution wanted to implement this idea and maintain it, they wouldn't use make.
Something for the usability folks to think about:
Ordinary users, and even many geeks, don't have time to figure out what every service does and whether they use it. A policy of aggresively turning off services (mostly for security, partly for boot time) carries a risk of turning of a service that is needed.
I suggest that there should be a standard framework for dealing with "a needed service is not running" problems. On a desktop Linux, this should pop up a window explaining what service wasn't running, and giving options to do nothing, start the service on a one-time basis, or add the service to boot time start-up (and prompting for root password as required.)
(There can be extra options - don't start the service, and never ask me again. Don't start the service, and never ask me again if this particular program complains about it.)
Quattuor res in hoc mundo sanctae sunt: libri, liberi, libertas et liberalitas.
My best times were power on to init in about 2.7 seconds. By the time we got the "authentication code" and what not in it was closer to 30 seconds.
Take all that BIOS stuff out and create a truely lean and mean setup with minimal init scripts and you can blaze. Longest step was copying the kernel from slow-mo flash memory in to RAM...
What's wrong with Suspend/Resume? Powering off your notebook seems like a waste of battery and time if you ask me.
I would even start to apply this to desktop machines - just suspend it, don't turn it off all the way.
Cheers.
Andre
I think this "problem" you mention is some sort of urban legend. I have heard this same argument countless times, but I have never actually seen this happen. I have been a subscriber to a few Linux mailing lists for several years now, and I have never actually seen someone post "RTFM" as an answer to a question.
Myself, I try to sort of evaluate the person who is asking for help. If I think he has an adventurous soul, and is willing to go through a lot of documentation, I try to orient him to the relevant how-to's. In the other hand, if I feel the person is somewhat impatient, I recommend a Mandrake installation, since it's the most likely to get the user safely past the most annoying problems with minimum fuss.
Kernel vulnerabilities are fairly rare. A new kernel is the only thing that mandates a complete system restart when upgraded. Linux and the BSDs got this right; it's the one overall thing I don't like about OS X. At most, OS X should only have to restart the GUI and close apps for most system patches....but nnnnooooo. Sheesh guys, you trumpet "the Power Of Unix"; use some of it.
The traditionaly Sys V init is archaic, crude, and disgusting. What, 6 hardcoded numeric runlevels? Wow, how useful is that. And I love ordering my startup scripts with two digit integers.
*nix needs a major boot/shutdown system upgrade. I have migrated to minit, but that is primarily for low memory usage. It allows a rudimentary mechanism for specifying dependencies, but is geared mostly to be minimalistic. This 2003, I think we can come up with something better than Sys V init.
Features of a next gen boot/shutdown service manager:
* uses real dependency traversal on startup and shutdown (maybe using a small theorem prover like CML2, or maybe something like make)
* allows configuration of arbitrary and unlimited sets of services, which can be named by arbitrary string literals - no longer chained to 7 numeric choices. e.g. "roaming laptop", "docked server", "minimal services", etc.
* built-in service start/stop/restart/status/enable/disable tools, and standard service API with bindings for various languages (what, native services? imagine that...we do so for Windows NT+, e.g. apache) as well as Plain Old Shell Scripts. So every freakin' flavor/distro of *nix doesn't have its own fscking way to start/stop/enable/disable services.
A lot of the garbage that goes on during startup (have you looked at the standard redhat scripts?) mounting drives and file systems, setting network and hardware parameters, etc., could probably use being standardized also, and either pulled into drivers or services or something, in a standardized fashion. Ideally all these APIs could be exposed both through command line tools, but also through desktop-integrated GUI tools, so that modifications don't entail digging up some ad hoc script on disk and modifying it and hoping you remember what the fuck you did a year ago in some system script.
It's 10 PM. Do you know if you're un-American?
Back in 1999 I rewrote all the init scripts entirely from scratch. I did this after having spent a few years before hacking at init scripts in BSD/OS, OpenBSD, Redhat, Slackware, and Solaris. I experienced all the crankiness of these systems (Redhat and Solaris were the worst) and this time decided to avoid all that. I gave the scripts entirely different names so as not to conflict with existing scripts (was Slackware at this time). That way I could switch between them with just a change of /etc/inittab. It took a few hours, but I had a running fully functional system by the end of the day, and have been running on those scripts, as subsequently better debugged and tweaked, ever since. They booted up noticeably faster than even the Slackware scripts (which were about as fast as the OpenBSD scripts).
Irontically, I didn't do this to get the boot speed. The init scripts are fast enough now that the kernel initialization time is longer, anyway. What I did this for was because I hated having a bunch of separate directories with symlinks in them for each run level. I didn't like having to use specialized tools to manipulate the system (I wanted to routinely use the tools I would have available if I were running from a rescue floppy trying to fix it). That meant doing things with a basic set of shell commands. Yet I didn't want to abandon having separate scripts for each service/daemon being started (or stopped as the case may be). What I ended up doing was creating a single subdirectory for all the individual service scripts, and making the script name have a pattern that included both the startup sequence (stop sequence simply ran backwards), as well as the run levels. Here's what the names in /etc/sys on my system look like:
Figuring out which run level each service starts in is left as an exercise for the reader. BTW, I think most of the speed comes from the fact that I didn't add a lot of fat to my script system. That's easier to do when you do your own design.
now we need to go OSS in diesel cars
.. and it isn't the quickest to reboot either! Mind you, I don't find I'm rebooting that frequently compared to my Windows machine.
I did it and noticed a decrease in the boot time, probably 15-20 sec. give or take. YMMV.
-- Two in the pink, one in the sink.
I've seen the exact opposite approach taken - switching from parallel to serial to reduce startup times.
The reason this worked was that starting processes in parallel increased disk contention and the extra seeks brought the machine to a crawl.
Sure, that was on a system with very slow seek times, and not much in the way of disk caching and scheduling (an Amiga 500 with no HD). A lot of things have improved since then, but seeks are still extremely slow in machine terms, and we also have virtual memory and demand page loading which I imagine don't help the problem.
Maybe keeping data needed at startup centralised on the disk (e.g. in the boot partition) would help.
As disk accesses during startup are probably pretty predictable (consistent from one boot to the next), it may be possible to pre-load the disk cache to improve startup times.
A simple approach would be to log disk blocks accessed during startup, and then read them (in a sensible order of course, and in parallel across disks) at the start of the next boot.
I thought there is an outfit called "linuxbios" that supposed to make re-booting, especially cold-booting a very fast process.
Can anyone here tell me the recent progress of "linuxbios" ?
Thank you !
Muchas Gracias, Señor Edward Snowden !
That is an incorrect assumption. It is possible to patch the kernel while it is running. Also, it's very, very freaking easy to load a loadable kernel modules, especially when your kernel is linux. Some people have module support disabled though, and may have access to kernel memory disabled, so then they would be vulnerable.... unless the could use a bug in the kernel to allow write access to kernel memory.
Anyways, my point is: don't assume