Slashdot Mirror
How Kernel Hackers Boosted the Speed of Desktop Linux
chromatic writes "Kernel hackers Arjan van de Ven and Auke Kok showed off Linux booting in five seconds at last month's Linux Plumbers Conference. Arjan and other hackers have already improved the Linux user experience by reducing power consumption and latency. O'Reilly News interviewed him about his work on improving the Linux experience with PowerTOP, LatencyTOP, and Five-Second Boot."
Not only is this an impressive accomplishment, but if this can be applied generically to most distributions then it should present an excellent opportunity for advertisement. Showing how you can boot, check your email, read the latest news, and be done with all you need to have done while a fellow Vista machine is still booting says a lot. Even if we can get most distributions down to 15sec average, it's a huge leap. Grats to these guys.
I see a lot of comments on the LWN article of people talking about starting services after the user sees the desktop as cheating. However, I ask, does this really all matter. I'm not sure how everyone else uses their computer but I only need to boot my Linux machine about once every 30-60 days. I don't need to dual boot like I did back in say 2002 and comparitively, the amount of time it takes for Linux and X to start up are practically irrelivent. I can imagine laptop users may feel much differently about this, but I thought that was the point of being able to suspend/hibernate.
One thing that worries me is that a focus on ensuring a quick boot at the expense of a potentially less stable system is not a good thing. Fortunately however quick booting is not something that Linux requires, its something that distributions can decide to do or not, which is one of the strengths of the open source/Linux way.
Why?
ROM was a wonderful thing. Simply flip the switch and the software is already loaded into memory. There was about a second or two of initialization (on a ~1MHz 8-bit processor!) and you were ready to go. It's still possible to create such fast boot times using ROM. Especially with re-flashable ROM. These sorts of boot times are seen in systems like Game Consoles.
Unfortunately, desktop OSes are so complex that using re-flashable ROM adds a great deal of complexity and cost to the design. Thus you aren't likely to see any systems keep their OS in Flash. Compounding the problem is that modern OSes are rarely designed to boot from a ROM configuration and would require substantial changes to boot properly.
Javascript + Nintendo DSi = DSiCade
And no cheating. "Done booting means CPU and disk idle," Arjan said. No fair putting up the desktop while still starting services behind the scenes. (An audience member pointed out that Microsoft does this.) The "done booting" time did not include bringing up the network, but did include starting NetworkManager.
It seems to me that the five seconds could concievably be brought down to virtually zero with cheating! My work PC slows down so much sometimes from antivirus, inventory controls, etc that it takes longer than that to add a record or open a table in an Access database. With a keyboard buffer you could stick a fake desktop and login in, and have the real desktop and login take over before the user finished typing in his password.
Free Martian Whores!
My stepfather still has an old Pentium III laptop with Windows 95 running on it. Booting the laptop to read an E-mail takes around 20 minutes. His advice to anyone who wants to use it, "switch on the PC, do something else like have a bath, do the lawn, read the newspaper, or have a coffee, and the PC will be ready to use before you know it".
Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
O'Reilly News recently interviewed Arjan van de Ven about his efforts to improve Linux performance and reduce power consumption. Arjan works for Intel in the Open Source Technology Center. This interview is approximately 30 minutes.
One of the projects you're probably most known for in the past couple of years is the PowerTOP utility, which I found very fascinating. Looking at some of the gains you've made over the past 18 months, it seems like Linux-based devices are saving a lot more power than they used to. What do you consider the big successes in the past year and a half?
To be honest we fixed effectively the entire Linux desktop space. It's not--PowerTOP is more--it's not just what we fixed with PowerTOP is not individual pieces. We fixed everything. For me that was a success.
Is that everything in terms of not just desktop but servers as well?
Yeah; we fixed not just Evolution. We fixed Firefox; the thing with Firefox was that it wasn't one thing that was broken. Everything had problems and we had to fix all of it. So for me the success was how quickly everything got fixed; it was just amazing.
In this context you consider fixed--everything is no longer broken in the same way or--?
Everything is no longer keeping the CPU out of idle basically.
Do you have a reference machine? I guess I'm asking what's your benchmark for this, a particular software configuration stack or particular type of machine, or are you willing to say it's pretty much every Linux based machine out there?
I'm looking at several machines--my own laptop but to be honest, what runs on my own laptop is what I care about most. At least that's where I got more battery life, this is where I see the changes. I tend to run a quite rich environment on my laptop but I also look at service. We look at all kinds of machines and we see the same trend everywhere in that all the various pieces of it--never polling or keeping the CPU up. They all got fixed.
In fixing this, is there a component of education, for example, saying "Instead of doing a busy wait on a select loop or continually polling you should set a kernel timer and wait for that to call you"?
That's part of it but the biggest thing is that you had no visibility. Just two days ago at IDF I spoke with a developer of the GNOME desktop and he said, yeah; when I saw it happen I fixed it in 10 minutes, but you don't know it's there until you see it from PowerTOP. Adding the visibility turns out to be enough for people to start fixing it. They know how to fix--how to not poll most of the time.
You can't fix something you can't measure.
If you don't see that it happens you don't know it happens and you can't fix it.
Are you getting the same sort of results from other projects you run into?
GNOME was there but it's almost everybody goes oh yeah; we should have not done that; either they fix it themselves or some--a lot of people give them the fix and in general it's like oh yeah; we shouldn't have done that. Unless you see what's happening you don't know what to fix, so the biggest thing that PowerTOP did was add visibility. We can see under the hood what's going on and then we can fix it. And quite often the fix is very simple.
It sounds then, maybe I should be able to say that just about everybody is happy to see this. Is that the case?
Yes; people--all the developers I've worked with--and that's quite a few--they all go oh yeah. Thank you for the fix; we should have no problems in the first place. We didn't know this; it's fixed now. In the beginning I did most of the fixing when PowerTOP was very new and now days the people do it themselves. The developers learn
This is effectively related to an earlier Slashdot story about the changes Mandriva are making to speed up boot on their distro
In an attempt to head off the inevitable here's a link straight to the existing
Interesting but how useful, really? thread (Yes! No! I have a Mac! I use suspend! I use hibernate! Suspend is broken for me! Hibernate is broken for me! Hibernate takes too long with 500Mbytes! Why do Linux people always say change your habits? Etc.)
What I really want to know is what can be done about usb-storage and pciehp (PCI Express hotplug). I have an EeePC 900 using a kernel with Arjan's fastboot patches and with USB entirely disabled and pciehp turned off the kernel mounts the root filesystem in just over one second. With USB on and pciehp in use it's over 5 seconds....
Finally here's a link to Arjan's slides from the presentation about 5 second boot in PowerPoint format and a YouTube video of the 5 second boot on an EeePC 901.
I'd love for my MythTV box to boot faster. Since it's not silent (though the TV fans are louder, the TV isn't always on either), I leave it turned off, and the long boot time makes it less appliance-like.
desktop OSes are so complex that using re-flashable ROM adds a great deal of complexity and cost to the design.
Flash is almost dirt cheap. $10 buys you more Flash memory than most systems have RAM. Just save the state of a freshly booted OS in Flash and when the computer starts, load just what you need to access flash and handle page faults, then start as if you've already loaded everything into RAM and start copying from Flash to RAM. Whenever a page fault occurs, load that page from Flash next. This way you don't need to wait until everything is copied to RAM.
TinyME (my favorite) which is based on PCLinuxOS which is based on Mandriva
Vectorlinux based on Slackware
Mepis based on Debian
Or just sudo apt-get xubuntu-desktop
Note that nowhere in the article is there any mention of the processor, its speed or the number of cores. There's also not one word about how much RAM the machine has. With enough RAM, you can load your entire system into a RAMdisk and even if you don't have SSM access time becomes (effectively) zero. Also, of course, a 2Ghz quad core machine is going to boot faster than a 1 Ghz single core. I'm not saying they're cheating or anything, but these specs are something you need in order to evaluate what they've done, and they're not telling us.
Good, inexpensive web hosting
it's just a normal linux distro. for the demo we used an auto-login but you can have it start to a password screen as well, so it's just as secure as your base distro is....
I've administrated network authenticated openSUSE machines and they definitely benefited from booting faster (compared to older versions of openSUSE) - after all the sooner the kernel finishes the sooner you can start waiting for that DHCP lease...The key is that the moment someone says they want to run NIS/LDAP/NFS you just say "start everything that doesn't depend on the network while you wait for the network to come up". In your case NIS/NFS/autofs/xdm DO need the network so they have to wait until that DHCP lease is acquired. No functionality need be lost but the dependencies/order of certain events need to be maintained (this is what tools like Upstart are about).
Strangely enough in one of the article's comments you'll find that Arjan isn't advocating a parallel boot:
Ultimately I doubt people are advocating all of this work for your typical network workstation. For starters such systems don't tend to be using solid state disks with unattended login...
There is no such thing as end-user-OS-boot-time. It depends a lot on device drivers and system background utilities. For example, some piece of hardware AND some release version of its driver maybe causing your trouble... especially if that hardware is removed and the driver probes a lot of time just to be sure. Same for the AV software doing weird things in order to "secure" the system *before* user interaction... At least in the hardware side, this apply for Linux too.
CP/M was probably OK but my Zilog-based PC had floppies only so it sucked too.
MS-DOS 3.0 was up and running in 1-2 seconds (assuming you had a hard drive and empty config.sys and autexec.bat).
Then MS rewrote DOS in that punky and slow new language "C" and since then everything went down. The next thing you see is that HIGHMEM.SYS driver taking your precious memory out of 640KB for the promise of semi-useless XMS memory for overlays. Oh well...
Now my kernel sits during boot on 4GB RAM looking for un-present USB devices and waiting for eth0 to figure out DHCP.
I guess Bill Gates was right and 640KB is right amount for everybody so OS would not get confused with all those amounts of bits laying around.
Kernel modesetting is kinda here for Intel and ATI graphics cards (in Fedora) but it's still stabilizing.
So the answer to your first question is: soon you will be able to set the final video mode in the kernel. As for your second question, doing it in X is not the best solution (as doing it in the kernel means less flickering when X starts, the ability to support graphical kernel panics and nicer virtual terminal switching).
Linux runs many embedded devices; some of them may wish to boot quickly.
how to invest, a novice's guide
It's still possible to create such fast boot times using ROM. Especially with re-flashable ROM. These sorts of boot times are seen in systems like Game Consoles.
Uhh, if I turn on my Mac Pro and my Xbox 360 at the same time, I'm up and running with OS X a few seconds before the 360 starts to load the game. I for one hope my computer DOESN'T have boot times like a console.
I suspend or hibernate most of the time. I reboot my laptop only when the kernel gets a security fix. I don't care if that takes 40 seconds.
The real lesson from all this load time buisness is that our compilers still really really suck. I mean the truth is that when you boot your computer there is only a tiny bit of logic that really needs to go on because only a small amount of stuff changes between any two boots (and less between a boot and a power off).
A truly well desgined system wouldn't care about arbitrary boundaries between this program and that one, it would hunt down optimization opportunities everywhere and automatically reduce boot up to an extremely lean and quick procedure without adopting the harms of merely loading an old image.
I mean to take one example a substantial amount of time during start up is probably spent searching for and then parsing configuration files. So long as their is no cross cutting OS level JIT compiler that can deal with both system IO code and application code there isn't much we can do about this without massive investment of effort. However, in principle there is no reason that the system couldn't simply read the preparsed data from a cache and jump directly to the real substantive logic that needs to be done on boot (checking out network conditions, looking for changed hardware, dealing with changed configs)
If you liked this thought maybe you would find my blog nice too: