Slashdot Mirror
According to Linus, Linux Is "Bloated"
mjasay writes "Linus Torvalds, founder of the Linux kernel, made a somewhat surprising comment at LinuxCon in Portland, Ore., on Monday: 'Linux is bloated.' While the open-source community has long pointed the finger at Microsoft's Windows as bloated, it appears that with success has come added heft, heft that makes Linux 'huge and scary now,' according to Torvalds." TuxRadar provides a small capsule of his remarks as well, as does The Register.
"Okay, so the summary of this is that you expect that 12 per cent to be back to where it should be next year, and you expect someone else to come up with a plan to do it," joked Bottomley. "That's open source."
That is also the problem. Everyone adds pieces and eventually it starts to become a mess. Then someone else should fix it.
I've met the enemy and they is us.
[signature]
is finally having the last laugh? /dnrtfa
Of course nobody refers to Windows' kernel when people call it bloatware. Linus however is not talking about Linux as a distro or an operating system, it's just the kernel that's too bloated in his view. And with over 11 million lines of code, it's hardly even a flame.
Now if only he had developed a microkernel instead...
Pretty good is actually pretty bad.
However, Minix continues to maintain its girlish figure.
Uh, I'd love to say we have a plan. I mean, sometimes it's a bit sad that we are definitely not the streamlined, small, hyper-efficient kernel that I envisioned 15 years ago...The kernel is huge and bloated, and our icache footprint is scary. I mean, there is no question about that. And whenever we add a new feature, it only gets worse.
And also:
He maintains, however, that stability is not a problem. "I think we've been pretty stable," he said. "We are finding the bugs as fast as we're adding them -- even though we're adding more code." Bottomley took this to mean that Torvalds views that the current level of integration acceptable under those terms. But Mr. Linux corrected him. "No. I'm not saying that," Torvalds answered. "Acceptable and avoidable are two different things. It's unacceptable but it's also probably unavoidable."
I think that's very important to note. His quote by itself is very self-loathing but to add that tit's unavoidable really says a lot. You want to be popular? You have to satisfy more people and in doing so you become more bloated. He does maintain that Linux remains stable and that's usually the biggest problem I have with bloat. It decreases stability. I don't think there's any reason to get excited about level headed rational and reflection.
My work here is dung.
What "bloat" in software means to LT as the high priest of the kernel and what bloat means to me as a user are two different things.
To a user, bloat means awkward, slow, inefficient, and needlessly large (if my storage space or bandwidth is limited). But these are all *perceived*. I don't perceive Linux to be bloated.
In fact, I find *NIX with almost any window manager to be the most efficient computer OS I have ever used. Linux is the best of them, despite being a clone of the UNIX userland.
If an OS can boot from a floppy or small USB key and be totally usable, it is certainly not bloatware. Rewrite the Linux userland in MONO or Java and then we'll talk about bloat.
Rich And Stupid is not so bad as Working For Rich And Stupid.
"Now if only he had developed a microkernel instead..."
It would be bloated AND slow.
But hey, it would look pretty in a high level UML diagram.
About two years ago I tested wether my Gentoo kernel was really faster. Disabling 3/4 of the options really just improved boot time and memory footprint, but not overall performance that much, at least far from 12%. Compared to a modularized kernel with just the stuff loaded, that was needed, the difference was negligible. I'm not sure if Torvalds is telling the truth about the reasons. To me it seems that the central, overall kernel architecture has degraded over time with regard to performance.
Torvalds' use of the term "Bloated" in this case refers specifically to a loss of performance and an increase in size and memory usage, not of confusion.
I think there are two (competing) goals for the Linux kernel as a whole (well, there are as many goals as there are developers, of course, so the two competing goals are more of a continuum).
On one side, there is a desire for the Linux kernel to support more features so distros can be built to be more like popular mainstream operating systems like Windows and Mac. Ease-of-use, a pleasant user experience, separation/insulation from the dreaded Command Line, pretty graphics, massive hardware support, and support for more "oddball" configurations like multiple screens, etc. So it's desirable to have lots of driver support and lots of hooks into the operating system to support fancy stuff.
On the other, there is a desire for Linux to be small, sleek, and fast, particularly for embedded projects.
The former has been running the show for a while, and I think that's healthy and positive, but the kernel has gotten larger and slower at its basic job. For desktop users, this is good news since a lot of things that had to be done at "higher" levels can now be accomplished directly in the kernel, so they might actually have a faster user experience, and they've got resources to burn since most PCs are specced out for Windows, so Linux has a lot of spare growing room in that hardware.
But for embedded/minimalist supporters, it means they need to add more hardware to their machines to support the now-larger kernel, chock full of features they'll never need or want.
"This post contains words, known to the State of California to cause thought. Wash brain thoroughly after reading."
(1) Large feature set
(2) Compact/optimized
(3) Fast to market
Pick any two...
I come here for the love
more hardware support and more functional tasks with scope creep means larger code base. nothing to see here, move along.
What do you think is extra? What would you remove? Are you able to remove it?
For Ubuntu, I can easily answer these questions because the system is transparent
and I can act on my preferences without even being a developer because the system
is flexible, modular and open.
I can even get rid of a lot of Linuses kernel code because there's been a nice shiny
happy build GUI included with the kernel since the 1.x days.
A Pirate and a Puritan look the same on a balance sheet.
the difference is
make menuconfig & modprobe -r
bloating in the windows kernel is compulsory!
bloat in the linux kernel is optional and much of it can be removed at runtime, ofc if the whole kernel is getting worse every release then that is bad. So before making comparisons to windows it's important do remember that an extra 10% of something small (once you trim the crap you don't need) is less than an extra 10% of something big (because you can't)
IranAir Flight 655 never forget!
This is like the salesman's nightmare, where you take the guy from engineering to visit the customer. Things are going great, the engineer can answer all the customer's questions.
Then you realize, *the stupid bastard is answering the questions honestly*.
Honesty is a basic requirement to be a halfway decent engineer. Persistent and incurable dissatisfaction with how you did the last job is another. Even if you *know* you did a great job, deep inside part of you knows you could have done it *better*.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
Then let's do like most other open source projects when they reach that point : Analyze current version, find good things and bads things, find possible improvement that were impossible because of breakage and legacy. Once the analysis process is complete, start version 3.0 from scratch, implement the new stuffs and improvements, then bring current features in one by one. And don't tell me it cant be done, it has been. And dont tell me it wouldn't be supported : how much time did it take before the 2.6 line has been adopted by industrials and missing critial distro?
Bloated? Of course. Happens in every walk of life. It starts out lean and mean killing machine out of necessity, otherwise there is no success. Life is tough and to be other than at the top of efficiency is a death sentence.
After achieving success then being fat and lazy is a luxury that is no longer fatal.
This happens everywhere the jungle, in the business world, your job and governments. Evolution.
I still compile the kernel from time to time. Its not that different and the core kernel compiles quickly. But the modules take ages if everything is enabled. Generally you can disable more than 70% on any given system, then compile time is much faster. With the make -j2 thing on a dual core i wait less time with slackware 13.0 than I did with slackware 1.? on a 486. (can't remember the kernel numbers)
If information wants to be free, why does my internet connection cost so much?
Does anyone remember The Tanenbaum-Torvalds Debate??? I think that Linux is facing the problems that professor Tanenbaum stated more than a decade ago and Linus Torvalds did not take into account. Is something like minix3 (www.minix3.org) the future of operantig systems??
I think people are somewhat missing the big picture here. This is evolution in action. As time goes on and people need new things they get added. It's hard to say when to get rid of old vestigial features, so it doesn't get done. This leads to bloat. Eventually this will be corrected as the obsolete stuff gets more obviously unused. It's a problem, but unlike the dinosaurs, Linux will adapt due to the extreme ease of digital replication. If it ever gets so bad as to be unsustainable, someone will either use an old version, recompile or fork and get rid of the parts they will never use. You can't really do that with anything that you don't have source to. Of course the typical end user like me will have to wait for somebody to do this, but luckily my computer is fast enough that I haven't really noticed yet.
And is also utterly impossible while there is a single line of GPLv2-only code in it that the author doesn't give permission for, or whom is dead. There's quite a lot of code like that, there's a lot that can't be traced to an author, there's a lot of authors that won't give their permission, there's a lot that *can't* give their permission (employers, etc.) and there's so much of it that recreating it from scratch without reference to the original code would actually take longer than just starting a GPLv3 kernel from scratch.
And this has been discussed to death before. Ain't gonna happen - not out of some inate personal reasoning, but sheer impossibility.
Clearly whoever modded you up has never tried what you are suggesting. I can only name a handfull of open source projects that backport security fixes to old versions and of those, they only backport to versions a few years old.
In fact, I'd say the longest lived "old version" is probably Apache 1.3. The 2.x series has been out for, what, forever and yet they continue to push out fixes for 1.3 (last was Jan. 2008).
I'd wager the biggest complaint I have with most open source is the a) dont understand what true stability means and as a result they b) rarely support old versions. It was one of the prime reasons I switched to FreeBSD. If I install FreeBSD 6.2 today, I know I'll get security fixes for at least a good half decade and probably a bit more if I track the 6.x series.
Yeah yeah yeah, debian, yeah yeah... but dont get me started on the other reasons I switched (cough crappy docs, cough, crappy unstable kernel, cough
Often the term bloated is misused meaning the speaker is at a point where he/she personally starts to find a technology confusing to wade through.
Linux today does not boot significantly faster than it did 15 years ago. That's bloat.
Bloat isn't bad.
Version 5.0 of Microsoft's flagship spreadsheet program Excel came out in 1993. It was positively huge: it required a whole 15 megabytes of hard drive space. In those days we could still remember our first 20MB PC hard drives (around 1985) and so 15MB sure seemed like a lot... In 1993, given the cost of hard drives in those days, Microsoft Excel 5.0 took up about $36 worth of hard drive space. In 2000, given the cost of hard drives in 2000, Microsoft Excel 2000 takes up about $1.03 in hard drive space...
In fact there are lots of great reasons for bloatware. For one, if programmers don't have to worry about how large their code is, they can ship it sooner. And that means you get more features, and features make your life better (when you use them) and don't usually hurt (when you don't). If your software vendor stops, before shipping, and spends two months squeezing the code down to make it 50% smaller, the net benefit to you is going to be imperceptible. Maybe, just maybe, if you tend to keep your hard drive full, that's one more Duran Duran MP3 you can download. But the loss to you of waiting an extra two months for the new version is perceptible, and the loss to the software company that has to give up two months of sales is even worse.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
I made this same comment on some /. story and instantly got modded -1 Flamebait.
Go figure.
I'll try anything once. Twice if it tastes good
Yea, I was rooting around in your system just the other say. You seem to be completely frugal when it comes to purchasing new software. On a plus side, your system is completely useless to me outside of just exploring around it a little. Good call and staying arcane.
Erm actually its quite the opposite, windows XP got security patches for years, i doubt you'll find a safe 2.6.8 (~2004) kernel about. Even "slow" distros like debian only backport security fixes for 3 years after that you have to upgrade, or start maintaining your own kernel.
IranAir Flight 655 never forget!
Comment removed based on user account deletion
Heh, read the stable_API_nonsense.txt file in the kernel source. Here's an html version:
http://www.kroah.com/log/linux/stable_api_nonsense.html
Next year he's going to claim that Minix was doing it right all along. We've seen a lot of Linusisms to that effect... $X needs to be outside the kernel... $Y shouldn't happen the way I've been screaming for years... I told $Z to fuck off because he's stupid but he was right and we need to go do that yesterday ... it's just how Linus is. He's an opinionated fat bastard, and then one day he realizes he's fucking wrong and just goes, "SHIT! Well let's do that then >:O"
Support my political activism on Patreon.
The goal of this competition would be to obtain the optimal factoring of the kernel architecture.
Seastead this.
What? No, that's not the kernel. That is:
> The BIOS - take a look at the LinuxBIOS or OpenBIOS work to see where that can be improved. But oh, my dear goodness, it can be improved.
> Incredible masses of new hardware that do need detection and configuration at boot time. That's been a sore point: it takes time to scan for all that hardware, and you can optimize it by leaving out tools, but people do like having their network cards and USB drives and graphics tablets work automatically at boot time. That's not the fault of the kernel: that's the fault of the time taken to detect and configure low-level hardware components, such as RAID controllers, which may be necessary to begin to load _any_ kernel with the actual drivers to run the operating system.
> Masses of init scripts starting up many, many, many services in a very lengthy sort of way. Those can be optimized far more than they are, and parallelized: but it takes a rewrite of the 'init' procedures to do so, and that's not the kernel's fault.
I'd like to see all of these improved: all of them are, in fact, plagued by bloat. But they're not hte kernel.
I think this applies to some distros. For example, Ubuntu comes with Pulseaudio. It is of little or no use to the average person. The average Joe isn't going to stream audio over his LAN or combine two stereo sound cards to make one surround sound card. Pulseaudio creates sound lag, which is why I always remove it when I do an install.
I like a system with the minimal amount of daemons running that are required for operation.
The biggest problem with that is USB devices. Who knows what weirdo USB hardware you're going to want to plug into your computer in the next couple years. Using the stock Debian kernel, that's something I really don't need to worry about.
Give me Classic Slashdot or give me death!
He should start a separate distro and call it leanux....not like we can't make do without another distro out there.
I did RTFA and I must say the article was poorly written - so much so that the author felt he needed to publish a correction that summarily states (what open source power users already know) that the Linux kernel can be "trimmed or fattened up." It is immaterial that Linux has gotten more bloated as the fundamental difference between it and Windows is that you as the consumer have the choice to "trim the fat." While I am an open source users, I am pragmatic and I believe it cannot be all things to all people and Windows has some advantages over Linux. For example, the choices of Linux can be downright bewildering and each distribution behaves differently with its own quirks. Windows is Windows. Even though distributions share a common kernel, they are really distinct OSes in their own right - applications run differently and have different behaviors. As Samba will tell you, sometimes compiling succeeds on three out four large distros. In theory, they should be all compatible.
init scripts especially are rather idiotic, and it's a testament to how much crap Windows is doing that Linux distros manage to load in roughly the same time.
It's especially dumb when things that could start after the system has finished booting, like samba and ssh, instead start first.
Likewise, driver detection. Um, no, you don't do that on startup, unless it's a first-time boot. You do that when the system is running, which means the very first time someone boots with that fancy new sound card the startup sound isn't going to come out it...but the other sounds should. That tiny tradeoff saved 15 seconds every boot.
And even just crap like cleaning out /tmp and remounting network drives and CD-ROMs and etc. That's background stuff.
That, right there, is the problem. For some totally unknown reason, Linux distributions have no 'deferred startup' script area. You either get run on startup, and everyone waits for you, or get stuck in cron when you only need to run once.
Or, hell, some sort of dependency based system, where you list what services you want up as fast a possible (On most desktops this would be X Windows and Gnome/KDE, but for servers it might be mysql and Apache, or postfix and courier, or whatever.) and each service has a list of things it needs. And then you should also list services that should come up when the system has finished that, in a non-time critical manner.
Redhat tries at this, but fails. Debian doesn't even try.
I know, I've working on a Ubuntu XBMC box. I would really like XBMC to startup and then have ssh and samba, and, hell, the virtual consoles start later. I've about given up on this concept, though.
It doesn't help that no distinctions seem to be made between 'these scripts must execute, at startup, in this order, to create a functioning and mounted system', and scripts that run later that are just services. Yes, there's rcS.d vs. rc2.d (or whatever), but for some reason, non-required services sometimes get put in rcS.d, probably because no one's ever bothered to set hard and fast rules what they mean by 'system'. (Hint...a sound daemon is not required to have a working system. Hardware detection is not required to have a working system. Mounting the fucking /dev partition is required.)
Of course, Linus can't do anything about all this except frown at the distro people.
If corporations are people, aren't stockholders guilty of slavery?
Let's take a look at the patch history of QNX. QNX is a message passing microkernel mostly used for embedded systems. But it can be run with a full GUI, runs on multiprocessors, and can be run as a server. Millions of "headless" embedded systems have QNX inside. I used it in a DARPA Grand Challenge vehicle. BigDog, the legged robot, runs QNX.
Drivers are outside the kernel. All drivers. File systems are outside the kernel. Networking is outside the kernel. And they're all application programs, not some special kind of loadable kernel module.
There have been 14 patches to QNX in the last two years. Only one is an actual kernel patch: "This patch contains updates to the PPCBE version of the SMP kernel. You need this patch only for Freescale MPC8641D boards." Only one is security-related: "This patch updates npm-tcpip-v6.so to fix a Denial of Service vulnerability where receipt of a specially crafted network packet forces the io-net network manager to fault (terminate)." Neither Linux nor Windows comes close to that record.
There's little "churn" in a good microkernel. Since little code is going in, new bugs aren't going in. Good microkernels tend to slowly converge toward a zero-bugs state.
QNX generally has a "there's only one way to do it" approach, like Python. Linux supports three completely different driver placement - compiled into the kernel, loadable as a kernel module at boot time, and run as a user process. QNX only supports one - run as a user process "resource manager". That simplifies things. A "one way to do it" approach means that the one best way is thoroughly exercised and tested. There are few seldom-used dark corners in critical code.
When QNX boots, it brings in an image with the kernel, a built-in process called "proc", any programs built into the boot image, and any shared objects ".so" wanted at boot. These last two run entirely in user space; they're just put in the boot image so they're there at startup. That's how drivers needed at startup get loaded. They don't have to be in the kernel. (In fact, you can put the whole boot image in ROM, and many embedded systems do this.)
A QNX "resource manager" is a program which has registered to receive messages for a certain portion of pathname space. The QNX kernel has no file systems; part of the initial "proc" process is a little program which keeps an in-memory table of "resource managers" and what part of pathname space they manage. This is similar to "mounting" a driver under Linux, but it doesn't require a file system up during boot. File systems are user programs which start up and ask for some pathname space, after which "open" messages are directed to that file system.
Another QNX simplification is that the kernel doesn't load programs. "exec" is implemented by a shared library. That library is loaded with the boot image, to allow things to start up. "exec" runs entirely in user space, with no special privileges, so if there's a bug in "exec" vulnerable to a mis-constructed executable, that program load fails and everything else goes on normally.
The price paid for this is some extra copying, since all I/O is done by message passing. This isn't much of a cost any more, because you're almost always copying from cache to cache. That's an important point. Message passing kernels used to be seen as expensive due to copying cost. But today, copying recently used material is cheap. On the other hand, some early microkernels (Mach comes to mind) worked very hard to mess with the MMU to avoid big copies, moving blocks from one address space to another by changing the MMU. This seems to be a lose on modern CPUs; the cache flushing required when you mess with the address space on recently used data hurts performance.
I used to pump uncompressed video through QNX message passing using 2% of a Pentium III class CPU. Message passing, done right, is not a major performance problem.
I'm afraid that hardware detection may well be required, because critical services (such as NFS exports or MySQL) which rely on mounted partitions in most large-scale environments must have those directories already mounted before running 'exportfs' or before starting the relevant services, or they can create incredible chaos. And the flushing of /tmp/ is tricky: it's much safer to do at a well-defined init step, before the other services are running, and not potentially scrub weird components out from under people. It's not that people shouldn't write these components more sensibly: it's that they don't bother because init scripts are often an afterthought. I suspect you've not personally run into some of the potential adventures of /var/lib/mysql not being mounted when you start the MySQL daemon. It's an adventure.
I completely agree with you about material being in the wrong init levels frequently, and the need for dependency management. (RedHat has some old mistakes in handling wifi devices _after_ network initialization, which causes real chaos.) And forcing network file systems such as NFS or CIFS to wait until the network is running would make complete sense.
On one side, there is a desire for the Linux kernel to support more features so distros can be built to be more like popular mainstream operating systems like Windows and Mac. Ease-of-use, a pleasant user experience, separation/insulation from the dreaded Command Line, pretty graphics, massive hardware support, and support for more "oddball" configurations like multiple screens, etc
I risk sounding like Stallman here, but in this case the distinction actually matters. We're discussing the kernel, not the OS. The OS is GNU, the kernel is Linux. There are various desktops for GNU, most people use KDE or Gnome (I like KDE). It's easy to use, like Windows there is a command line but you won't need it (at least with Mandriva) but you can write shell scripts just like with Windows you can write batch files (and GNU is far and away superior here), it has pretty graphics, far more massive hardware support (Linux will run on anything from a wristwatch to a supercomputer while Windows only runs on PCs and servers), and it is easily configurable for oddball stuff.
But the stuff I mentioned in the previous paragraph is, like your post, completely offtopic as none of them has anything to do with the kernel. You're talking about the shell/desktop when the discussion as about the kernel.
Free Martian Whores!
Things like device drivers can be easily diked out. When it comes to stuff like memory managers, VFS, CPU schedulers, basic networking, so on and so forth, I imagine that the bloat hurts the embedded guys more.
Jesus is coming -- look busy!