Why Was Linux the Kernel That Succeeded?

jones_supa writes: "One of the most puzzling questions about the history of free and open source software is this: Why did Linux succeed so spectacularly, whereas similar attempts to build a free or open source, Unix-like operating system kernel met with considerably less success?" Christopher Tozzi has rounded up some theories, focusing specifically on kernels, not complete operating systems. These theories take a detailed look at the decentralized development structure, pragmatic approach to things, and the rich developer community, all of which worked in favor of Linux.

Comments

Cuz Minix Dude Was A Old Guy

[Anonymous Coward]

And Linus was not.

Besides, the "copy" is the one that gets passed around (Linux was a copy of Minux).

Re:Cuz Minix Dude Was A Old Guy

[mark-t]

1. FreeBSD required a hardware FPU, at a time when many computers didn't have them.

For me, with a '386 at the time that I first heard of Linux, and with no fpu coprocessor, that was a key factor... although not the defining one, because I was soon going to be getting a '387 anyways. For myself, the deciding factor at the time was that FreeBSD did not support any sort of multiple OS system, where with Linux, I could boot from floppy which would then transfer control over to the hard drive after the kernel was loaded (or after lilo came out, even load the kernel directly from the hard drive), and leave my DOS partitions and the hard drive boot sector completely unaltered.

Re:Cuz Minix Dude Was A Old Guy

[Anonymous Coward]

I would add two more to that list.

Distro size. Distros at the time fit on 3-5 high density floppies less if you stripped out some niceties. BSD took 20 and splitting it apart was not obvious.

Download that over a 2400 baud modem. I did. Linux was done in a few hours. BSD took longer.

Secondly XFree86 catapulted it to the top of the pile. A mostly working x11 stack? Yes please.

Re:Cuz Minix Dude Was A Old Guy

[fisted]

The only difference that results in (resulted in, it's getting better lately) is that BSD-licensed code gets used, while GPL'ed code doesn't get used, for commercial purposes. Furthermore, getting upstream to add your changes is cheaper since you no longer need to maintain them yourself, so companies still have good reasons to contribute back.
The OpenBSD people's take on the matter is:

People sometimes ask if it bothers us that our free work is put into commercial products. The answer is, we would prefer that our good code be widely used rather than have commercial software vendors reimplement and create badly coded or incompatible alternative solutions to already solved problems. For example, it is likely that SSH is a widely used protocol due to this freedom, much more widely used than if restrictions had been placed on how people used the OpenSSH code. If a free SSH solution was not available for vendors to use (in their multitude of rapidly developed products), they would have written or purchased some crummy off-the shelf version instead.

Now, about

If you publish the same code under GPL, and even a single other developer shows some interest and adds something to your work, and in turn decides to distribute that to others, you may get rewarded by additional functionality.

FTFY. It's kind of important to note that as long as you don't plan on distributing your modified version (and in many cases, why would you?) , you're not required to, well, distribute your modified sources.
Then, many modifications will just be horrible hacks to scratch some immediate itch, which you wouldn't at all want in your source base, so GPLing stuff is far from providing a guaranteed enhancement return, even if there is interest.

Re:Cuz Minix Dude Was A Old Guy

[Snotnose]

Back in '94/'95 when Linux was new I asked Alan Cox a couple questions via email, which he replied. Made some changes to the driver to the ethernet chip we were using, he took those and thanked me. No experience with BSD so I can't contrast/compare.

Re:Cuz Minix Dude Was A Old Guy

[Anonymous Coward]

I'm a bit surprised that anybody is puzzled about this because I thought it was obvious too. I guess it just goes to show that I'm getting old. For those who weren't around at the time, my recollection is pretty much exactly as you have it, but I suppose I can elaborate a little bit.

1. The FPU issue was definitely important, but to be honest by the time the 486 became popular I don't think the Linux camp was tremendously bigger than the FreeBSD camp. I know that when I got a 486 I was torn between the two and actually was leaning towards FreeBSD because I used BSD at work. But I chose Linux because...

2. The lawsuit is probably the most important issue. It was just enough of a swing to give momentum to Linux. I remember when the MCC distribution came out, it was more convenient to go with Linux than FreeBSD and I pretty much stayed there after that. My impression was that Linux was many people's second choice and that a lot of people were waiting to see how the lawsuit turned out. By the time it was over, Linux had such a good infrastructure that it was hard to think about switching. I think the first time I started thinking about switching again when when FreeBSD ports came out (which Wikipedia says happened in 94).

3. The user communities issue is probably the most overlooked, but I think it is important. Personally, I don't think that there was *that* much to choose from between the Linux and FreeBSD people -- at that time if you were used to using some kind of Posix system you were already fairly hard core. You had to build the kernel yourself to support whatever hardware you had. I did that for years.

Where you see the differences in approach is between Linux/BSD and HURD. I wanted to work on the HURD because I had done a project on Mach in university. My memory is hazy about the details, but when I approached the maintainer I was told in no uncertain terms that help from someone as unaccomplished as me was not welcomed. I really believe that Linux was so successful mainly because Linus just accepted patches (and even thanked people!). It was very, very, very easy to work on. That low barrier to entry attracted some really good people and things took off.

It is something I that I have taken with me throughout my whole career, even when working on non-free software. It's easy to have an elitist attitude and to think that you must protect the software from the unwashed masses. You feel that you must restrict development to a few blessed people who you know won't screw everything up. Having a more optimistic strategy where you assume people are good actors until they prove themselves to be the opposite is much more successful -- especially for free software.

Having said that, there are of course times when certain people exhaust even Linus's patience and I am impressed with how he chooses to draw the line. It is definitely something I wish I could get away with in my day job. Alas, those kinds of decisions are often not possible for me.

Re:Cuz Minix Dude Was A Old Guy

[Anonymous Coward]

I was a CS student at UC Berkeley when Linux was first announced on Usenet... I also think that the ease of sharing a hard-disk with existing Windows installs, the initial focus on x86, and the familiarity of the basic DOS partitioning scheme were hugely important. The relative homogeneity of the PC platform also allowed a friendly community to bootstrap very quickly. By comparison, the "home team" BSD variants were already fragmented across many different CPU and system architectures, with much more specific hardware requirements or much more do-it-yourself work to bootstrap a new system.

Even at Berkeley, it was easier to self-help via the Internet to install Linux, than to get local BSD zealots to help you bootstrap a personal BSD machine. So, a large number of my Berkeley classmates adopted Linux to actually do our class work on personal machines, while only the most masochistic of my peers were still using or trying to use *BSD. We used a wide variety of Unix flavors on the servers and workstations in school labs, but when we went home we often found that our little Linux boxes were just as fast or faster for doing our actual school work. It being Berkeley, the BSD users certainly supported each other and liked to deride Linux, but they were better at turning away new users than they were at BSD advocacy.

The Linux community was already growing rapidly by the time we started seeing Linux getting ported to other CPU and system architectures. I had both a 486 PC and a DEC Alpha running Linux on my own LAN and connected to the Internet via SLIP before I graduated. Within a couple more years, people like me were now out advocating its use in companies, universities, and national labs. The truth behind the "Beowulf" cluster meme was also significant, as some small fraction of government funding was going into improving Linux by leaps and bounds instead of all going into the extravagant margins of SGI, Cray, Sun, etc. We quickly improved networking, commodity SMP support, and GPU acceleration that made Linux relevant in many high-performance tasks.

Ironically, think about how the Android platform is now fragmented for many different phone chips and models with very different sub-communities, tools, and download sites. This is how the BSD variants felt back in those early days, while Linux was much more standardized and approachable! This is also the reason that I hope mobile platforms do not kill off the PC, as this seems a huge step backwards for future students.

Meh

[Anrego]

Amazed that neither the GPL nor the legal uncertainties surrounding BSD at the time (hey, remember those days!) were really focused on, but meh. I think like everything else that became wildly popular in spite of plenty of seemingly equivalent or better alternatives, it just came down to dumb luck and momentum.

Somehow Linux got the ball rolling, people gathered around it, it gained steam, and here we are.

Re:Meh

[TheGratefulNet]

about 10 or 15 yrs ago, at least in the bay area, people WERE afraid of the linux gpl. I worked at many places that avoided using linux code in their products and we used any form of bsd we could.

now, I never see bsd mentioned anymore in job ads. its ALL about linux.

wonder what finally made the c-levels unafraid of linux? the gpl is still the same and we did have gplv2 and v3 back in the old days when bsd was 'the thing to use' for networking boxes.

Re:Meh

[sconeu]

The GPL and BSD legal uncertainties may have been a part of it, but that still leaves us with the question, Why Linux and not HURD?

HURD was also GPL and free of the BSD uncertainties.

Re:Meh

[danbob999]

about 10 or 15 yrs ago, at least in the bay area, people WERE afraid of the linux gpl. I worked at many places that avoided using linux code in their products and we used any form of bsd we could.

The success of Linux isn't because of it's users 10 or 15 years ago. It's because of its contributors 10 or 15 years ago (and today). And GPL attracts contributors while BSD push them into forking and keeping their local changes for themselves.

Re:Meh

[jbolden]

Hurd was never a viable kernel. It never did much. Moreover micro-kernel design didn't work very well. QNX is really the operating system that got micro-kernels working well and they took it in non-desktop directions. On the XNU (OSX kernel) the kernel has moved more and more monolithic for performance, speed matters. Citrix is doing some interesting work on microkernels so we'll see.

Ultimately though, it is just too expensive to add even a few extra simple operations to most kernel functions. And that's why Hurd lost.

Re:Meh

[lactose99]

Plenty of users of the BSDs contribute their changes back. Yahoo and Apple are both rather giving in this respect. Part of the reasoning is that once you make your change proprietary you then have to effectively fork the code and continue all maintenance yourself. For infrastructure changes its simply easier to contribute the change back. If someone keeps one bit proprietary and submits ten other bits back as a result of keeping that bit proprietary I still consider that a win.

Stone soup

[cant_get_a_good_nick]

When I was a kid, i read a book called Stone Soup. It was about these guys that wanted to eat, had nothing but a pot. They put in some stones... called it stone soup.

Eventually people got curious, and added things.. the soup became real. Going from water and rocks, to where real ingredients went in, and the stones just fell away. A seed, but then dropped when something real came.

I always thought of Linus as a guy who managed the Stone Soup well. It wasn't specially good in .01 version. But he made people want to add to it. The GPL helped some. Linus chose that license, not as a "hey Im a zealot and you need to give me everything you write" but he thought "if people do cool things they need to let me see their cool things"

That, and FreeBSD had a few handicaps. The biggest one was the AT&T lawsuit. Linus himself once said he'd probably not have bothered with Linux if BSD was clean. The second, BSD had a slower model of improvement. You needed to have the commit bit to do anything constructive. Meanwhile Linus (later Cox) took code from pretty much anyone that made sense. Third, BSD5 had a radical new kernel design that added a lot of complication for threading with little gain. DragonFlyBSD was forked because of this.

So, IMHO, there were a few things... all of them dented (Free)BSD, and there really wasn't another competitor out there.

Re:Not heard of BSD

[jfdavis668]

386BSD wasn't available until 1992. Since Linus was making an OS to run on the 80386, he started his own in 1991.

I once asked Linus about this

[GerryGilmore]

While working at Intel, we had a large Linux conference with Linus and a few other noteworthy OSS dudes. Afterwards, while we were all millng around, I found myself next to Linus Himself and asked this very question. My belief was that it was the GPL vs BSD license which forced all changes to at least be available for inclusion in the next version. Linux felt that it was more of a timing thing where Linux just kind of hit at the right time. Who really knows?

Single case anecdote.

[aussersterne]

I had been trying to afford a Unix installation at home as a CS student. All I knew was the Unix vendors. I was not aware of the social structure of the Unix world, various distributions, etc. I was crawling university surplus lots and calling Sun and DEC on the phone to try to find a complete package that I could afford (hardware + license and media). Nothing was affordable.

I was also a heavy BBS and UUCP user at the time over a dial-up line. One day, I found an upload from someone described as "free Unix." It was Linux.

I downloaded it, installed it on the 80386 hardware I was already using, and the rest is history. This was 1993.

So in my case at least, Linux became the OS of choice becuase it had traveled in ways that the other free Unices didn't. It was simply available somewhere where I was.

This isn't an explanation for why Linux ended up there instead of some other free *nix, of course, but by way of explaining the social diffusion of the actual files, I saw Linux distros as floppy disks around on BBSs and newsgroups for several years, with no hint of the others.

For someone with limited network access (by today's standards), this meant that Linux was the obvious choice.

As to why Linux was there and not the others—perhaps packaging and ease of installation had something to do with it? Without much effort, I recognized that the disks were floppy images and wrote out a floppy set. Booted from the first one, and followed my nose. There was no documentation required, and it Just Worked, at least as much as any bare-bones, home-grown CLI *nix clone could be said to Just Work.

I had supported hardware, as it turned out, but then Linux did tend to support the most common commodity hardware at the time.

My hunch is that Linux succeeded because it happened to have the right drivers (developed for what people had in their home PCs, rather than what a university lab might happen to have), and the right packaging (an end-user-oriented install that made it a simple, step-by-step, floppy-by-floppy process to get it up) while the other free *nix systems were less able to go from nothing to system without help and without additional hardware for most home and tiny lab users.

For comparison, I tried Minix around the same time (I can't remember if it was before or after) and struggled mightily just to get it installed, before questions of its capabilities were even at issue. I remember my first Linux install having taken an hour or two, and I was able to get X up and running the same day. It took me much longer to get the disks downloaded and written. Minix, by comparison, took about a week of evenings, and at the end, I was disappointed with the result.

Licensing, mostly

[swillden]

It was because Linux more or less worked, and people could use it and add to it because of the GPL. The competitors all had problems:

* Minix was cheap but not free, and couldn't be redistributed with modifications. People worked around that by maintaining patch sets, but that was even more painful then than it is now (we have better tools now).
* The BSDs were in a quagmire of legal uncertainty and competing claims. Nobody knew for sure if BSD was free or not, so everyone assumed it wasn't.
* Xenix: Not free.
* Microsoft: Are you kidding me?
* SYSV: Not free
* HURD: Didn't work, and had such an elegant architecture that it wasn't clear if it could ever work.

That was the space when Linus Torvalds started hacking around (except HURD didn't even exist yet). If he'd been able to hack on Minix, he would have. But the license prevented it, so he took the opportunity to start his own. Lots of other people saw exactly the same situation and joined him in hacking on something that (a) worked, more or less and (b) they could hack on.

It's not that Linux lucked out and the rest of the competition failed. There was no other competition that satisfied the requirements of being free and hackable. It was also important that Linus was an excellent Benevolent Dictator that gave people few reasons to fork. Actually, on that last point it's rather impressive that Linus is still in charge, even after it's become an incredibly valuable property, used and contributed to by lots of megacorps.

The kernel was tied to the culture

[jbolden]

Fundamentally the BSDs and Linux were on par with one another in the mid 1990s. BSD386 was getting sued and that cost a year but it started ahead. Where Linux thrived though was it recruited from Unix users and Windows powerusers. Linux focused on the ability of Unix users the ability to run Unix at home, and focused on the ability for not particularly good system admins to setup servers. It aimed for ease of use. The BSDs conversely aimed to offer a Unix like environment for Intel/Western Digital Hardware; a free version of SCO.

Then of course came the licensing issue. Linux was already ahead by 1997. But GPL allowed companies whose primary goal was not to sell software (or at least the software in question) to cooperate safely. It turned out those companies were larger supporters of free software than companies who were making extensions to base packages.

I think the reason the Linux kernel was successful was that

a) It was very good on par with the BDS kernels originally
b) It was tied to the Linux culture.
c) The GPL

Linux culture beat the BSD culture. The GPL beat the BSD license. The kernel just went along for the ride originally, though of course it had to be good enough to not hamper Linux-OS. After the initial ride the commercial interest led to greater development for the Linux kernel than other kernels which led to it fitting more uses better and more commercial development and interest.... A self feeding cycle that does make the kernel a winning point for Linux-OS.

Re:The GPL

[jcdr]

Systemd is a collection of small executables each with a precise goal, in line with the UNIX philosophy.
Do a "ls -alSrh /lib/systemd/systemd-* /usr/bin/systemd-*" if you want to verify this fact.

Re:why no longer afraid of Linux?

[Anonymous Coward]

time and nobody important being sued

Until CISCO at least. When that happened, the suits went nuts. Where I worked there was a huge initiative to make sure we weren't at any kind of risk of the same thing happening. Every bit of open source we were using was reviewed, a database was created, and a whole process for approving and tracking use of anything we didn't pay for was implemented.

They became really paranoid about it. For awhile you'd have better luck asking for a piece of software that costs $10k than asking for a BSD licensed tool to do the same thing. It's cooled down now, but yeah, it was a real eye opener for a lot of management types.

Re:Snowball effect

[MrDoh!]

That's it, it worked, and the license let it pop up on a few CD cover discs with no hassle. At the time, we were a SCO Unix house, solid, worked, no complaints but the cost wasn't trivial. Still, worked and worked well. Messing about we looked at other unix things, one was.. (going off memory now) MKS Unix I think it was. And... can't recall the other, some 99 quid one that was also rough around the edges. Now, Linux was free, no 'get in touch for commercial reasons' and within a couple of weeks, we got an updated version. The other cheapo unixes we looked at had problems and didn't look like they were being fixed quick. Loading up all our code, and doing a make... it worked. It all flippin worked. Was /really/ scary to see that the compiler was very decent, the headers all included, if it DID need any tweaks, they were so inconsequential that we knocked them out in minutes. Bosses were impressed, but worried about support, and that's why as a company we didn't go official, but all of us said "this is going to dominate one day, it works and works well". But overall... availability/cost was the thing that got us looking at it. Plus, compared to other unix versions, it felt very similar to SCO in it's layout at the time (at least Slakware did, Yggdrasil had better gfx support for the cards I had at the time (Trident...? maybe?) but Slackware was very familiar. Though I'd not have ever expected that one day I'd be lugging a phone around with me that ran Linux at it's core.

Re:The GPL

[tristes_tigres]

Well, no it isn't. Those "small executables" can not function outside the systemd infrastructure. Moreover, systemd people keep trying to expand the range of software that will not build on non-systemd platforms. Please stop your shilling.

Don't break user space!

[duckintheface]

If you read the Linux Kernel Mailing List http://lkml.org/ for a while, you will see why Linux was successful. Over and over again, Linus Torvalds over-rides the antics of his minions and explains to them (again) that the changes they made must be backed out because they break something that users actually need. Linux is elegant and beautiful, but not just because it's a work of art. Linux is the most functional piece of code in existence. It is the beauty of function.

Re:Don't break user space!

[Anonymous Coward]

Linux is elegant and beautiful, but not just because it's a work of art. Linux is the most functional piece of code in existence. It is the beauty of function.

Drinks kool-aid?

Linux is a monolithic system. Not so great, read efficient in a distributed space.

Reason Linux was successful:

Timing: Everyone was fed up on FOSS OSes.
Timing: Everyone was fed up with UNIX
Timing: crappy Windows reliability
Timing: Mac too $$$
Timing: natural for Academia to use Linux (it's free! source viewable!)
Device Driver Development Volunteers (that's a BIG, BIG, BIG, BIG item)
"Do one thing and do it well"
Free (as in beer, aside from Linus being an arse to you)

If BSD had a rich device driver suite, I'd be there in a heartbeat. Many reasons why OSX went that why. It just works better for me compared to Linux, but like Windows in its haydays--device support pushes me to Linux.

UI--who cares, Windows and OSX still win.
Apps--same deal.

Re:Don't break user space!

[amRadioHed]

Don't forget Mac's were at least as unreliable as Windows back when Linux was catching on. Mac OS X only beat Windows XP by a few months.

Re:Don't break user space!

[Runaway1956]

Monolithic kernel. You have a somewhat valid argument there. There are modules that I just don''t use, that are routinely compiled into the kernel. Simple solution: Compile it yourself, without the modules. Strip the kernel down to exactly what you need, and compile it native to get rid of all the 32-bit support. You're left with a monolithic kernel, of course, but the monolith is much smaller.

Re:The GPL

[Aighearach]

Did you even know that almost all of the SysV init scripts call out to system-wide scripts? Try running SysV init scripts just by themselves, without having the monolithic SysV init directories in your path. You can't. Ever. Does one thing without dependencies my ass!

For example on Fedora:

# grep -r '. /etc/init.d/functions' /etc/rc.d/ 2> /dev/null | wc -l
210

This shows how little you understand the arguments you're making, and the commonly proposed solution of using SysV actually achieves it.

The funny part, of course, is that with systemd the start/stop scripts/programs are indeed standalone in many cases, and the best practices will lead to that naturally. Most of the ones that require systemd only require it because they're using a compatibility layer that calls out to the old script.

Re:The GPL

[fisted]

So on Fedora, there are (at most (*)) 210 occurances of init script sourcing one particular system V helper script which, judging by the name, defines common shell functions used by multiple init scripts. What exactly is your point here? The content of /etc/init.d/functions should be replicated 210 times in each of the respective scripts rather than being sourced? With that idea of good design, it doesn't surprise me you're a systemd proponent.

Why do i have the feeling that your real intention was to make a casual reader believe that there were 210 "system-wide" scripts (whatever the fuck thats supposed to mean in this context, what is /not/ system wide at init time?) littered all over the place that the init scripts would depend on. Nice try.

And after failing that hard...

This shows how little you understand the arguments you're making

which leaves me thinking about pots and kettles.

(*) of course the actual command is a failure too, since that period is a wildcard and no boundary is given either. So even a comment somewhere in a script reading "# by the way, we would never source /etc/init.d/functions" would mistakenly be included in your already pointless measurement. For the record, a competent AC would have used

grep -lr '^\W*\.\W\W*/etc/init.d/functions' /etc/rc.d/ | wc -l

(note the -l) rather than

grep -r '. /etc/init.d/functions' | wc -l

Not that the result is anything meaningful.

Re:The GPL

[jcdr]

The point is that you compare a simple script with a systemd-* binary, complaining that you can't run any systemd-* binaries without the systemd infrastructure.

What you fails to understand is that the equivalent of init scripts in systemd is not the systemd-* binary but a unit configuration file. If you want to compare a systemd-* binary, take a binary that provide the same kind of service. What you will find is that the usual code (or libraries) in the traditional service application is replaced by a more standardized API in the systemd-* application. For example, instead of depending on libdaemon, the application will depend on libsystemd. See https://github.com/systemd/sys...

At the end systemd will reduce the number of dependencies required by the applications that use his API compared to an application that will try to provides the same features without the libsystemd API. From the architectural point of view this is a good move.