Slashdot Mirror
Why Was Linux the Kernel That Succeeded?
jones_supa writes: "One of the most puzzling questions about the history of free and open source software is this: Why did Linux succeed so spectacularly, whereas similar attempts to build a free or open source, Unix-like operating system kernel met with considerably less success?" Christopher Tozzi has rounded up some theories, focusing specifically on kernels, not complete operating systems. These theories take a detailed look at the decentralized development structure, pragmatic approach to things, and the rich developer community, all of which worked in favor of Linux.
And Linus was not.
Besides, the "copy" is the one that gets passed around (Linux was a copy of Minux).
That and the Unix philosophy.
Amazed that neither the GPL nor the legal uncertainties surrounding BSD at the time (hey, remember those days!) were really focused on, but meh. I think like everything else that became wildly popular in spite of plenty of seemingly equivalent or better alternatives, it just came down to dumb luck and momentum.
Somehow Linux got the ball rolling, people gathered around it, it gained steam, and here we are.
I'm surprised he didn't include a "BSD IS DYING, NETCRAFT CONFIRMS IT" in the article.
Trolling is a art,
WTF do you guys even try and English?
GPL fights fragmentation while BSD does not. Also Minix was proprietary.
To piss off RMS...
Also: What? "That Succeeded"?
Maybe some ideological reasons, but primarily he didn't know BSD was around.
Ahahhahahahaha If you are going to troll, you should try and do so intelligently, /. _used_ to have those.
Nobody will upvote you 'cause 'teh linux rules, android is proof!!!11' but you speak the truth. At least for those of us who do real work on computers.
| wonder what finally made the c-levels unafraid of linux?
time and nobody important being sued
Why was the grammatical badly?
Do they make you guys demonstrate any proficiency in the English language at all?
Lost at C:>. Found at C.
20MB disk drive - $1000
386SX based PC - $3000
Three days to get the X server doing something useful with a 1024x768 interlaced, no-name monitor - priceless!!
PS: I really thought this 20MB disk drive is all I will need for the rest of my life...
When I was a kid, i read a book called Stone Soup. It was about these guys that wanted to eat, had nothing but a pot. They put in some stones... called it stone soup.
Eventually people got curious, and added things.. the soup became real. Going from water and rocks, to where real ingredients went in, and the stones just fell away. A seed, but then dropped when something real came.
I always thought of Linus as a guy who managed the Stone Soup well. It wasn't specially good in .01 version. But he made people want to add to it. The GPL helped some. Linus chose that license, not as a "hey Im a zealot and you need to give me everything you write" but he thought "if people do cool things they need to let me see their cool things"
That, and FreeBSD had a few handicaps. The biggest one was the AT&T lawsuit. Linus himself once said he'd probably not have bothered with Linux if BSD was clean. The second, BSD had a slower model of improvement. You needed to have the commit bit to do anything constructive. Meanwhile Linus (later Cox) took code from pretty much anyone that made sense. Third, BSD5 had a radical new kernel design that added a lot of complication for threading with little gain. DragonFlyBSD was forked because of this.
So, IMHO, there were a few things... all of them dented (Free)BSD, and there really wasn't another competitor out there.
If you think that is the truth then it would indeed have been better for it to have remained unsaid.
It's not a big mystery. Linus released a primitive kernel that worked, at the right time, with the right license, and then diligently kept rolling up contributions and releasing the result.
Expanding a bit:
While the first release of Linux was primitive, it worked. People hungry for a free *NIX were able to grab it and run it. At the same time, HURD was a big research project; if you just wanted to run *NIX on your own computer, HURD was not at all ready.
Linus released it at the right time. It was just becoming possible for large numbers of people to get the kernel from him, and to send contributions back to him. Something like Linux might not have worked at all before the Internet and/or BBSs; it would have been too difficult to send releases out and collect contributions of code.
Linus used the right license. He has said that the choice of releasing under GPLv2 was the best decision he ever made. Arguably a BSD license could have worked as well, but many people have a fear that if they contribute to BSD projects, that evil companies might benefit from their work; with GPL they are comfortable contributing. Also, IBM contributed some code that uses IBM patents; because of the GPL, IBM knew that commercial entities wanting to use the patents would still need to license the patents from IBM. So Linux got the largest possible pool of contributors.
And then there is the fact that Linus worked hard managing Linux. He collected patches sent by people and rolled them in. These days he writes very little code himself; almost all he does is manage patches. I'm not sure how much code he wrote in the early days, but I think his diligent application of patches sent to him helped Linux to become stable and useful.
Given all of the above, Linux was a success, and success bred more success (the "Snowball effect"). People who just wanted something that worked could grab Linux as it worked better and better all the time; people who wanted to join an active project joined it as it was the most active project.
If Linus hadn't released when he did, another project might have gotten the snowball momentum thing going and become the big popular project. But he did, and he worked hard to keep it going, and the result is the kernel that changed the world.
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Overallocations, OoM Killer, hangs and freezes all because the kernel can't be bothered to keep track of every last bit of it's memory make the Linux kernel a piss-poor substitute for a real UNIX kernel.
There are other problems in the memory subsystem as well. For example, if you are not using swap and the system begins to run out of memory, it starts throwing out pages of active programs from memory. Very soon they are loaded again from disk when those parts are needed. This causes a disk-grinding circus that feels like swapping.
BSD did not in any sense fail or fizzle, it is ubiquitous in printers, network gear and other appliances. Funny Apple could take the Mach that GNU HURD mishandled and make a working core for their BSD variant. Stallman really does suck at leadership/management/implementation, idea man only.
Slackware distribution. Drop the mic.
While working at Intel, we had a large Linux conference with Linus and a few other noteworthy OSS dudes. Afterwards, while we were all millng around, I found myself next to Linus Himself and asked this very question. My belief was that it was the GPL vs BSD license which forced all changes to at least be available for inclusion in the next version. Linux felt that it was more of a timing thing where Linux just kind of hit at the right time. Who really knows?
I had been trying to afford a Unix installation at home as a CS student. All I knew was the Unix vendors. I was not aware of the social structure of the Unix world, various distributions, etc. I was crawling university surplus lots and calling Sun and DEC on the phone to try to find a complete package that I could afford (hardware + license and media). Nothing was affordable.
I was also a heavy BBS and UUCP user at the time over a dial-up line. One day, I found an upload from someone described as "free Unix." It was Linux.
I downloaded it, installed it on the 80386 hardware I was already using, and the rest is history. This was 1993.
So in my case at least, Linux became the OS of choice becuase it had traveled in ways that the other free Unices didn't. It was simply available somewhere where I was.
This isn't an explanation for why Linux ended up there instead of some other free *nix, of course, but by way of explaining the social diffusion of the actual files, I saw Linux distros as floppy disks around on BBSs and newsgroups for several years, with no hint of the others.
For someone with limited network access (by today's standards), this meant that Linux was the obvious choice.
As to why Linux was there and not the others—perhaps packaging and ease of installation had something to do with it? Without much effort, I recognized that the disks were floppy images and wrote out a floppy set. Booted from the first one, and followed my nose. There was no documentation required, and it Just Worked, at least as much as any bare-bones, home-grown CLI *nix clone could be said to Just Work.
I had supported hardware, as it turned out, but then Linux did tend to support the most common commodity hardware at the time.
My hunch is that Linux succeeded because it happened to have the right drivers (developed for what people had in their home PCs, rather than what a university lab might happen to have), and the right packaging (an end-user-oriented install that made it a simple, step-by-step, floppy-by-floppy process to get it up) while the other free *nix systems were less able to go from nothing to system without help and without additional hardware for most home and tiny lab users.
For comparison, I tried Minix around the same time (I can't remember if it was before or after) and struggled mightily just to get it installed, before questions of its capabilities were even at issue. I remember my first Linux install having taken an hour or two, and I was able to get X up and running the same day. It took me much longer to get the disks downloaded and written. Minix, by comparison, took about a week of evenings, and at the end, I was disappointed with the result.
STOP . AMERICA . NOW
It was because Linux more or less worked, and people could use it and add to it because of the GPL. The competitors all had problems:
* Minix was cheap but not free, and couldn't be redistributed with modifications. People worked around that by maintaining patch sets, but that was even more painful then than it is now (we have better tools now).
* The BSDs were in a quagmire of legal uncertainty and competing claims. Nobody knew for sure if BSD was free or not, so everyone assumed it wasn't.
* Xenix: Not free.
* Microsoft: Are you kidding me?
* SYSV: Not free
* HURD: Didn't work, and had such an elegant architecture that it wasn't clear if it could ever work.
That was the space when Linus Torvalds started hacking around (except HURD didn't even exist yet). If he'd been able to hack on Minix, he would have. But the license prevented it, so he took the opportunity to start his own. Lots of other people saw exactly the same situation and joined him in hacking on something that (a) worked, more or less and (b) they could hack on.
It's not that Linux lucked out and the rest of the competition failed. There was no other competition that satisfied the requirements of being free and hackable. It was also important that Linus was an excellent Benevolent Dictator that gave people few reasons to fork. Actually, on that last point it's rather impressive that Linus is still in charge, even after it's become an incredibly valuable property, used and contributed to by lots of megacorps.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Fundamentally the BSDs and Linux were on par with one another in the mid 1990s. BSD386 was getting sued and that cost a year but it started ahead. Where Linux thrived though was it recruited from Unix users and Windows powerusers. Linux focused on the ability of Unix users the ability to run Unix at home, and focused on the ability for not particularly good system admins to setup servers. It aimed for ease of use. The BSDs conversely aimed to offer a Unix like environment for Intel/Western Digital Hardware; a free version of SCO.
Then of course came the licensing issue. Linux was already ahead by 1997. But GPL allowed companies whose primary goal was not to sell software (or at least the software in question) to cooperate safely. It turned out those companies were larger supporters of free software than companies who were making extensions to base packages.
I think the reason the Linux kernel was successful was that
a) It was very good on par with the BDS kernels originally
b) It was tied to the Linux culture.
c) The GPL
Linux culture beat the BSD culture. The GPL beat the BSD license. The kernel just went along for the ride originally, though of course it had to be good enough to not hamper Linux-OS. After the initial ride the commercial interest led to greater development for the Linux kernel than other kernels which led to it fitting more uses better and more commercial development and interest.... A self feeding cycle that does make the kernel a winning point for Linux-OS.
It's more accurate than the post it is responding to, and states it in a neutral manner.
We would probably be using FreeBSD by now.
I know because I remember BSD and BSDi back in the 1990s in highschool for running BBSes. MkLinux I kind of heard of but didn't know exactly what it was until the late 1990s when Linux was the only OS.
FreeBSD was the new myspace and momentum was on GNU/Linux at this stage. In tech you need to be at the right place at the right time. Linux was there when the world wide web became available to the public and BSD unix came out during hte exact same time.
http://saveie6.com/
Why was Sanders the Colonel that succeeded?
Why Was Linux the Kernel That Succeeded?
Linux is (just) "A" successful kernel - NOT "THE", NOT EVEN THE "MOST" (NOT EVEN AMONG THE "OPEN" KERNELS). Excluding the most successful kernel (i.e., Microsoft's), and among only open-source kernels, the most successful (open-source) kernel is... BSD!
Thank you, i will go now to some "/." story and make fun of APPLE's fans who think that their glorious shit come straight from Steve's (R.I.P.) pure ass.
Antisthenes: "Wisdom begins by examining the words/names." - excuse my English, i am (slightly...) better with my Greek!
Some people may debate me on the easy part, but it was easier that most free OS to install. I tried FreeBSD(or OpenBSD, it was a while ago) but quickly gave up when it wanted me to put in what Cylinder Head Sector to start the partition at and which CHS to end at. Slackware at the time needed you to tell it which CHS to start at, but also told you the first CHS that was free, then it would ask for the size of the partition, telling you the maximum size for the CHS you had chosen.
Linux took some thought, and a couple of tries as you had to leave room to create a swap drive, which it didn't tell you at first, but it gave you some hints about how big to make it after you had made your OS partition. It was vastly easier than BSD.
I had already had the fun of installing and using WinSock on Windows 3.11, so the Microsoft world had prepared me for the pains of setting up PPP and TCP/IP under Linux.
It may not have had the best coding, but it worked on most X86 computers that people had at the time. DOS with Windows was the only OS that was easier, and it wasn't much easier. Also DOS with Windows wasn't free. Timing is everything.
It was almost as easy to setup as the best paid product at the time. One it was working, you could easily download and use many productivity tools. Bash was familiar to Unix people, and anyone who had done any programming. X was pretty easy to get running, even it if was really hard to get running at the full resolution/colour depth of your video card.
It hit at the right time, was easy to setup, easy to install more software, and free.
Now it is easier to initially install than Windows. Where I find it lacking is that tweaking of the setup after installation that has lagged behind. Windows has the control panel. Each X windows manager has its own control panel which all seem to miss this or that feature. I'll admit, it has been a few years, so this may have improved, but Linux on the desktop just needs easy, graphical configuration tools for everything and it will be ready for everyone to use. Microsoft has given everyone else a window of opportunity with the debacle of where is this setting in Windows 8.x.
Microsoft, Apple, Google, Amazon what's the difference? All steal money from devs and control with walled gardens.
"Extremism is defense of liberty is no vice!" - Barry Goldwater
The appeal of the cute fluffy penguin coupled with religious angst about the BSD daemon sealed the deal for Linux.
Nullius in verba
Due to latitudinal issues, everything that happens in Finland is cool. For instance, our babies eat your marines for breakfast.
Escher was the first MC and Giger invented the HR department.
People who really know how to troll mostly grew up and either stopped or found ways to make money doing it. We now have what's left.
"Be particularly skeptical when presented with evidence confirming what you already believe." -
Back when I first ran across it, a tech company was running it for it SQL and SMB servers, without needing to pay the high lic fees.
In those early days, the ease of which those services could be set up on a network were key.
So was the ease of development.
Yes, many others existed. But it was harder to get SQL running on them, or you couldn't use them to share your one or two big hard drives with the windows 3.11 or windows 95 machines on the network.
On top of that, it was easy to do other useful stuff with. Easy to write applications for it.
Microsoft and IBM ended up in a spat over OS|2 and parted ways. That left IBM very angry at Microsoft and without an x86 operating system. IBM spent $1B on Linux in the early 1990's and made a major marketing campaign out of doing it. They even ported Linux onto their sacred mainframe - the Z-series. This IBM support legitimized Linux and propelled it into becoming what it is today.
Let's just imagine if Apple had produced more affordable PC's early on rather then the gray box of Windows PC's. Do you think Apple would have had more developers do software for Mac's and maybe Windows and OS X or at that time OS 9 would have had more equal users? Just imagine if both Windows and OS 9 sucked so bad that many user gravitated to Linux because it was free? Stuff just happens sometimes and while Linux to me suffered only from so much fragmentation that it had a hard time with creating a identity. Chrome OS has had success because of Google's identity and while many Linux purists won't say Chrome OS is Linux. It is indeed and a definite success at that.
Over the next couple years I worked there we looked onto potentially OS/2 and... I want to say DRM DOS? as a potentially cheaper multitasking alternative to Xenix on our systems. Even though I was nominally aware of BSD, the amount of tinkering to even try to get it working was intimidating, and we didn't have the immediate need for it.
By the mid 90's people were really starting to talk about Linux in exactly the sort of way they were NOT talking about BSD. I looked at the procedure to install it -- download a bunch of slakware install floppies off the newfangled internets (24 install floppies as I recall, which took for-fucking-ever! And I accidentally FTPed the first two in text mode. Shit!) and boot that shit up. I specifically remember finding the installer to be far less sucktastic than either the OS/2 installer or the Windows 3.1 installer that you ran shortly after pirating MS DOS (Which you typically would do even if you had a legitimate MS DOS install on your system.)
In short order, I had a working Linux system with a working C compiler, no fuss, no muss. Well some fuss -- couldn't run X11 very well on the VGA controller I had, but I was fine with a text console until I bought a computer that wasn't made out of duct tape and baling wire, that being the custom of the time. I almost immediately set up a TCP/IP network between the real computer and the baling wire computer, too, experimented with NFS, all that fun stuff. Got my system pwned several times, you know, all the usual stuff you go through to learn how to become a halfway decent Linux admin.
So yeah, for me at least it was all about accessibility. Minix was just a toy and BSD required a wizard hat and robe.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
XNU/BSD has been incredibly successful. Hard to argue whether it's more or less so than Linux, but it's not a flat out win for Linux by a long shot.
http://en.wikipedia.org/wiki/Darwin_(operating_system)
opensource.apple.com
At the heart of it, kernel development seems to always have been very YAGNI and agile, while all others were thinking about great concepts and not producing any real-life results.
If you read the Linux Kernel Mailing List http://lkml.org/ for a while, you will see why Linux was successful. Over and over again, Linus Torvalds over-rides the antics of his minions and explains to them (again) that the changes they made must be backed out because they break something that users actually need. Linux is elegant and beautiful, but not just because it's a work of art. Linux is the most functional piece of code in existence. It is the beauty of function.
"He took a duck in the face at 250 knots." -- William Gibson, Pattern Recognition
Because Theo is an asshole obviously. Then again, I consider the BSD kernels a spectacular success just as well. A big part of the internet as we know it was built on VAXen running 4.x BSD long before Linus started working on a kernel.
Period. And BTW, Linux still blows chunks compared to any BSD.
/. community are. systemd anyone? LOL!
Which only goes to prove what idiots 95% of the
Tozzi overthinks it in the article. The kernel succeeded by being in the right place in the right time and then continuously being good enough that there was insufficient reason for change.
Linux, the OS not the kernel, was the first mostly complete Unix available on a college student's budget that would install on hardware the college student mostly already had. Right place, right time. Hurd didn't exist in any usable form, Minix and Solaris were $$ and the *BSD's didn't start to release for a year or two later.
Fast forward four years and when those graduating college students met the Internet bubble, Linux was the server OS they knew. Right place, right time.
Byeond that it was a game of, "don't eff it up." That's where Torvalds' pragmatism came in to play.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
The way I remember it, it was easy to get the kernel, drivers for most major devices were available.. .and then the GNU utilities appeared. Suddenly there were a bunch of usable commands in a distribution which was free.
Hmmm, a "community" being worse off because of more freedom. Interesting thought. I would say 'communities' always enforce some restrictions, laws and customs. They have to work out how to resolve the inevitable conflicts with many people living together. Granted, they can carry this too far and be stifling. (People leave small towns for the big city to get away from everyone minding everyone else's business for instance.) So, there has to be a balance, and a careful evaluation of what freedoms to respect. (Freedom of speech is very important, but some Supreme Court Justice said that it doesn't give you the freedom to yell 'Fire!' in a crowded theater.)
Other submitters have given plausible reasons why the 'freedom' of the BSD license means people don't contribute back and that weakens the whole. I had never thought about the issue before, but now, reading them, those arguments make sense to me.
I've noticed that abstractions like 'Freedom' often seem to break down when you look at them very carefully and think about edge conditions, so that when people advance an argument solely on the basis of an abstract principle, I tend to be a bit cautious.
In theory, theory and practice are the same; in practice they're different. (Yogi Berra & A. Einstein)
if you are not using swap and the system begins to run out of memory, it starts throwing out pages of active programs from memory. Very soon they are loaded again from disk when those parts are needed. This causes a disk-grinding circus that feels like swapping.
What you describe /is/ swapping, which makes it odd since you started with "if you are not using swap".
Care to elaborate on what exactly you did, under what circumstances?
CLI paste? paste.pr0.tips!
The whole initscripts mess will not work outside the sysv-rc + sysvinit infrastructure too.
Others platforms only have to add the required syscalls to get systemd, or a why to get the same kind of functionality. Linux has added new specific syscalls since very long time, nothing new on this subject.
I can take a Tomcat SysV/Linux start-up script and probably drop it on a BSD system and probably have it work. Just edit rc.local to call it with the "start" option and we're done.
Can I take a systemd-foo binary, put it on a non-systemD system and have it work?
Yes, they are separate binaries, but they are tightly coupled [1] to the overall architecture and cannot be used outside of it.
[1] https://en.wikipedia.org/wiki/Coupling_(computer_programming)
I'd also be curious about what a real UNIX would do differently in this situation (after answering parent's questions)...
LAMP offered a way to rapidly spin up a .com on commodity hardware that could then woo the VC.
comment first, facts later. http://chem.tufts.edu/AnswersInScience/RelativityofWrong.htm
You could install a fully functional system from a box of floppies and it ran great on clunky 386sx. Multitasking and networking capabilities blew other choices of the time - Windows 3.1, DOS and MacOS classic - out of the water. BSD distributions were not nearly as complete software wise or easy to install. Lack of shared library support made X apps impractical.
Without Slackware the would have been no Linux of today.
I think what made people stand up and take notice of Linux was IBM's decision to port Linux so it works on IBM's mainframe hardware. In short, having an open operating system work on IBM mainframes showed that Linux was viable even running highly mission-critical tasks.
Indeed, that success paved the way for the Linux kernel to be used on consumer devices--Google's Android for cellphones/tablet computers and Chorme OS for low-cost laptops both run on the Linux kernel.
Complete rubbish but I don't blame you since it's commonly believed rubbish. As early as 2003 I was getting software sold by Halliburton, about as commercial as anyone on the planet, which contained the binary of EMACS among the other files. So long as they kept to the terms of the licence (which was really just supplying a text file with little more than the licence in it) they were legally entitled to do it.
Their user interface had an EMACS back-end but that didn't stop them for selling their software for many thousands per year per seat.
Fast forward to now and "busybox" is in just about every little device with a CPU you can think of. It's GPL licenced software.
This isn't an attempt at trolling.
The memory manager in Linux kernel based OSes does not keep track of exactly how much memory has been allocated or reserved.
This is why you can have a server with 32gb of RAM + 8gb of swap, and enough apps allocate and reserve more than 40gb combined. Then when the apps actually try to use their "reserves" the OoM killer kicks in, randomly killing databases, web services, etc..
A "real" UNIX system will not allow allocations + reservations to exceed the configured resources.
A "real" UNIX system will say sorry bub, nothing left.
as long as I'm throwing this out there, I'll add one more...
A "real" UNIX system doesn't require that RAM and SWAP be two distinct resources, but combines them into one single "virtual memory pool", where your app only has to malloc what you need, and no reservations for swap required.
A "real" UNIX system keeps track of exactly what's been allocated and where, and then says "Nope, sorry bub, all out." to any programs trying to request more memory or to reserve more swap.
Seriously? It fails on so many of those points - "simple parts connected by clean interfaces" and "Design for visibility to make inspection and debugging easier" for a start.
It's introducing MS style voodoo debugging where you just have to keep unplugging things until the machine boots because nothing on the screen is going to tell you anything.
Uh no, all Unixes at that time allowed over-allocating.
Since nothing related was logged or echoed to the screen (there was a PulseAudio message but that's all) all I know is it hung until I came back and rebooted it, then hung again a second time. The third time I pulled the dongle until it had got as far as X then plugged the dongle back in. Fine since then on that Fedora desktop.
So I very much doubt it's udev alone since it's worked the couple of times since so long as I plug it in after it's made it to the X runlevel (or whatever it's called in systemd).
That's just the latest of a list of things that have convinced me systemd just is not ready to be shipped in distros yet despite whatever is decided with Redhat and Gnome office politics.
What you describe /is/ swapping, which makes it odd since you started with "if you are not using swap".
Care to elaborate on what exactly you did, under what circumstances?
My description is accurate. I have just been normally using the computer. Even without any swap, the HDD goes "krrrrrr..." and the system becomes very unresponsive when you begin to run out of memory. You can easily try it yourself, as it is reproducible every time.
It seems to throw out program pages from memory if it knows that they are disk-backed. It seems to be hard to trigger the OOM killer in this condition as well, even though it should happen.
is open source and probably runs on a billion devices.
Any sufficiently advanced technology is indistinguishable from Macintosh...
I picked it because it had operational sound, and FreeBSD (the only alternative available to a programmer in the middle of nowhere) did not.
However, I think the picking of linux by NASA folks to develop beowulf clustering was the real prize - they turned the networking layers (and many other points of performance) into an absolute dream. The experimentation levels with optimization didn't hurt either - when Linus Torvalds was going for his PhD particularly.
COW pages were particularly a breakthrough, oddly enough, as was the whimsical clone() call and threading built on that.
What you describe /is/ swapping, which makes it odd since you started with "if you are not using swap".
Care to elaborate on what exactly you did, under what circumstances?
My description is accurate. I have just been normally using the computer. Even without any swap, the HDD goes "krrrrrr..." and the system becomes very unresponsive when you begin to run out of memory. You can easily try it yourself, as it is reproducible every time.
It seems to throw out program pages from memory if it knows that they are disk-backed. It seems to be hard to trigger the OOM killer in this condition as well, even though it should happen.
I believe these memory problems can be somewhat mitigated by some hand tuning of the vm parameters "swappiness" and "vfs_cache_pressure". Unfortunately I don't have the time at the moment to setup a good test, but they're worth a shot if you find yourself in that situation in the future. Ref: https://www.suse.com/documenta...
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
So far as I know, by default, Linux still over allocates up to 50% of total RAM (not 50% of available memory if you count virtual memory). This parameter can be tuned or set via sysctl. But, yeah, I love Linux, but it's got memory management issues that I don't have on other operating systems. I've found I can usually tune around them somewhat.
Although I've never found a way to have it do what I'd really like and dedicate lots of memory for buffers when writing large files. For instance, when I'm writing a disk image via dd to a reasonably fast RAID array and I've got 32 GB of RAM, I wish I could tell the kernel to not waste the RAM caching the pages I'm writing (they'll be evacuated when memory exhausts, and if I decide to compress the image, the 'end' of the file that's cached will be evacuated to make room for the stuff I've compressed), but rather to dedicate them to the buffer so that once the disk I'm reading is at the slow end of the disk, my RAID array isn't just sitting idle waiting on IO.
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
At that time downloading a couple of MB was quite difficult and the more so if you needed to store them on floppies.
Yes, Prentice-Hall released MINIX source code and binaries on floppy disk with a reference manual but it was priced at 50 bucks (A.D. 1992). I remember Tannebaun proclaiming that the price was right and students should pay up.
Back in the 90s, Tannenbaum's licensing for MINIX was free to use it unless you made money. Then Tannenbaum wanted his cut. Linus tried to get the doc to change license to GPL, but Tannenbaum told him ' if you do not like it, develop your own'.
I prefer the "u" in honour as it seems to be missing these days.
It "succeeded" because GOOGLE pushed billions of dollars into it to keep it alive.
swappiness won't help him if he doesn't have any swap. He probably does have swap but isn't aware of it.
For the record, I built CyanogenMod the other day on a Debian machine with actually no swap, and at some point when linking chromium, the linker would consume more RAM than i had available (roughly 2.5 gig). When it reached that point, the process received a SIGKILL by the OOM-killer (a questionable concept, but that's beside the point).
CLI paste? paste.pr0.tips!
You can easily try it yourself, as it is reproducible every time.
I wouldn't have been so curious if i hadn't happen to have encountered a similar situation, but without the mysterious swap-but-no-swap stuff happening. (See my response to sibling for details).
If what you say is true, then you're swapping. If linux does that automatically now, even when swap is disabled, I'm happier than ever to use a BSD...
CLI paste? paste.pr0.tips!
Apparently someone built a better moustrap.
Tracy Johnson
Old fashioned text games hosted below:
http://empire.openmpe.com/
BT
Did it actually evict all the page cache or did you hit the overcommit limit to trigger the OOM killer?
I have mixed feelings about the killing of processes; while I don't necessarily agree with the method, I can't argue with the results. Still, it'd be nice to try a SIGHUP and SIGTERM first.
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
Linux is definitely a success story, both according to its original author and many in the community. However, it's not *the only* success story of a widely-used open source OS kernel other than Windows.
For example, the OpenSolaris kernel (and the rest of the operating system) is free software and open source, mature, well-tested, stable, and has a pretty large install base. Solaris is a different matter entirely since it's no longer open source, but the community that used to be behind OpenSolaris is still very active on e.g. Illumos, SmartOS, etc.
Sure, OpenSolaris is no longer a legitimate contender for the desktop (there was a time around 2006-2008 when it was more or less on-par with GNU/Linux on the desktop, believe it or not!), but it's still widely deployed on servers for all sorts of tasks, and it has an incredible compatibility story, too. You can run binaries compiled in the early 90s on a modern SmartOS machine. The Linux devs would just tell you to recompile from source, after fixing any build errors.
And let's not forget that (Free/Open/Net)BSD are also widely used. Again, their viability for *modern* gaming/desktop use is pretty limited (though some would argue otherwise, they're still way behind Linux, if for no other reason than proprietary games only run properly on a "real" Linux kernel), but *BSD OSes are used in a lot of routers, home servers, and yes, production servers for pretty important websites and web services.
I don't believe that Linux is the only winner in the battle for having a viable FOSS operating system based on a FOSS kernel. It's definitely the best we have when it comes to playing games and watching video, but that's because a lot of the proprietary elements that want to protect their content are only willing to support platforms with a huge landslide of installed users, and *BSD and *Solaris/Illumos/SmartOS are definitely not that.
But it would be irresponsible for us to judge winners and losers solely by their ability to present a nice GUI, since we use software for a lot of things that don't need a GUI, or can provide their GUI via a web app (and run that web app on whateverOS, be it Windows, Mac, Linux, Android, etc.), and can very viably be hosted -- with high performance and security to boot -- on *BSD or Solaris derivatives.
Yes, there have been power grabs, politics, and any other kind of thing we could guess (I only assume someone has tried to 'buy Linus off' but thankfully he and his family have been well taken care of so that wasn't a primary need.
There are other great leaders and curmudgeons in the Linux arena, but Linus is still top dog, and for one, I approve.
... "When you pry the source from my cold dead hands."
Did it actually evict all the page cache or did you hit the overcommit limit to trigger the OOM killer?
I don't know; I only used Linux here because the Android build system doesn't seem very portable. Not too familar with the implementation details of Linux' OOM killer.
Here's the log
The linker (gold) is responsible for the OOM-situation by having by far the largest amount of allocated memory, but it's the java compiler that actually triggers the OOM killer, causing gold to be killed. This somehow makes sense, but at the same time, meh. In my particular case, I'd have preferred the linking to succeed, at the cost of javac. But well, it's heuristic.
I have mixed feelings about the killing of processes; while I don't necessarily agree with the method, I can't argue with the results. Still, it'd be nice to try a SIGHUP and SIGTERM first.
Yeah. Actually in the logs i linked above, there's no indication of what signal was delivered; I think on the command line it gave something along the lines of ``terminated by signal 9''. I might have been mixing it up, with something else, too.
Then again thinking about it, the "more polite" signals are probably futile in a OOM situation.
CLI paste? paste.pr0.tips!
Eh, swallowed some quote tags there, sorry.
CLI paste? paste.pr0.tips!
I didn't have access to that tier of internet service at home at the time—I had UUCP and a Fido tosser, both of which dialed out over a modem once a day for a multi-megabyte sync.
Sun3/4, HP900, and some DEC boxes were all in use in the CS labs at my school at that period, though they were getting old. So I had ftp access there, but there was no removable storage on those machines, and no direct way to access them from my dial-up, and my account quota was pretty low on the NFS server, just enough to hold a bunch of C code for class and the binaries it produced, so space was pretty precious. For those reasons, it never occurred to me to ftp or gopher around at school for software to figure out how to take home. Plus lab access was limited and you really wanted to spend your time there doing your homework problems and getting them to compile and run, not dinking around looking for freeware.
Minix I read about on Usenet and tracked down some binaries, IIRC. But it wasn't impressive. Every now and then I'd hear something about *BSD, but it really wasn't clear how to get my hands on it, and nobody could really tell me. The people "in the know" as far as I knew were in the CS department and they were using the commercial unices and knew those pretty well.
In the Fido and Usenet groups I was in, Linux was the thing that turned up often and easily. Maybe I was reading the wrong groups, who knows. There were a hell of a lot of them in those days, if you recall, and it was all pretty noisy.
But Linux seemed then and over the several years that followed to come up over and over again. Others—not so much. It is what it is.
STOP . AMERICA . NOW
Sure, the hardware was pretty cool, but the prices were just really, really high.
I remember a call to Sun where I was asking about a SunOS license for a used 3/80 I was thinking of buying from the department, and it was going to run me like $3,000 for the OS or $5,000 with development environment or something along those lines, and it was quoted to me as the *academic* price, and no matter which media delivery I wanted I would have had to buy additional hardware to read it, and so on. I mean, that's $5,000 to $8,000+ in today's cash for an already old, low-end Unix system at the time. I remember pleading with the person on the other end of the line to help me brainstorm and find other options, as I was a CS student and needed to be able to do my homework at home and all of that, but of course, they just felt like I was tying up the line—I looked sort of ridiculous from their perspective.
And then they told me that it was really too old to be useful, that I wanted an IPC or an IPX, I forget which, and the prices were well into five digits, again *academic* price for a bare bones configuration. And here I am a broke CS undergrad already struggling to pay $4k/year in tuition to a state school. It was a total non-starter.
Meanwhile, the first purpose-specific Linux box that I assembled (as my Linux excitement grew and I knew I wanted to do a dedicated build) was a 386/40 with 8MB RAM and about 1GB ESDI storage, along with a Tseng ET4000 VGA card. It was all used gear, again bought in surplus channels, but the thing was that it got me beyond what the 3/80 would have provided in terms of performance, and was perfectly servicable at the time as an X+development box, and it cost me a total of like $200. That was just plausible.
I built a sync converter circuit to connect an old Tektronix 19" fixed-sync color monitor to the VGA port and felt like I had a real, honest-to-god Unix workstation for $300, with a very competent development environment and Emacs, NCSA Mosaic, etc., rather than having had to spend 10x that much for less in the end.
People talk about *BSD, but the driver support on *BSD wasn't as good even a few years later when I looked at it. Linux supported crap hardware in addition to great hardware, which sounds bad until you realize it means that any broke student could scrounge around in the boardbucket and put together a fairly decent Linux system for peanuts.
STOP . AMERICA . NOW
https://www.youtube.com/watch?...
I think it was the 'go team' effect. People had a share in building it up and felt like part of community... I didn't start using it until 1995, and Not being a person who thinks in C or assembly didn't have much to contribute at the time... Always been a proud user though... I love the simplicity, elegance and low overhead of shell scripts for getting things done and automated.
For me as a young college student still eager to learn about all the alternative OS-es it was the availability of Linux. I could just go to the local computer shop and buy a box with Suse or Redhat. All other alternatives were simply way too expensive for a student or were only available on the Internet in a time were you would block all incoming and outgoing calls for days as long as you were downloading your alternative OS.
swappiness won't help him if he doesn't have any swap. He probably does have swap but isn't aware of it.
He might have swap, but one doesn't have to have swap for that problem to occur.
Throwing out the read-only pages of programs without writing them to disk and then reading them in later when they're needed is different from paging read-write memory to disk and paging them back in again (swapping)
The difference is that in the first one you are not writing anything to disk. For example program A, a 1 megabyte program using ten 100KB pages is running. Those 100KB pages are loaded from disk when you start program A and are then executed. They are read-only, in that the contents of those pages do not change during the execution of program A.
During a context switch when program A is not running, program B is started, which loads three 100kb pages from disk. The OS then loads program B into the first three pages of the memory used by program A (after all, program A is not running *at* *this* *moment*). When the scheduler switches back to program A, the OS then reloads the first three pages of program A from the disk before program A starts running again. Another context switch later program B needs to run so the OS loads program B back into the first three pages of program A....
Considering that the scheduler might decide to give each of program A and program B equal time and context switch thousands of times per second, it's no wonder that the disk might not be able to keep up. However we see this happen very little these days, mostly because executable pages tend to be so very small relative to the data memory that they use, most people have swap enabled and disks are a great deal faster than those 80MB drives that I used to see this problem occur on.
I'm a minority race. Save your vitriol for white people.
You are correct - systemd's lack of output and logging at that came to the rescue so it is absolved of blame!
It's not that I hate it. It's just that I would prefer it to be beyond alpha quality before I use it.
I've seen how people who report systemd problems are treated so no, I'd rather avoid it than get into a shouting match with someone who perceives wishes that they get what they deserve someday as death threats. It's easier to decide not to use that bit of cheap hardware and move on - or install CentOS6 for a more stable environment that was tested a bit instead of rushed out by sheer force of ego.
So examples are worthless now are they? Sorry about being "reality based".
Yep, I'm certain that I had no swap allocated. The only explanation I could make was that program pages were thrown away and then reloaded soon again.
For example program A, a 1 megabyte program using ten 100KB pages is running. Those 100KB pages are loaded from disk when you start program A and are then executed. They are read-only, in that the contents of those pages do not change during the execution of program A.
During a context switch when program A is not running, program B is started, which loads three 100kb pages from disk. The OS then loads program B into the first three pages of the memory used by program A (after all, program A is not running *at* *this* *moment*). When the scheduler switches back to program A, the OS then reloads the first three pages of program A from the disk before program A starts running again. Another context switch later program B needs to run so the OS loads program B back into the first three pages of program A....
Thanks for the explanation, I didn't know this is actually a thing. It sounds slightly^W insane, though, mostly because (as you note yourself) how much longer disk I/O tends to take compared with context switches, or the timeslots remaining at typical scheduling frequencies..
CLI paste? paste.pr0.tips!
The smoking gun was a few posts above: hangs on init repeatedly - remove device, doesn't hang - insert device after systemd has pissed off out of the way and lo, and behold, udev and the kernel get their shit together and use the device with no problems.
I think it's just a little bit disgusting that you are ignoring that for the purpose of evangelism and treating me like an idiot that can't see what you are doing.
now, I never see bsd mentioned anymore in job ads.
Then you aren't looking at the iOS or game development boards. Apple iOS is based on FreeBSD, and the PlayStation 4 game console runs Orbis OS which is also based on FreeBSD. I guess *BSD is more common in environments where you want to prevent end users from running programs that they themselves developed.
The tendency to deploy software on a leased server, called "cloud computing" by some and "service as a software substitute" by others, is why the GNU Affero General Public License (AGPLv3) was created. All AGPLv3 software must be a quine.
In my opinion, the shift toward single-CD images and DVD/USB images that could fit on a mini-DVD are consequences of about three factors.
The first is wanting to run a live CD to see whether it has drivers for a particular piece of hardware before buying the hardware. PCs are a lot more variable than, say, PlayStation game consoles. A lot of PCs are made to work only with Windows, and exercising the hardware in X11/Linux helps you avoid those PCs.
The second is changes in home Internet connections. Shipping an entire repository on four to seven CDs made sense in the dial-up era when people would buy or trade home-burned discs at user groups. But nowadays, broadband and forums have replaced dial-up and in-person user groups, yet many home and mobile ISPs still impose caps to discourage their subscribers from using their burst capacity all at the same time. So people want to download something working at home without having to pay overages to the ISP. Burning the image is also a problem, as repositories of free software long ago exceeded the 8 GiB capacity of a dual-layer DVD, and BD-R drives never became a standard feature on PCs.
And the third is updates. During the era of multi-CD Linux distributions, there wasn't nearly as much awareness of Internet-facing desktop applications and libraries that can be exploited as a vector through which to infect a PC. If an application is likely to be a vector through which to infect a computer, it needs to be updated often. So why ship an application on the CD if it'll just get updated later? It uses just as much Internet bandwidth to ship a fully updated application the first time as it does to ship an updated application immediately after installation, so they might as well be distributed in fully updated form over the Internet through the same mechanism that updates use.
Ah the good ol` days where you could have "which window manager do you want" as an option in the installer rather than as a completely separate distribution.
The Debian 8 installer includes a desktop environment selector. Or with Ubuntu, you can install the server distribution and then sudo apt-get install xubuntu-desktop later.
There is a secret...
Linux was the first free kernel that worked well - except the BSD's, which were being attacked through the legal system.
Linux also eschewed having a 'ports' system and instead decided to contribute ports back upstream. Relatedly, when common software failed to build on Linux, it was popular to change linux to be compatible instead of changing the software. This resulted in linux becoming ectremely compatible.
Yup. The cool name and the penguin. Never underestimate the power of marketing.