Slashdot Mirror
Kernel Fork For Big Iron?
Boone^ writes: "ZDNet is running an article on the future of Linux when used on Big Iron. Just a bit ago we read about running Linux on a large scale Alpha box, and SGI wants NUMA support in Linux so it can support their hardware configuration. The article talks about how memory algorithms used with 256GB machines would hamper performance on 386s with 8MB ram. So far Linus et al have been rejecting kernel patches that provide solutions for Big Iron scaling problems. How soon before a Big Iron company forks the kernel?"
Just as importantly, forks are probably necessary when a significant part of the user/developer base disagrees with the direction of the project. This usually implies that the forked version and the original version are aiming at solving different problems within the same vein. If the original project wants to continue in the original direction and some people want to use the source to solve a slightly different project, then they pretty much have to fork in order for the project to achieve its maximal result of being most useful to the most people.
This isn't a bad thing if it's done right. It's just that most of the big forks you hear of are at least partially the result of bitter, angry wars (OpenBSD anyone?). You don't hear that much about the ones which are completely amicable.
What I would believe to happen for the big iron machines is that they would have a different directory for them and the memory management code will be under there. So now you have the memory management for big guys and the other memory management code under the same source tree. When you compile your kernel it looks for the proper managemnt code. I know not that simple there is more to it than that but is what will happen if they fork and come back together.
whyn do we need one huge kernel anyway? Probably several kernels are needed. One for big-ass servers, one for tiny-ass routers, one for mainstream workstations, and one TBD. Having one all encompasing kernel makes building the kernel a pain in the ass. I've been using linux for four years, and I still have to build my kernels a couple times before I get it right. So many freakin options, i'm bound to get something wrong.
but that's just my opioion...
-earl
One of the issues that people seem to fail to realize is that Linus is not necessarily rejectiung the patches because of what they do, but how they are implemented. If patch code is submitted to Linus and the patch is going to make mataining that system difficult (read messy unmaintainable code) Linus will reject it. Linus also does not like large patches either. He likes bits and peices and clean fixes. Hey he started this whole thing, I think he has that right.
Another thing to think of is that ZDNet is a news network. Everyone has been saying that the kernel will fork and blah blah. There are already forks in the kernel but people just don't realize this.
Redhat kernels: Have you ever tried to apply a patch to a stock redhat kernel? I know that since RH5.2 they ship the Linux kernel with there own patches.
SuSE kernels: Last SuSSE I installed (5.3) had both a stock Linux kernel and a custom SUSE kernel with custom SuSE patches.
Corel: never tries them but they patched kde and made it hard to compile other kde software with there distro.
Point? There are already forks in the Linux community, yet it goes on. That is the whole thing about open source. There can be forks. If an idea is good it gets into the mainstream kernel. But these 'forks' need to be tried first and become tested and cleand up in such a maner that they can exist with the rest of the linux kernel.
If you think that everyone is running P200 or P500 or GigHz machines you are wrong. I am sure that there are lots of people out there that are running old 386 / 486 with Linux as routers firewalls, etc. After all you do not need a superfast machine for a firewall if all you are going to firewall is 3 or 4 other machines.
I don't want a lot, I just want it all!
Flame away, I have a hose!
Only 'flamers' flame!
I am constantly putting Linux onto old hardware. Need a quick, dirty, and cheap NAT box? Throw Linux on a DX2/66.
MP3 file server for the geeks in IT? Throw in a big drive, but a 486 will do.
Hell, my company's web server is running on a low end PII, and I think it's a horrendous waste! It could be doing *so* much more.
Linux is a UNIX for cheap Intel hardware first. That's where its roots are, and I don't see why it should sacrifice its roots for big iron that can quite happily run a UNIX designed for big iron.
Neither does Linus, apparently.
Is this post not nifty? Sluggy Freelance. Worshi
So many things are distributed as kernel patches that it doesnt really matter. Anyone with that kind of hardware will obviously have the expertise and the money to install an appropriate kernel patch. No box that big is going to run an out-of-the-box kernel anyway, if you're using that sort of hardware, you're going to want to tweak it. As long as there is not a division in the majority of users' needs, there is not likely to be a major fork.
It depends on how pervasive the code changes have to be. If it involves #ifdeffing every single file, then it's going to be very difficult to maintain that, and it's going to be very unlikely that the maintainers of the project are going to allow that feature to remain part of the major distribution.
That problem is a dual-edged sword. It also means that maintaining one big patch is a complete nightmare. Every version of the kernel that comes out has to be separately patched, with two important considerations:
- The code which needs to be inserted has to be reinserted. If this is all separate files, that's easy, but if it's not that's a complete nightmare. And the code to call into that separate file is then a nightmare.
- Any changes which have broken the patch have to be investigated and possibly changed. If you're working on filesystem patches, for example, someone working on the core fs work may have broken your patch without your knowing it, because they're not including your code in their coding/debugging process. So every time there's a change to the kernel, you have to figure out whether that change will potentially break your work.
The only way to resolve the second is to keep the patch inside the actual kernel, so that the authors of the rest of the system are aware of it, and will either try their best not to break it, or will do first-round of changing the new functionality to work with their changes.Basically, it comes down to how pervasive the work has to be. If it's a really pervasive change which touches on almost everything, then the only option from a software engineering perspective is a fork. Anything else is being done from a feel-good PR perspective, because it just doesn't make any sense from a technical perspective to try to maintain a huge patch that covers everything.
Because now is not the time.
A great many features start out as independent kernel patches.
When thigns stabilize down the road, I'm sure they will gladly put 'Big Iron' flags in the compile stuff.
The point is, linus (et al) can't just stick everyting everyone submits, big OR small, into the main kernel, especially if it's not even developed yet!
Also... the feature set for current kernels is already listed... and this isn't one of htem.
You don't just add shit to a project partway through because someone wants you to.
I'm sure than by the time 2.5 kicks up, we'll see a 'big iron' flag in main kernel options.
This makes it sound like Linus has been rejecting them because they provide solutions for Big Iron scaling problems. Having read kernel traffic and the linux-kernel list enough, this statement looks immediately suspicious. I have never seen Linus ever purposely reject a patch that's an all-around good fix for a problem. Usually it's "Well, Linus rejected my patch even though it does all this cool stuff and fixes all these problems, so it's probably because he just doesn't like such-and-such feature/platform/interface" and then Linus replies, "no, I rejected them because you're a dumbass and your patch sucked."
The link to the SGI page somewhat confirms this:
I just kinda heavily doubt that Linus wouldn't want awesome NUMA support if the potential was there. My best bet is that the people pushing for it just aren't on exactly the same wavelength as Linus (is anyone?) and it's slowing down progress.
Another quote that points in this direction
I don't think Linus mean any solution, just the solutions presented to him.
It sounds inevitable that a Big Iron fork will occur, and as Linus says above, this is not necessarily a bad thing. The problem comes when you have competing factions trying to do the same thing and causing confusion (as in the UNIX wars of the past). But when you have different solutions for different problems, yet everyone is moving forward together overall, it should be manageable. Indeed, it should be helpful, for it maximizes the solution for each platform.
The biggest potential problem of forking an OS is binary and API incompatibility. The reason most people use computers is to run specific applications. I want to be able to walk into my local CompUSA/log on to Egghead and get a copy of application X and run it on my computer. I don't really care what the OS is, as long as it runs application X.
If I've got Linux on my system, I'd like all applications that run on Linux to run on my system. The more forks that introduce binary or API incompatibilites, the less chance I have of being able to run the apps I want, and the more reason I have for removing Linux from my computer.
If Linux wants to be a mainstream desktop OS, it needs to make sure it doesn't fork too much. That was a big part of the reason desktop UNIX failed to take off in the late 80's/early 90's.
Sailing over the event horizon
Yes, it is nice that it will still run on a 386, but there are other factors to consider:
1. Earlier platforms generally had no CD-ROM. Most Linux distros (except for fringe distros) come on CD-ROMs. Most people do not want to buy a CD-ROM for their 386, 486s. There are places that offer small "floppy-disk-sized" Linux distros, but they are obviously chopped. 1400K on a 500MB HDD.
2. Earlier machines usually had a 5 1/4" floppy disk, until the late 486s started really using 3.5" floppies. Most people are not going to spend money and time ripping out an old floppy.
3. Earlier machines had RAM limitations, aside from the fact that no one wants to really waste the money on putting more EDO memory into an obsolete machine.
4. Some earlier machines had fscked BIOSes, aside from Y2K-unfriendly BIOSes; Most people will not research whether the particular BIOS is okay to determine whether or not to spend money on the first three items.
5. Earlier machines had ISA, EISA, etc. Oh, what, you want to run GNU/Linux in something other than CGA?
6. Earlier network cards are not all supported to get around many of these limitations... I tried to get around not having a CD or a 3.5" floppy in an old 486 by using some sort of older ISA-based network card.
Obviously, there are many issues to consider before nodding one's head to allow Linus to try to preserve performance in ancient boxen for nostalgic purposes.
Lucas
--
Spindletop Blackbird, the GNU/Linux Cube.
The process of non-standard kernel patches is just fine with Torvalds. "On the whole we've actually tried to come up with compromises that most people can live with," he said. "It's fairly clear that at least early on you will see kernel patches for specific uses -- that's actually been going on forever, and it's just a sign of the fact that it takes a while to get to a solution that works for all the different cases." He continued:
"That's how things work in Open Source. If my taste ended up being the limiting factor for somebody, the whole point of Open Source would be gone."
It sounds inevitable that a Big Iron fork will occur, and as Linus says above, this is not necessarily a bad thing. The problem comes when you have competing factions trying to do the same thing and causing confusion (as in the UNIX wars of the past). But when you have different solutions for different problems, yet everyone is moving forward together overall, it should be manageable. Indeed, it should be helpful, for it maximizes the solution for each platform.
________________
________________
Private Essayist
Even in the cases where Linus has outright rejected BigIron patches, nothing stops a hardware vendor from patching the source after the fact - almost every major Linux distribution does this now for x86/ppc/sparc etc. (NFSv3 is a great example)
The other reason that we are scared of the monster systems patch is the number of Linux kernels that come out. How often do we recheck the patch? Which kernels do we release the patch officially for? How do we decide? There are no really good answers to any of those questions which is why the big patch is to be avoided if at all possible.
Go Badgers! -- #include "std/disclaimer.h"
I wouldn't be shocked if the stretching of boundaries that comes from:
- "Big Iron" changes, as well as
- Embedded System changes
winds up turning into there being some clear demands for forking.The fundamental problem with a fork comes in the code that you'd ideally like to be able to share between the systems. Device drivers float particularly to mind.
After a 2-way fork, it becomes necessary to port device drivers to both forks, which adds further work.
And if a given driver is only ported to one fork, and not the other, can it correctly be said that
or do we need to be forever vague about that?If you're not part of the solution, you're part of the precipitate.
Read KT. Read KT often.
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
Of course then you would have to ensure that both offer a similar interface so that either can be used transparently. This *could* be a maintainence nightmare. I think there are a lot of ways that this *could* be done, but it depends greatly on the details involved whether it will be practical or not. I find it hard to believe that Linus would have looked over something as obvious as ifdefs or makefile tricks, so he's probably used his (undoubtedly god-like) judgement to decide that it would be a bad idea in the long run.
Forks are usually justified only if the original maintainer pollutes the source with hacks or changes the license.
Why are the patches being rejected? Couldn't they be conditionally compiled in?
There is a patch out there that stores the computer's state to disk before shutdown and then give you an instant boot. My home machine is used that much, and my UPS needs repair. This patch would be useful for me, but I'd have to patch it in by hand and then I'd be out of sync with the official Mandrake kernel. That means I'd have to patch in security update by hand.
The problem is, this patch is as useless to Big Iron as support for 256GB of memory is to me (right now). But why can't both Big Blue and I have our way with conditional compiles? All it would take are a couple of more menu selections in xconfig.
Do you have more that 2G of memory?
Would you like instant-on?
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
Perhaps the time has come to fork the older machines.. Few of us run Linux on anything less powerful than a Pentium, and even fewer on a 486.
I don't know, it depends on where the split of cost/benefit falls.. ZD doesn't say...
`Sides, having a Compaq/SGI/IBM 'approved' kernel patch doesn't hurt much..
.sig: Now legally binding!
I'm not claiming to be a kernel expert, but forking the kernel so that there would be kernels specialized for specific applications only seems logical. A builder doesn't go around hammering everything in site because the hammer obviously isn't the correct tool in every situation. It's great for pounding nails into 2x4s, but isn't so good when it comes to painting walls.
Specialized kernels are good, so long as the support behind all of these kernels remains great enough. I don't think I need to point out the possible pitfalls of forking the kernel and thus, effectively forking the developers behind the kernel into two or more camps. But at some point, the linux kernel that runs on a 386 should be different than the one that runs on the XYZ super computer, just because it can take full advantage of all the wonderful scaleability that the XYZ super computer offers.
Anyway, as I said I'm not an expert but this just seems logical.
There's kernel "forks" for hard (deterministic) real-time (RT-Linux, etc.). There's kernel "forks" for non-MMU machines (ELKS, uCLinux, etc...). So, why not a "fork" for big iron? If the fork for big iron doesn't hinder current modern machines or improves overall operation- it will become the main fork with the one that just supports the older machines becoming like the other "forks" we see today.
I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
Okay, first there were systems, and the were all different.
Then someone "abstracted" them with "BIOS"
Then there were lines of systems, and they were all different.
Then someone "abstracted" them with "C"
Then there were platforms, and they were all different.
Someone (Transmeta?) will come up with a way of abstracting platforms (or architectures) and
make them "seem" the same.
This relates directly with performance increases. When you find yourself wondering what is going to make a 10GHz system better than a 1GHz system I think the answer is the level of abstraction.
Any number of quibbles can be made with the above statements, but I am illustrating a point, not
being a historian.
-Peter
This was bound to happen sooner or later. The Linux kernel's flexibility is being taken to the limit, and people are forgetting the easiest way to improve performance for their particular rig: Customize your kernel! You can add all the code in the universe, and then you pick and choose the particular things you need or don't need! Say I run a 486/25 with 16 MB RAM as an IP Masq router. The hard drive is an old IDE with 600 megs of space. I have two network cards, and that's about it. Do I need SCSI support? Do I need to support joysticks, X, Pentiums, AX.25, or anything else? No! I compile a kernel specifically to run the IP Masq, and run it well. My P100 laptop, on the other hands needs a bit more. I use it for packet, so I need AX.25. It uses PCMCIA, so PCMCIA support needs to go in. I use XWS to run Netscape and the GIMP, so I need graphics. But, my HD is not SCSI. I yank out SCSI. My CPU is subject to the 0xf00f bug, so that gets included. I brew a custom kernel, and boot time is a lot shorter. My big-rig is a C433. I need just about everything, as I have a 3dfx card for Quake3; XWS; a SCSI scanner; and a connection to my Packet base station. I optimize compilation for the higher-end computers. I plan on getting a Cube from Apple and putting SuSE on it. Again, by optimizing the options I optimize my system. Get the point? If you want a once-size-fits-all kernel, use Windows. If you want a kernel which can be adjusted for your particular and peculiar environment, use Linux and customize your kernel! Now, for my laptop.
I used to be someone else. Now I'm someone better.
Real life is underrated.