One-Machine Linux Cluster

An AC wrote: Forget Beowulf ^? clusters, Jacques Gelinas has made available a kernel patch to enable many virtual servers running on the same machine, even the same kernel. Read his original message posted to the Linux kernel list." Imagine what this will mean for hosting companies...

Comments

wat r the implications for mainframes?

[deadmantalking]

As far as i know... this was supposed to be one of the big wins for the mainframes... i recall some note about 44000 linuxes running together on a single IBM mainframe? sorry dont have the link handy...

Re:wat r the implications for mainframes?

[Buck2]

Interesting point ... will a box containing a processor named Athlon ever have as much IO as a z900?

Probably not, but you _will_ assuredly see a commodity item with a z900's IO in your lifetime (assuming you're of average /. age :)

Beaowulf not the target audience

[Genady]

This has just about zero to do with clustering, if anything this is the opposite of clustering. However this IS very very interesting for Web Hosts and just about anyone else that wants to create and maintain multiple environments for developement, test, etc. Image, being able to carve up a mid-range machine like you can an S390 (or other Mainframe class machine Like Sun's E10/15K). So suppose IBM takes this an runs with it. Linux is already ported to RS/6000 and AS/400, now you could get 8 processors of RS/6000 goodness, run production on 4 processors, Test on 2 processors, and Dev on 2 processors.

The devil will be in how you refresh test and dev from production, but that can probably be done inside Logical Volume Manager.

This is very very cool stuff it will be very ineresting to see how it stacks up against the big boys in Virtual machine space.

Re:Beaowulf not the target audience

[Doktor Memory]

now you could get 8 processors of RS/6000 goodness, run production on 4 processors, Test on 2 processors, and Dev on 2 processors.

What you're suggesting is pretty much the opposite of how this package works. As the author himself states, you cannot dedicate hardware resources to a vserver. Only one kernel is ever running, and you use all of your cpus or none. Process- and user-space isolation is provided, but if a process in one vserver tickles a kernel bug that crashes the system, the whole ball of wax will come down with that vserver. (Likewise, it's very likely that a kernel-level root exploit will allow you to break out of the vserver and attack the whole system.)

Essentially, vserver is to the process space what chroot is to the filesystem layer.

This is not inherantly better or worse than the "system partitioning" approach; it's just a different approach, and will have different uses.

Is it similar to ensim?

[aD docwolf]

Is it similar to this (commercial, closed source) package for redhat?

I believe this package is very popular with webhosts. One user can totally hose the machine, the rest are not impacted. Trust me, I know.

mosix

[morcheeba]

I wonder how this would work with mosix... it could be a dream system!

You could use mosix to combine the compute resources of several boxes to look like one box. And then, you could use this divy up the space so that people don't step on each other. When anyone (working in thier own space) kicks off a large compile, the load would transparently be distributed among all the boxen.

Of course, I have zippy experience with any of this, but it sounds possible.

Think about restricting services

[mendepie]

Think about a system where you want to use IP filter to control what a network host/ports a service (or the hacker that has cracked your service) accesses.

If it addresses many of the issues that normal chroot has, then it may be good.

Isolation of applications against each other.

It's going to be intresting to see how much overhead this has when compared to vmware, usermode linux, or just chroots. (Tried 'em all).

If the overhead of this is not higher than chroot then it will be a big win.

How might this impact Linux SMP scaling?

[Anonymous Coward]

OK, I don't have a clue what's really involved but I wonder if this could be significant for Linux SMP maybe moreso in time than for single cpu systems. I remember reading a couple of years back about an idea someone had for keeping Linux well optimized for 1 & 2 cpu systems , which is something Linus appears to care about, while also aquiring massive SMP scaling capabilities competitive with, say Solaris. The basic idea was to find a way to operate a multicpu machine as a collection of 2 or 4 cpu Linux systems. Mainframe-like virtualization within Linux I think is what he was proposing. Of course, proposing it and actually achieving an 64-way implementation that doesn't impose so much overhead as to squash the original purpose, namely keeping Linux fast on modest systems, are 2 different things.
This patch sounds somewhat similar. Uhh...is it at all similar, aside from the fact that it virtualizes in the other direction? Could it be turned inside out and extended in that direction ?

Security flaws

[BuGless]

Having these calls available to non-root opens up a can of worms. The system provided looks clean, except he should limit its execution with yet another capability.

Read this article commenting this patch

[lekter1]

There is an article (spanish only) commenting this kernel feature here:

http://www.hispacluster.org/modules.php?op=modload &name=Sections&file=index&req=viewarticle&artid=2.

In fact, this article was generated collecting the opinions of many users who post comments about this topic.

I hope it could give you some ideas about the implication of this important feature in the Linux future.

Re:*BSD Jail?

[InvisibleCraterFunk]

Interesting. FWIW, several of jail()'s shortcomings are going to be addressed in FreeBSD 5.0.

See jailNG

chroot safe?

[tal197]

The documentation says...

Unix and Linux have always had the chroot() system call. This call was used to trap a process into a sub-directory. After the system-call, the process is led to believe that the sub-directory is now the root directory. This system call can't be reversed. In fact, the only thing a process can do is trap itself further and further in the file-system (calling chroot() again).

And...

The vserver is trapped into a sub-directory of the main server and can't escape. This is done by the standard chroot() system call found on all Unix and Linux boxes.

But, I thought you couldn't (safely) run root processes in a chroot jail, because escape is easy if you can call chroot? Eg, create a subdirectory in your jail and chroot to that (keeping the same current directory), then chroot("../../../../") to get out of jail. Is it really safe to give someone the root password to a vserver in this system?

Advantages of Clusters

[ncon]

This is much like the jail() of BSD. This does not give any of the benefits of a clustering arrangement. That is, the benefit of having a cluster is that you can distribute process across multiple machines and run from a common storage server. Although this technology is very useful (and can be applied in all sorts of ways- We run Bind in a jail) it does not provide extra process space if only running on one machine.

Having sufficient RAM is the largest factor in commodity grade webhosting services, so having mutlitple instances of a cluster on the same machine does not really make sense, when the whole point of a cluster is to give faster computation and access time.

btw- we offer both of these services here, and we do it on FreeBSD.