Slashdot Mirror

I don't know if it's a single system image, but it could be: http://www.openssi.org/

The oldest being "single system image" clusters. DEC/Compaq/HP had an SSI project for Linux a while back, see openssi.org.

There's also a number of platforms such as 3Tera's AppLogic that abstracts away the invididual servers using virtualization. AppLogic looks fantastic in many respects - you get a nice GUI to connect together "components" to applications that can span hundreds of CPU's, and you can then deploy applications on a "grid" of host nodes and have their system provision everything, or you can snapshot an application or create a scaled down copy for development/testing etc.

It's great technology, and actually not too badly priced (prices for hosted AppLogic grids are roughly the same as for managed hosting of similarly spec'ed servers from some of the providers I've looked at).

You'll see a lot of these spring up over the next few years - deploying large scale apps "manually" is a real pain, and most successful companies that have to scale end up reinventing a lot of operational infrastructure. Systems such as AppLogic finally start making it possible to outsource or purchase a lot of that stuff and spend your inhouse resources on your core competencies instead.

Where you'll really see this start to shine is when people start cycling hardware. Cycling hardware to keep using the most cost effective platform is damn resource intensive (time/manpower) if your operational infrastructure isn't automated. Virtualization is a godsend for that. If I can constantly pick the most cost effective hardware and abstract it away completely from our apps, and take down or add hardware nodes transparently, then that'll save me a huge amount of time and money.

For number 3 (automatic clustering) try these:

http://openmosix.sourceforge.net/
http://openssi.org/
http://www.kerrighed.org/

All of these systems will let processes migrate between networked GNU/Linux machines.

Do yourself a favour and contact the team at Bullet Proof Linux http://www.bplinux.com/.

We needed a cluster for load balancing a typical web application, with OpenSSI http://www.openssi.org/ being the chosen SSI system. Sadly, OpenSSI is far from a "working" solution, and needs quite a bit of massaging - especially if you have newer equipment.

The guys at Bullet Proof Linux have been professional, helpful and incredibly patient in their efforts to get us going. Worth every cent.

Any they did it all remotely via ssh from the other side of the planet...

OpenSSI seems to be a good solution also

Here's the plan:
1. Set up High Availability router with pfsync. (using computers rescued from the trash)
2. Set up a HA Network RAID system using DRBD or something similar. (using more computers rescued from the trash)
3. Build a Kerrighed or OpenSSI Single System Image cluster. (using the latest and greatest computers one can rescue from the trash)
4. ???
5. Profit! (and thus, have enough money to actually buy equipment)

I've already set aside Tuesday evening to upgrade my bandwidth throttling OpenBSD router. I set it up the day before 3.6 came out, so I didn't feel like upgrading until now. I'm tired of the typical hardware failures you tend to get out of computers people throw out (maybe that's why they threw them out in the first place) but mostly I'm looking forward to getting a learning experience hundreds of times more valuable (personally) than getting my MCSE 2003.

\/\/\/

I'm no clustering guru, but I tried OpenMosix and wasnt impressed with the performance at all. While I havent yet tried OpenSSI, I see time and time again that it is far superior to OpenMosix and accomplishes the same objective (SMP emulation over network).

If you're into clustering, consider trying openssi as well as OpenMosix

I'm no clustering guru, but I tried OpenMosix and wasnt impressed with the performance at all. While I havent yet tried OpenSSI, I see time and time again that it is far superior to OpenMosix and accomplishes the same objective (SMP emulation over network).

If you're into clustering, consider trying openssi.

All my "corporate desktops" are running as a cluster 24/7.

Initialy the idea was just to simplify maintenance, but doing a make -j 128 kernel_image is quite fun.

Big Iron:

BigTux Shows Linux Scales To 64-Way

My current test system has 16 CPUS:

zeus0:~ # tail -15 /proc/cpuinfo
processor : 15
vendor : GenuineIntel
arch : IA-64
family : Itanium 2

(yes, it is Itanium!! Anyone got a 16-way Opteron box? Anyone? Buhler? I thought not...)

And, of course, we all know about Linux clustering:

Beowulf Clusters
Single System Image Clusters for Linux

Ignoring the oddity of Oracle being in that group, none of the rest of the members actually make a scaleable Linux box, just ones that compete with them. The slant is obvious.

- Necron69

It's simply not true that "a transparently cluster-capable system implementing native SSI" is "something that no other operating system can do today." We were doing it at Locus in 1994 with SVR4 then with Tandem in 1996 with NonStop Clusters for Unixware. Now some of the same folks at HP have introduced OpenSSI [openssi.org], which is essentially the same code, less all the Unixware-related bits, ported to GNAA/Linux and placed under the GPL. They are coming up hard on their 1.0 release, which is not bad for five people and such a large task.

OpenSSI is the real thing, it has processes that migrate from node to node, distributed file systems, the works. And it's running now on clusters literally all over the world. (Not many clusters, true, but maybe that will change if the Slashdot crowd finds out about it.)

I'm happy to say that there's a lot of my code in that system, as well.

I know a little about what Matt wants to do with his SSI in Dragonfly, but he should certainly take a look at OpenSSI; we had to solve a lot of the problems you run into when you build such a beast.

(And a beast it is. As complex as a kernel can be, when you have what is essentially a distributed kernel across several nodes, the complexity goes up by orders of magnitude. Makes tracking down those weird hangs pretty exciting, in a painful, time-consuming kind of way.)

OpenSSI (Single System Image) Clusters for Linux

The main features are: single root and single init, cluster filesystems and DLM, single process space and process migration, load leveling, single and shared IPC space, device space and networking space, and single management space

sounds like they are combining OpenSSI and DRBL.

Don't forget OpenSSI as well. This is a single-system-image clustering product with a long lifespan and great support within HP.

And in doing so you are essentially building a super computer. However you'd have to keep in mind that it isn't all about total bandwidth - latency also needs to be extremely low. That said, HP is working on an open source Single System Image clustering support for Linux on "normal" hardware

List of applications you want to protect

Budgeted amount

What threats you are trying to protect from

What kind of failures you will tolerate(do you need 99.9% uptime? or better? worse?
You could, for simple applications, like web service, bump up a pair of linux machines, gimmick some replication between the two, and hope nothing goes wrong, if you have a very low budget, and you'd probably spend a fair amount of work debugging later on, "synchronisation problems". But for redundant storage. The openssi project is working on highly-available single-image clusters for linux, in an open source model, they might be your first place to look. It's not however, something for the unprepared to do, nor is it something that I'd recommend if you do other tasks for this company. Maintaining such a beast will require a significant implantation investment. The good news is that once everything works to your satisfaction, you can probably take a 4 week vacation somewhere with golden beaches and much sun, and let it take care of itself. I can't stress this enough, this is a hard problem, if you really want to do this right, you'll want to surround yourself with qualified people with experience in this field, it's non-trivial, and mistakes can lead to severe data-loss.

How does this compare to OpenSSI? OPenSSI is nice because of the single system image approach, that makes administration very simple. AFAIK, an OpenSSI cluster also supports PVM and MPI in addition to exec and run-time load balancing (a'la mosix).

OpenSSI has a lot of "HA-" support, including support for various clustered filesystems, failover of network interfaces across nodes, and failover of the first node (hopefully soon without needing shared SCSI storage but using something like drbd).

It's simply not true that "a transparently cluster-capable system implementing native SSI" is "something that no other operating system can do today." We were doing it at Locus in 1994 with SVR4 then with Tandem in 1996 with NonStop Clusters for Unixware. Now some of the same folks at HP have introduced OpenSSI, which is essentially the same code, less all the Unixware-related bits, ported to Linux and placed under the GPL. They are coming up hard on their 1.0 release, which is not bad for five people and such a large task.

OpenSSI is the real thing, it has processes that migrate from node to node, distributed file systems, the works. And it's running now on clusters literally all over the world. (Not many clusters, true, but maybe that will change if the Slashdot crowd finds out about it.)

I'm happy to say that there's a lot of my code in that system, as well.

I know a little about what Matt wants to do with his SSI in Dragonfly, but he should certainly take a look at OpenSSI; we had to solve a lot of the problems you run into when you build such a beast.

(And a beast it is. As complex as a kernel can be, when you have what is essentially a distributed kernel across several nodes, the complexity goes up by orders of magnitude. Makes tracking down those weird hangs pretty exciting, in a painful, time-consuming kind of way.)

Maybe it wasn't originally delivered as a single system image, but they could still install this on it instead of beowulf/pvm/mpi/mosix and then it becomes a single system image cluster.

Here is a free single system image clustering system for Linux. IIRC it was HP that helped that project get a jumpstart.