World's Fastest Macintosh Cluster

gabeman-o writes: "The Grupo de Lasers e Plasmas has created the fastest Apple G4 cluster. The cluster runs on 16 Dual PowerPC G4/450, 32 processors, 12GB of RAM, .5TB of space, and Mac OS 9. Apparently, they have utilized the AppleSeed technology developed by UCLA. According to the website, the cluster will be used for simulating plasmas. Not too shabby!"

Comments

Re:The worlds prettiest cluster

[SoupIsGood Food]

> Wouldn't it be cheaper to do this with a bunch of PC's in stead?
> Not as pretty but it's someones tax dollars that pay for these...

Fact of life: hardware is cheap. A $3,000 Mac vs. a $2000 PC is peanuts, even if the total difference is 30-40k for the whole installation.

Re-tooling the development environment, retraining IT and programming personell, porting code to the new platform and the cost to set up and configure the new and cheaper hardware far outweighs the cost of the hardware itself. Then there's the time lost in accomplishing all of the above.

If your admins know Mac, and your coders and QA staff are used to building and testing Mac code with Mac tools (MPW, Codewarrior, Macsbug), you go with Mac. End of story.

SoupIsGood Food

.. parent makes good points

[victim]

Sorry, I got ranty on this. To summarize, the parent article makes some good points, the previous replies are too extreme in their criticism. I supply better anecdotal evidence than them. :-)

For CFD the altivec units will be heavily used, it is correct to use them for comparing speed to PIIIs. There may be a case where it matches 50 dual PIIs, but that sounds extreme.

I just bought three lowend Gateway 2U rackmount boxes. They come in at ~$2200 for a single 800MHz PIII which will be slower than the dual 533MHz mac at ~$2500. That certainly meets `almost comparable'.

Neither of these machines are very cheap computers. There definately is a quality problem when you get into very cheap computers. There are applications for them, but if your diagnostic time is worth much, then I believe it pays to buy better hardware. (Plus you don't bleed when you have to open them up and add memory or drives. Once you get above the nasty low end machines they take time to deburr the stampings. My brother runs a metal working factory and their worst injury this year has been an IT guy that stumbled while holding a cheap PC and sliced several fingers to the bone.)

The new Macs do come with gigabit ethernet (although I don't think those old 450s had gigabit) and an OS. That gateway price is a bare machine. I put Linux on `for free', but the 2.4 kernel series (which I needed for iptables) had bugs in the interrupt routing for the chipset and it took me weeks of effort to get it all worked out into a stable configuration (manifested as a AIC-7xxx problem, took a while to find the interrupt controller problem). I knew I should have bought a 4th scratch box.

Gigabit ethernet hubs are still $250/port, but that will come down quickly. I remember when I bought a Powermac and thought it was just plain silly to put a 10/100 adapter in it, only servers could afford to have 100mbit ports. In 12 months all my new hubs where 100mbit.

I've got loads of Apples and loads of PCs. The apple hardware failure rate is less than half our PC hardware failure rate. Depending on the cost of a failure (in terms of ruined work, lost work, diagnosis, and repair) I do lean toward Macs because of their better reliability. (And before we get into a flame war on your reliable PC... I'm sure there are reliable models of PCs, but I need a vendor I can count on to make every model I might buy reliable. I can't look at historical data and say "look, here is a model that was made two years ago that was a good one" because I can't buy that anymore. (And yes, I own a PB5300, I know the counter argument, but Apple fixes it for free whenever it breaks so thats not so bad.))

Re:.. parent makes good points

[2nd Post!]

How about time?
Say it takes N people 1 day to set up the Appleseed vs the same N people 1 week to set up the Linux array?

What's 7 days worth of time? $300*N a day? $4k*N a day? While price is a legitamate concern, so is time to ramp and operating costs, and from what I've seen, the costs of an engineer and of support staff dwarf the costs of a PC; what's $800 when your engineer makes that much in a day? So 3 days where the array is not spent computing is 3 days where you're paying for no work done.

Geek dating!

Re:One day to set up

[marcsiry]

Although Mac OS9 doesn't natively support dual processors, application support for duals can be added via a system extension.

Several Mac OS9 apps are specifically coded to use the second processor... Adobe AfterEffects, Adobe Photoshop, Cinema 4D, and other graphically intensive apps. Users of those applications appreciate all the cycles they can get.

Since the second processor is dedicated to the single application using it at the time, you usually see pretty high effeciencies for a kludge-- 190% speed increases in those specific apps.

Apple has flirted with duals since 1996's 9600 MP, but it wasn't until OSX that they really made sense.

You'd have to set up a case factory...

[Gorimek]

...since you can't buy these machines in any other casings that tower or cube. I doubt that it pays off for 64 machines.

How much extra does the case cost? It's just a few pieces of cheap plastic. And the extra features (handles & door) makes it a lot easier to work with than the standard beige box.

I see a lot of posts complaining about the cases, of all things. I don't think this crowd is really upset about the cost. I think it's that they just can't stand functional things being beautiful. And that's very sad.

Not suprising...

[nutty]

When I competed in a sceince fair back in March, amoung other awards I won the "Princeton Plasma Physics Award", an award sponsored by the Princeton Plasma Physics Laboratory (PPPL) here in New Jersey, and as a winner I was given a free tour of the facilities. For those of you who don't know, PPPL is "The hottest place in the Universe", lying at the forefront of Nuclear Power and Plasma research...very cool.

So I was going around the facilites, visiting their $125,000,000 tokamac's and torsotrons and all this crazy equipment (very cool science plamsa physics is, too much to elaborate on here), and I get to the control room, from which they run all their Data Acquisition (DaQ) and such to monitor the expirements, and the room is filled with .... MACS? The engineer giving me the tour explained that it was in the personal interest of most of the researchers. Yes, there were Sun's and other UNIX boxes scattered on the control room floor, but I would look closely, and sure enough, amounst the three or so monitors at each workstation, one of them was hooked up to a mac. There were g4's and g3's scattered all across the floor. Wack.

So yea, Mac's are playing a key role in plasma research, helping achieve effecient fusion, one step at a time.

For another cool plamsa physics project (unrelated to mac's), check out Garrett Young's ISEF project Quasi-Elliptical Torsatron - A Study of Induced Radial Electric Fields and Plasma Turbulence. He is a senior in high school and on the cutting edge of plasma physics research. Quite the talented individual.

Apparently

[evilviper]

It's not that they're trying to set a record, just getting enough power to try and run OS X

---=-=-=-=-=-=---

Document Mirror

[evilviper]

Well after 5 hits, the server is barely responsive, so here's the page before the server completely dies:

Researchers at the Grupo de Lasers e Plasmas (GoLP) achieved the first milestone of the GoLP simulation program on Extreme Plasma Physics: the installation of the first Macintosh G4 cluster in Europe, called epp (or ep2), which is based on the AppleSeed paradigm developed at UCLA by Viktor Decyk et al. The epp cluster is capable of delivering over 50 GFlops of peak power, and it is based on 16 Dual PowerPC G4/450, 32 processors, 12 Gigabytes of RAM, 0.5 Terabyte of hard disk space, running Mac OS 9, over 100 Mb/s Fast Ethernet, switched by one Asanté Intracore 8000. This is the fastest Macintosh-based cluster in the World. The installation and set up of this cluster took less than 1 day (including moving the machines to the computer room, unpacking the machines, and making all the cables!), and it did not require previous knowledge of networking: a one-page recipe for Mac OS clusters can be found here (AppleSeed website) (Portuguese translation coming soon). This "supercomputer for the rest of us" will be used for the numerical simulation of plasmas, novel plasma particle acceleration schemes using ultra intense lasers, and relativistic shocks in astrophysics. This work supported by the Fundação para a Ciência e Tecnologia. More info available soon (also in Portuguese), as well as science using epp. For more information also contact Luís O.Silva(+351 21 8419 336).

---=-=-=-=-=-=---

Re:Propaganda control

[connorbd]

I rather like your statement about old-line Mac users, though we Mac folks are inclined to consider IBM the good guys these days :-)

Those AppleSeeds are actually pretty nice hardware; I've read the technical report on the original and it's actually a better design than a Beowulf. MacOS Classic's cooperative multitasking is actually a plus in this arena as it means that your computation code can completely take over the system and grind away. If you really want to have some fun, though, write a lightweight bootloader that emulates just enough MacOS and MPI to get the job done; you don't need the GUI overhead for a cluster anyway.

I would make two changes to the design, though: I'd use OS X as a front-end system (since you're not doing much computation on the coordinating server, I'm sure) and I'd start calling the clusters Apple *Orchards*...

/Brian

WTF is with all the Cray comparisons?

[AndyChrist]

On the Appleseed page, there are graphs in which they usually include the Cray T3E 900...including "single processor performance." I'm sorry, but...huh? When have you ever heard of a T3E with just ONE processor? Or 4 or 8 for that matter?

Okay, I get that they're trying to make it look better by comparing it with something completely out of it's league, to turn people who would just be using a bit of time on some big iron to having their own personal power playground...but lets be more fair, and see some stats comparing 100 Mac clusters to 100 CPU Crays. More realistic.

Re:The worlds prettiest cluster

[iphayd]

>Wouldn't it be cheaper to do this with a bunch of
>PC's in stead? Not as pretty but it's someones
>tax dollars that pay for these...

This myth has gone on enough. Every TCO study around shows that Macs are cheaper than PCs. Most of the TCO of a computer is the support time and training, NOT the purchase price of the hardware/software.

Macs require less support staff, for more computers. If I recall correctly, Intel has one IT person for every 30 computers. Except in their graphics dept, where they have 1 support person for all 300 Macs.

Because the Mac's interface is more standard across most applications, when you train on one app, that training can be utilized in other apps. This means that you get better training than if you have to train for every app.

It was recently figured that the difference averages between $400-1000 per machine, per year.

If you want to save your company $, you'd switch to Macs, but noone will do that, because it would cost (at least) half of your IT support their jobs.

Re:Some cluck at MacNN

[grammar nazi]

Did you notice the following statement at the bottom:

Note: To build a Beowulf, a Linux-based cluster, we think the following 230-page book is an excellent introduction: T. L. Sterling, J. Salmon, D. J. Becker, and D. F. Savarese,How to Build a Beowulf, [MIT Press, Cambridge, MA, USA, 1999

I think they are making fun of the difficulty of building a Beowulf cluster since they refer to the 250 page book at the end of their single page OS 9 clustering guide.

Fast Ethernet?

[Ando[evilmedic]]

Why the hell they're using Fast Ethernet is beyond me. If you're going to shell out the money for 16 dual G4s, why wouldn't you spend the extra money and install gigabit networking? Or does 100Mbps suit the purpose fine.

- Ando
You are the weakest link, goodbye.

Instead of calling it a cluster...

[glebite]

What about calling it an orchard?

The worlds prettiest cluster

[KarmaPolice]

Wouldn't it be cheaper to do this with a bunch of PC's in stead? Not as pretty but it's someones tax dollars that pay for these...

Some cluck at MacNN

[3G]

"While I think this is a sweet application of technology, it continuously galls me to see the misuse of the term cluster when applied to distributed parallel computing arrays such as AppleSeed or Beowulf. These are not clusters! There is no distributed lock manager present, no hardware-level device sharing (as opposed to software-level file sharing a la AFT, SMB, or NFS), no means of transparently starting, controlling, and stopping any process (not just those written to a custom API) on any node from any other node. While an excellant and usefull technology, this has a long way to go before it could be considered a true cluster..."

Re:Some cluck at MacNN

[glenmark]

I am said "cluck" (odd term).

I don't know about old (I'm 33), but I am a bitter VMS system manager. As I've repeated on this forum many times before (just about any time a story about Beowulf or AppleSeed is posted), that bitterness stems from usage of the word "cluster" in contexts which fall far afield from the original usage of the term by the inventors of clustering, DIGITAL's VMS Engineering team, thus diluting the meaning of the term. The casual application of the term to distributed parallel computation arrays, web server farms, or other RAICs (Redundant Arrays of Inexpensive Computers) incorectly leads many computing neophytes to believe that simply setting up a Beowulf array (much better term for this than cluster) or something similar gives them the same level of technological functionality as that provided by a state-of-the-art OpenVMS cluster. Nothing could be further from the truth!

"What's missing?", one might ask. Here is what I feel constitutes a cluster. (Note -- none of this is meant as flame-bait - just trying to rectify some misinformation which has become pervasive over the last few years. I regard Beowulf and AppleSeed as useful tools. I simply feel that it is inappropriate to refer to them as clusters.)

• A Distributed Lock Manager, critical for much clustering functionality. All file operations, regardless of whether using standard C I/O routines or OS-specific I/O APIs, should automatically and transparently be mediated by the DLM. Without this functionality, one is left with a so-called "share-nothing" cluster, such as that available from Microsoft, thus limiting one to simple failover operation.
• Cluster-wide File System. No file server such as NFS, SMB, or AppleShare involved here. All nodes in the cluster should be able to directly access any disk or tape drive in the cluster. The DLM is critical here for mediating simultaneous accesses.
• Cluster-wide process control, either directly spawing processes on other nodes or through a batch queue system.
• A Connection Manager to ensure that nodes correctly enter or leave the cluster. The CM manages the Quorum mechanism and keeps cluster transition times to a minimum.
• Shared System Disk. Multiple nodes should be able to boot from a common system disk. This greatly simplifies management.
• Single security and management domain. NT almost got that right, except for the dependance upon domain controllers and that domain database replication nonsense. Many flavors of *nix can be configured to get into the ballpark on this with add-on tools. If one can't transparently manage any node in the cluster from any other node, you don't have a cluster (and I'm not talking about something as trivial as spawning a new Telnet session).
• Cluster-wide Process Control API. At least Beowulf and AppleSeed provide this much, which is a step in the right direction, although the API used is a bit primative.
• Mixed Architecture Support. Shouldn't matter what hardware a given node is running on, as long as it is supported by the OS and the applications. All cluster communications protocols should be platform-neutral. Just out of curiosity, can one mix ix86, Alpha, and PPC nodes in a Beowulf array, thereby giving one the ability to divy up processing chores based upon which processor is best suited to a given task (Floating Point vs. Integer speeds for instance) or for gradual migration of the array to a different platform? (This isn't flambait -- I'm honestly curious.)
• Rolling Upgrade Support. It should be possible to configure a cluster such that one can perform OS or application upgrades some number of nodes at a time, thus negating the need to bring down the entire cluster.
• Parallel I/O support. All nodes can issue I/O requests to storage devices at the same time. This ties back in with the DLM and Cluster-wide file system.
• Interconnect failover. All cluster communications traffic should route itself through whatever interconnect pathway has the lowest latency. (Hmmm, FiberChannel is a bit congested right now. Time to route through the 100baseT connection.)
• High-end scaling. This is where I snicker at NT's so-called clustering solution. Two nodes only? Hah? (That is finally improving, but they still have a long way to go.) Beowulf and AppleSeed do okay in this department, although they have nothing that compares to the Galaxy Software Architecture.
• Load Balancing. Generically available to most server operating systems these days, although usually through a separate hardware based unit.
• Cluster Alias. Can be faked for any OS with DNS tricks or by sticking the cluster or array behind a NAT router.

Re:I don't understand...

[3G]

The G4s that Apple sells don't come in a rackmount config, but you can put them into one pretty easily.