Slashdot Mirror

And a Mac Pro 8-core can run at 80 Gigaflops (that's about $5.5k), or $68/Gflop. It stands to reason that a manufacturer could beet them, this project isn't that ambitious.

Geezuus, it's not like OpenMOSIX is unusable as is, or that there aren't alternatives. For that matter, while coding one's own cluster controller isn't trivial, it isn't string theory either. Our shop has released (eg. given away) two schedulers, and we've got another that's stayed in house. When I've strolled the booths at the SuperComputing conference, it seems that every other university is giving away their own cluster controller.

OpenMOSIX is neat, but it ain't the end all be all, and it's been my experience that any shop that's serious about running a cluster manages to find/attract someone with the chops to get it up and running. Can just any elementary school pull one together for "free"? Maybe not. For them, there's Pooch or AppleSeed.

Second, we've successfully recompiled scads of old ( and not-so-old ) fortran code for OS X already. I guess the programs we had didn't need to much in terms of special support- we used g77. It's good to hear GCC is ( of course ) keeping up with the times.

A quick google search and a peak at the ADC website and it does look like GCC and the Macintosh are being used by the HPC folks. Altivec libraries available and everything, pretty much like you'd hope for, if FORTRAN is your bag.

learly, something monumental must be going on in the world of computing for these technology titans simultaneously to discover something that is so profound and yet so hard to name.'"

Absolutely. But I don't see large scale distributed computing or "utility computing" working in the public domain for more than a few conceptually cohesive projects (think SETI and Folding@home for publicly available projects). On the other hand individual companies could certainly take advantage of this concept for internal projects while harnessing the computing power that many of them already have in abundance. The problem is bringing all of this computing power (desktop systems) together easily and without hassle. Software like Pooch and Xgrid are decidedly the way to go here allowing companies to harness space CPU cycles for anything from rendering to bioinformatics to modeling airflow or turbulence. For instance, how many computers are at organizations like Lockheed Martin? Or Genentech? Or at most Universities?

Virginia Tech put together a spectacular number 3 ranked supercomputer for a (comparative) pittance in a (comparative) heartbeat. They did it with Apple's latest/greatest. Is it surprising that Apple wants this story told? I'm just shocked that they aren't filling the airwaves with the story (at the very least on every news program that PHBs watch).

This is especially true given that an equivalent setup could now be put in place in a fraction of the space required by Virginia Tech's setup. The 1U Xserves are decidedly the way to go here. In addition, with Xgrid (and before that, Pooch) it is now possible to have clusters configured from existing machines on peoples desktops that are recruited into the supercomputer cluster.

Anyone here seen POOCH? Been around for, what, the last year and offering the same thing? Now XGrid is big news?

Launch Den Mother and Launch Den Puppy are a distributed computing application pair for older Macs. You can download a 68k version despite their claims of requiring a Power Mac.

Its not as automagical as XGrid and you will have to write your own multiprocessing code, but it is a way to do distributed processing on old hardware. Clustering a bunch of LC520s may not be the best use of time and electricity, but then who said that a hack must be cost-effective.

You should check out Pooch from Dean Dauger. He has adopted OS X for obvious reasons but older versions of Pooch ran on MacOS 9.

Pooch was the original easy to use and implement solution for clustering Macs used originally at UCLA for physics modeling.

There's some software called Pooch that lets you run parallel clusters across PPC Macs. It's pretty minimal but you can still use the Macs for regular work and then run parallel software on them when they're not being used.

Done a long time ago (although not with hypertransport :) daugerresearch

Actually, I am seeing a number of folks either 1) migrate to or 2) seriously consider Apple's Xserve for purposes sort of in-between. The Xserve runs UNIX, it is absurdly easy to manage, they are cheap, and give pretty good performance especially when code is optimized for Altivec.

I've been on the lookout for this (and possibly webobjects uptake) and just haven't seen it... where I have seen xserve adoption has been in certain areas where macs would have been the preferred platform (ie, mac clients) but for many reasons they had to go with a higher-end unix or NT server as apple just didn't have much to offer for that market. There was a small market for people who wanted to use Apple servers but they just didn't meet their needs- so Apple saw a big spike in sales for them when they were released. Heck, some of these people use OS9 with webstar... I'll be more impressed if they can actually grow unit shipments quarter after quarter.

I mean... for 99% of the server tasks out there where are you going to see massive improvements due to altivec? You don't see apache getting big gains from using SSE on x86. Much more important to the xserve's large performance increase over past apple offerings was the new DDR bus, and the just as important new architecture with some mondo bridge chips for cutting the processors out of the equation as much as possible via DMA requests. This is because the bandwidth to the processor is very limited with the current machines in general, and in a dual config they have to share it... cutting them out of the process helped a lot.

That's why you hardly saw any improvement with the new DDR machines for things like photoshop over the past towers, as bandwidth but for server related tasks it helps sooo much.

Add to that the power consumption (or rather lack thereof), and for large numbers of servers, the Xserve becomes even more attractive in terms of lower electricity and cooling costs.

I'd be interested in knowing just what thermal savings there actually are in using an xserve over a competing x86/sparc 1U server. I've used them, and they are NOT cool running machines... they're very hot, and extremely noisy as a quick google will show.

People always think that the PPC is so much cooler than x86, and in general it is... but we aren't talking about 1/4 of the thermals here. Crack open a new P4 or AMD box and there is some big heatsink stuff going on there... kinda funny. Crack open a G4 quicksilver where Apple has been having to essentially overclock the processor and they're ungodly big also, and run really, really hot with huge fans that have made their customers pretty peeved. Just look at these pictures to get an idea of just how big the heatsinks are in new mac towers... and realize that the fans are very, very loud.

So we know that the current G4's are hot as hell, as are the pentiums and amd processors. Apple uses some monster chipsets as well, and it isn't as though apple uses different disk drives or memory... so where are the big thermal savings with the xserve? Companies make custom enclosures like this and this just to make them run cooler and quieter... I doubt people would spend the money on them if there simply wasn't a problem.

Now if you were talking about something like this... gotcha. But they're a whole different ball of wax.

Don't get me wrong- the xserve is cool, and a lot of the points you make about it are valid... as are things like this (largest xserve cluster i know of). But it isn't a miracle worker and it isn't a cool and quiet server.

The wild guesses about distributed computing are still a pipe dream, Rendez-vous or not.

LOL! Distributed computing with Rendezvous is already here! You should check out the Pooch Application (Parallel OperatiOn and Control Heuristic Application). It discovers clients using Rendezvous...

http://daugerresearch.com/pooch/whatis.html

Feature list:
http://daugerresearch.com/pooch/features.html

The wild guesses about distributed computing are still a pipe dream, Rendez-vous or not.

LOL! Distributed computing with Rendezvous is already here! You should check out the Pooch Application (Parallel OperatiOn and Control Heuristic Application). It discovers clients using Rendezvous...

http://daugerresearch.com/pooch/whatis.html

Feature list:
http://daugerresearch.com/pooch/features.html

Here is an interesting list of some mac clusters that use the Pooch application (Parallel OperatiOn and Control Heuristic Application)...

Could be of interest to some people here:

http://daugerresearch.com/pooch/users.html

Sorry, the URL I posted above seems to be broken... here is the correct URL:

http://www.daugerresearch.com/fractaldemos/JPLXSer ves/JPLXServeClusterBenchmark.html

I sincerely hope that soon small-to-medium enterprises can own supercomputers. With all the low budget physics stuff going on at Universities around the world, cheap supercomputing can only be a good thing.

Actually they can with software like that from Dauger Research, Project Appleseed and Wolfram Research with gridMathematica

The cool thing here is that this code can be run on all of the desktop computers that already occupy companies and universities world wide allowing for easy access to supercomputer level computational speed (for those problems that can be attacked using parallel computation of course) using the same computers normally used for productivity.

Very cool.

with a combined capacity equal to the 500 best of todays computers.

As reported a few days ago, these guys already have 33 of them!

They did. And it seems to be missing from the Top 500 list. According to this, 33 XServes reached 217 GFlops/sec. Now, according to Apple, they should be able to reach a much higher speed than this (roughly twice the performance they actually got), but part of the reason might be that they used 100BaseT instead of Gigabit, and theoretical != real world anyway. This earlier cluster of 76 G4's even acheived higher results. JPL found Macs to be "capable of excellent scalability in performance. "

Actually, Mac's are used in super computer clusters. JPL has an intresting benchmaark of 33 Xserves. They get 1/5th of a TeraFLOP of performance. Not bad, considering how cheap they are.

While I agree that such a configuration would not rival ASCI White, if he is in fact right that this would cost 1/20 of ASCI White, you could buy a whole lot of these things and network them with gigabit enet (or i hear that there will be support for networking over firewire in OSX soon...and speed increases in firewire).

Obviously this machine was not meant to compete with ASCI white, but for companies who would want apple for renderfarms, data processing, data storage, etc. It looks as if Apple is going after the digital video effects market or other people who will have lots of mac workstations (eg scientists) but also have data warehousing and mass data processing needs. this is a very nice first offering in the rack server market!

see these guys and this article for further applications of this new server.

http://www.daugerresearch.com/pooch/PoochManualX.1 .pdf

Well, Dauger Research is touting their 1 page manual and right they should. The simplicity in setting up this cluster is pretty amazing. The link is to a 46 PAGE technical document that goes into much greater detail. Still a couple of hundred pages shorter than the referenced Linux manual.

Now, if people would stop bashing Apple's documentation and realize that it is Dauger Research who wrote the documentation for Pooch, I'd be very appreciative.

adso writes 'Macintosh' and 'Cluster' aren't two words you see together very often. Some enterprising folks at USC have created a cluster of 76(!) dual-processing G4s (56 DP G4/533 + 20 DP G4/450). You can check the info here . Glad to see parallel computing isn't just for the *nix crowd (well, they are running OSX, so technically...). I wonder if they just had 76 G4s lying around, or else there must be some very upset department secretaries. "

The fact that a manual is shorter doesn't mean that it is a better or easier to install program.

I would agree that comparing manual lenght is not a reliable guide to judge the relative complexity of two programs. The one-page doc is even a "quick start guide" not a complete manual. But I still suspect that the writer is correct that Appleseed clusters are easier to set up and maintain than a Beowulf cluster. Reading over the directions myself it did looked pretty brain-dead simple - most of that one page didn't even have much to do with the actual installation of the program but with such complicated tasks as connecting your Mac to an ethernet hub: "For each Mac, plug one end of a cable to the Ethernet jack on the Mac and the other end to a port on the (ethernet) switch." and noting a few system requirments (CarbonLib 1.2 or OS X 10.1) The installation instructions consists of "Double-click the Pooch Installer and select a drive for installation." Instructions on how to use consist of dragging and dropping the program you want to run in parrallel onto the Pooch app and "click Select Nodes..., select the computers you want to run it on, and, in the Job Window, click on Launch Job."

Besides, if you are going to have a cluster, you want cheap, off the shelf machines such as PCs with plenty of spare parts that can be customised to suit your needs : why pay for a good 3d graphics card in every pc if you are going to do number crunching !

This is only the case if the individual PC's are dedicated nodes and not being used for anything else. Most Appleseed clusters are made up of computers that are primarily being used for something else. School Mac computer lab by day; clustered "supercomputer" by night. The cluster of that did 233 gigflops (76 dual G4's mostly 533's with a few 450's) was simply all of the Macs at UMC working as a cluster over Christmas break. This is where the easy set up, maintenance and the ability to cobble together computers with different processors and even different OS's (some nodes may be running MacOS 9 and some nodes may be running OS X) is an advantage. The Appleseed clusters that are made up of dedicated machines are probably discarded computers they already had kicking around so cost is not an issue there either.

Funny you should say that.. according to the more complete 45 page manual, it uses at least a subset of MPI.

Reading is fundamental.

"Before Pooch will accept commands from another Pooch, it must receive a passcode that matches its own." In other words, all of the Pooch systems need to be from the same install. Which, for the free download would be a max of 4.

Viewing the media (slashdot being just one example) is like watching a student eagerly tell the teacher and class an utterly uninformed and wrong answer to a question nobody asked. I don't mean to take this out on you pb, but my god. Pooch has a 1 page quick start guide. Good... its manual is 46 pages long. It is pretty obvious that Pooch is easier to set up, and for that may win the hearts of many researchers.

Personally, my imagination leads me to believe that the NSA has a few thousand or more mac cubes Pooched together in a subterranean cluster hive in Alaska translating satellite communication and doing linguistic analysis.

Now, if what I have outlined isn't possible, please let me know... :)

peace

Available from here

Uh, the article has the link to the one page PDF.

"'Macintosh' and 'Cluster' aren't two words you see together very often. Some enterprising folks at USC have created a cluster of 76(!) dual-processing G4s (56 DP G4/533 + 20 DP G4/450). You can check the info here . Glad to see parallel computing isn't just for the *nix crowd (well, they are running OSX, so technically...). I wonder if they just had 76 G4s lying around, or else there must be some very upset department secretaries. "

I didn't see any mention of the best Beowulf clutstering project of Macs, so to "Think Different": Project Appleseed, put together by the physics department at UCLA. They've accomplished phenomenal speeds, etc.; mostly, it's just as possible on Macs as it is on anything else.

In fact, they have even developed a drag-n-drop interface for setting up Beowulf jobs.

Take a look at that picture on the right. There are two monitors in the frame. Look at the right monitor. Look closely. If I'm not mistaken, that's OS X.

Default OS X background, Dock on the bottom of the screen. Oh yeah, they're using OS X. In a way, this makes sense. They say they're using Pooch, and Pooch is available on OS X.

Very very cool!