Slashdot Mirror
Xgrid Clustering Software and Demo
no_demons writes "Along with a selection of other goodies, Apple also unveiled their Xgrid clustering technology from their advanced computation group today. Xgrid can turn a number of networked Macs into a supercomputer, detects nodes automagically via Rendezvous, and can run in or out of a screensaver mode. You can download a technology demo (including a BLAST test app) here."
Does this work on the small scale as well, like OpenMOSIX? We have a few G4's at school that could benifit from clustering.
You can't judge a book by the way it wears its hair.
It'll be interesting to see if I can make this work with the stacks of old LC520s in my garage. I've been wanting to cluster them for a while. If Xgrid will work on those, Mac just saved me a ton of work. Not that I wasn't going to have fun with it....
Damon,
http://actionPlant.com
What's the cost? I can't find any info on pricing, etc. Sounds like a great product if it works as advertised.
But for me, the model I want is a broker model. I want to sell my processor time to a broker who will resell it on a day to day basis to whoever is the highest bidder. E-bay of grid computing, ya know. I don't want to pick projects, download clients, etc. I just want to pariticipate (i.e. make money) from whoever is willing to pay the most at any given moment.
And when I feel like it, I'll volunteer x% to non-commericial stuff like SETI@home.
"If you want to improve, be content to be thought foolish and stupid." - Epictetus
Why limit this to Macs only? Wouldn't this work even better if it were cross-platform -- like many other distributed computing solutions such as SETI, distributed.net, and the UD Cancer projects.
Now every single story posted on slashdot for the next 6 months is going to have a comment of "Imagine an XGrid of these!" The old incarnation was slowly dying too...
What are we going to do tonight Brain?
1 = super computer
1100 = supercomputer
See, with that many more computers you DO run out of space
*rim shot*
Thanks folks, we're here all week. Try the veal; it's delicious. And please, don't forget to tip your waitress.
when the G4 was out the concept of a supercomputer was still the old how many gigaflops it ran, at that point the G4 was considered a supercomputer because it had just broken the old gigaflop barrier.
but with that Apple while advertising it was a supercomputer secretly wanted that concept to change cause it ment they couldnt sell the computer to a LOT of different countries due to export laws and supercomputers (the same thing happpened with Japan and the PS2 I think, or it might have been a urban legend in that case) Intel and AMD where also in on it since their chips ALSO where breaking the gigaflop barrier and where technically classified as "supercomputers" based on old 70's classifications.
Since then the rulling was changed and it will be a long time before any of the makers will reach the current mark with a one or two prosessor machine.
Now in Virginias case, it really IS a supercomputer, actually the third fastest at this point and THE fastest educational supercomputer
before I think the G4 was like 6 or 7 gigaherz slower than the closest "true" supercomputer
"Slashdot, where telling the truth is overrated but lying is insightful."
Oh good, you mean I can actually talk about XGrid now after signing an NDA over 6 months ago? :)
;)
We had the second installation of XGrid, the only other group using it at the time was NASA. I haven't had much time to play with it personally but we had our coop do some genetic sequence analysis using it and he was quite impressed. Plus the speedometer-like gauge measuring performance just looks soooo cool.
I don't know much about clustering, but maybe someone can fill me in. Would it be possible to run something like Postgres on a cluster of XServes? Would the clustering be transparent at the application level, so that any program could take advantage of the clustered resources or would specific distributed algorithm apps need to be written?
I'm hoping that I could just get a stack of XServes and run an OSS db on it for free (as in no extra effort required), but I'm guessing that's not the way it works.
I wonder if I could offload some of my compiles to TiVo...
:-)
Share and Enjoy!
Here is another wonderful example of Apple beating Microsoft on more than one level.
.......... and now Beowolf clustering out of the BOX!!!
1. Over all Coolness AND Geek Factors.
2. Usefulness of coolness factor.
3. Ease of use of Geek factor.
Microsoft hasn't done anything remotely like this in thier existance, rarely do they push the tech envelope beyond the what has become ordinary and benign. And when they do, they end up with over analized and engineered products nobody want (Bob?).
Apple of the otherhand is OFTEN making stuff WAY before the curve. Newton/PDA, USB/Firewire, CDROMs, Floppylessness (is that a word), standard networking
If Apple is smart with this, they could play against the new model of "ON DEMAND" services from companies like IBM.
Imagine a corporation that could automatically timeshare, timeshift computing resources based on such technology using the workstations they currently own/purchase. Peak periods of processing could be syphoned from little or unused desktops "On Demand", back filling any need left over from the DP center.
If this is NOT revolutionary it will be at least evolutionary.
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
From my own inexhaustive observations, Apple gives consumer stuff "i" and geeky stuff "X", hence iPod, iApps, iMac, iLife etc against XServe, XCode, XGrid, etc.
To be frank I've always wondered about Apple's name syntaxes. When the Mac IIx and the SE/30 came out - improvements of the Mac II and the SE with then top-of-the-line 68030 processors - it seems they really should have gone with Mac II/30 or SEx. Mac II/30 sounds like a third grade joke about chinese dentists but the Macintosh SEx would have probably made them billions.
Read ClusterWorld and you can figure this out yourself.
HPC for Primates. Read Cluster Monkey
What's more important is what it's clustering, 90 nanometer G5s. Apple and IBM are the first company to bring 90 nm processors to the market. Xserve White Paper
The whole point of the ban on export of supercomputers was that they could be used to simulate nuclear explosions
Not exactly, it was also their usage in brute-force decryption. The former point was indeed emphasized in the days of the old Cray supercomputer, but the latter was stressed as the most important factor in mid 1990's, when the aging cold war COCOM was replaced by the Wassenaar Agreement.
I wonder how long it will take someone at MIT or some other tech university to figure out if I put all the Macs located on or around a college campus I could or University could make it's self a mini-Super Computer for very few dollars.
:)
We did this 6 months ago when we were first beta testing XGrid. I never had the opportunity to personally play with it, but a grid of a hundred EMacs was a pretty good start to a mini-supercomputer I was told.
Isn't this just a batch queueing system, much like Sun's Grid computing, Platform's LSF and the like.
If it is, it is different in nature than the often joked about beowulf clusters and the mosix systems.
I think its great that Apple has one too, the more ideas thrown into the pot, the better all of them become.
-Tim
-I just work here... how am I supposed to know?
"Macs aren't cheap."
Oh, yes they are.
Ok, original post shoulda been modded funny or troll, but there are cluster solutions for old Macs, so here goes:
NetBSD/68k, supports 68k cpus and various free cluster architectures.
Appleseed, supports OS 8.6 - 9.x on PPC.
Quite a few older PCI Powermacs can be coaxed to run OS X 10.1.x using XPostFacto and some patience. Won't support XGrid for now, but the other free suspects will work.
Luke, help me take this mask off
what a phenomenal idea. to take advantage of the idle processing power of machines that sit unused at least 50% of their time. virginia tech is just the beginning.
any college with a several ~25-machine labs can use this app to do supercomputer stuff, AND get the return on investment from normal users being able to utilize these machines during the day.
www.pixelectric.com
Try the veal.
Breakfast served all day!
A code snippet from here(!) reads:
/* disable conditional is constant warning */
#ifdef _WIN32
#pragma warning( disable : 4127 4706 )
#endif
Why would developers at Apple ifdef for Win32?!
Clustering databases has different issues/concerns than clustering computational problems. I wrote an article about database clustering a while back, available here, if you're interested.
Breakfast served all day!
From Apples Website: http://www.apple.com/acg/ Xgrid Xgrid clustering software is intended primarily for scientific researchers who have a set of networked computers that are not yet being used to full collective CPU potential. The software provides a remote execution environment and file staging abilities that coordinate the running of tasks on distributed computing resources and ensure that each computer has access to all of the files necessary to execute the tasks, thus freeing both user and developer of such resources from this tedious work. Xgrid does not solve all clustering problems. It does not replace clustering software such as MPI or clustering hardware such as InfiniBand. Xgrid does not accelerate or "grid-enable" existing applications on your computer. For an application to take advantage of the Xgrid technology, you must update it to use the Xgrid APIs. However, if you currently run long computations in your Terminal windows using, say, an already-compiled executable, you should be able to use Xgrid out of the box to run batch jobs for this executable on your clusters. In this way Xgrid offers high-throughput computing with minimal development work on your part.
The guys at Genentech must be having heart spasms today. They do a lot of Altivec-optimised calculations, and between XGrid and the G5 XServes they're in hog heaven right now. Any Mac-equipped research facility is now sitting on an extra supercomputer they don't have to pay for.
Imagine that, an extra supercomputer.
Lockheed and Boeing are excited. Everyone in biotech is going nuts. Everybody's IT department should be thinking: what could we do with a crapload of extra CPU ?
Veteran, Bermuda Triangle Expeditionary Force, 1992-1951
I think that Virginia Tech would disagree with you on that last point there. There are not many applications that you could or would want to use 64-bit computing anyway so I don't understand your point. These days it's either DBs or number crunching. For the latter, having distributed system helps it even more.
But if you're not happy with your G5, you could send it to me. I'm sure I could use it to play Pong or something.
Well, there's spam egg sausage and spam, that's not got much spam in it.
Actually, a G5 starts out at $1799. You are thinking of the Cluster Server XServe, at $2999. If you actually were to do even a little bit of research you will start to notice that in order to match the performance of these machines you will have to spend a lot more than the $1000 you quoted.
Remember you have to have:
Gigabit Ethernet (the XServe has 2 ports built in).. I think this will probably account for $300-$500 of that thousand right there
SATA
Very high performance memory systems (with ECC on the XServe)
FireWire800 (drives and networking)
PCI-X (can you say Infiniband?)
And if we are focusing on the XServe:
Hardware fault notification (very well implemented)
1U rack space (slide out drawer, including cable management)
MacOS X Server (so nice to admin)
I don't think you know what you are talking about. After all, Virginia Tech just hit #3 on the supercomputing list with a cluster of G5's, and everyone is talking about how cheap they did it. The guy behind the project did a lot of research and discovered that this was the best price, Dell didn't even come close (they gave them 3 tries to do so).
Coincidentally, I was working on setting up Sun Grid Engine on a couple of G5s here at work yesterday when I heard the news about XGrid. I dropped my work on SGE immediately.
It was a little confusing at first to set up, but I eventually got 3 machines total configured as part of my grid - 2x2GHz G5s + 1x867MHz G4. Perhaps if I read the instructions, I would've better understood some of the terminology (agent v. controller v. client, etc.).
The tachometer is sort of flaky. Sometimes it's stuck at zero on one machine while it it is actively moving around on others. Other times it's stuck at some non-zero position. Opening up 2 tachs on the same machine (XGridBlast has its own tach) will show different speeds. Though I should have in theory 8.867 GHz total speed, I could never get it to go over the default 8 "red line"- I was curious if it would rescale once it exceeded 8.
The XGrid client (where you submit jobs) has some default demo type things (I've mostly been testing the Mandelbrot one as it runs in a continuous loop), but it also has a way to build "custom plug-ins" which allow you to submit arbitrary jobs. In other words, executables don't have to be modified per se. Of course for any kind of parallel execution, they do, but if you need to to run 1000 iterations of the same command with slightly different arguments, then it should be able to distribute that "run" pretty well. The GUI for building up such a run is pretty easy to use.
One potentially big issue I've noticed is that once you submit a job, you have to keep the XGrid app open until it is done. For a run that may take days or weeks, I think this could be a problem. I'd like for you to be able to submit the job, quit, and log out (or shutdown the client even) then come back later to check your results.
Also, there doesn't seem to be a queue manager where you can see a list of jobs and their states.
I think that for any file-dependent commands, you need local copies of the files on each Agent node. At least that's how it appeared from perusing the documents for XGrid Blast - each node needs a copy of the database.
FWIW, we're using SGE successfully on a bunch of RedHat based servers, but though OSX maybe *nix, the installation and config was turning to be a pain.
Anyway, those are my initial impressions. I'm sure some of these things will change in the "release" version. In the meantime, I'll have to get working on getting some real work stuff to try out (I work in a university bio department).
-h3