Slashdot Mirror

← Back to Stories (view on slashdot.org)

Xgrid Agent for Unix

Posted by pudge on from the beowulf-strikes-again dept.

mac-diddy writes "Someone on Apple's mailing list for Xgrid, Apple's clustering software, just announced an 'Xgrid agent for Linux and other Unix platforms' available for download. There are still some issues being worked on like large file support, but it does allow you to simply add a Unix node to your existing Xgrid cluster. Just goes to show that when companies embrace open standards and code, the world doesn't fall apart."

18 of 219 comments (clear)

  1. My Experience by artlu · · Score: 5, Interesting

    My company has had experience using XGRID on our G4 notebooks. We always leave XGRID running and when we are at the office it is like having 20-30CPUs available at any given time. Now with Linux, we can have about 300 CPUs available, I just wonder how efficient it really is in the non-osx atmosphere.
    Time to find the download.

    GroupShares Inc. - A Free Online Investment Community

    --
    -------
    artlu.net
  2. How many clusters by Anonymous Coward · · Score: 5, Interesting

    actually have hetergenous hardware platforms? It would be interesting to see a G5/Xeon/Athlon cluster make the top 10 in speed.

  3. Plenty of power to be had.... by Grant29 · · Score: 3, Interesting

    This is really great news as it's becoming more popular to add CPU clusters to improve performance. Google is probably not the originator of this type of computing, but they have definately pushed it into the mainstream. Anyone living in NC might want to check out this new cluster going into RTP NC. I wonder if this will be the biggest cluser ever

    http://www.rtp.org/index.cfm?fuseaction=in_the_new s_item&id=159

    --
    9 Gmail invitations availiable

  4. I've been dying to know.... by numbski · · Score: 4, Interesting

    I have my G4 powerbook, 866 and my 800mHz iMac on my LAN at home.

    If I use XGrid on the two, what kind of performace could I use it for day to day?

    Faster compiles of applications would be the first thought. Any usefulness, say running photoshop? How about Quake? MAME?

    --

    Karma: Chameleon (mostly due to the fact that you come and go).

    1. Re:I've been dying to know.... by genericpenguin · · Score: 2, Interesting

      Pardon the expression, but "Not so, not so".

      While the Xgrid application does indeed allow you to create custom interfaces for command line programs, there is still the issue of data. Xgrid will start processes on remote machines but as to how data is read and distributed is another matter.

      i.e. If you have an application that simply generates data(eg. a calendar) then that would work well with the custom plug-in feature. However, if your program needs to be fed data(eg. sort a list read from stdin), your program would have to have a way of splitting the data and giving it to the appropriate process. To achieve this, you would really have to use the Xgrid API to write your own plug-in.

      --
      "Why, Johnny Ringo. You look like somebody just walked over your grave." Doc Holliday, Tombstone.
    2. Re:I've been dying to know.... by Graymalkin · · Score: 3, Interesting

      Feeding a remote process data is not as difficult as you're describing. If the program you're using doesn't support ranges in arguments it isn't too difficult to wrap a script around it that does understand input ranges. The Xgrid client makes it pretty simple to use ranges as arguments for programs. It's possible to use the likes of Blender and Xgrid to do distributed rendering. Ergo input control doesn't seem to be a terribly difficult hurdle to overcome with Xgrid.

      --
      I'm a loner Dottie, a Rebel.
    3. Re:I've been dying to know.... by Graymalkin · · Score: 2, Interesting

      You're trying to shoehorn the wrong sort of application into Xgrid and for some reason expecting it to work. Problems that can't be solved in a loosely coupled parallel environment are not well suited to be run on something like Xgrid. In the case of your fax numbers a custom Xgrid plug-in would probably be the least efficient way of doing the work. It would likely take longer to distribute the list and binaries to agents than it would for one fast system to run through the program.

      Xgrid isn't meant to solve all computational problems. It is designed to solve the ones involving long independent or at worst loosely coupled problems.

      --
      I'm a loner Dottie, a Rebel.
  5. Home cluster by vaguelyamused · · Score: 3, Interesting

    I wonder how effective this really is for home use? Will the performance improvement on my Powerbook be worth running XGrid on it and firing up a couple older computers (600Mhz IMac, Pentium III 1.0 Ghz) on Linux/OS X and adding them to the cluster. Would 100Mbs Ethernet cut it, what about WLAN?

    --
    STOP ROCK VIDEO
  6. GridEngine by Anonymous Coward · · Score: 2, Interesting

    I find Sun Grid Engine better than other similar grid tools...

    http://gridengine.sunsource.net

    1. Re:GridEngine by Anonymous Coward · · Score: 1, Interesting

      that's because the other tools are PBS, Condor and LoadLever, all complete crap. Grid Engine is also crap, but at least doesn't have the 80s baggage.

  7. Probably a silly question but... by Thaidog · · Score: 2, Interesting

    Can anybody confirm if the linux and unix ports are smp aware?

    --

    ||| I still can't believe Parkay's not butter.

  8. Re:So could someone please inform me by tupps · · Score: 5, Interesting

    Stop thinking of developers as individuals who are trying to sell a product and think of developers as people who work/contract for organisations.

    Instead of buying a product that is 95% of what I want I can take a OSS package that is 90% of the way there and pay a developer to customise it to exactly my needs. Now I have a solution that is perfect for my business, maybe given something back to the OSS community. While if I had bought the product I would probably have to change my business to use the product. The company now is also free of licensing and upgrade issues. Also they do not have to worry about the vendor going out of business or introducing a new version with no support for the old version.

    If you think of software as tools for business rather than something that a developer trys to sell OSS makes a lot more sense.

    --
    Go out and get sailing!
  9. Re:Mixed Company by DonGar · · Score: 4, Interesting

    What's worse.... that often ends up happening for loopback connections.

    --
    plus-good, double-plus-good
  10. Good for home use too. by Gordon+Bennett · · Score: 5, Interesting

    Some households have a mix of computers and one can begin to see the benefits - for example, to halve the video compression time of iMovie when making a DVD.
    Considering Apple's ease-of-use for heavyweight *NIX apps this would empower more people to have more computing resources available rather than the big fish out there - schools with low budgets would be able to stretch their capabilities that bit further. And so on.

    1. Re:Good for home use too. by zalas · · Score: 4, Interesting

      Might video compression work if the first scanning pass is done on one computer and the keyframe locations are extracted and then each computer in the grid/cluster would render the chunks between keyframes in parallel?

    2. Re:Good for home use too. by JamieF · · Score: 2, Interesting

      It might work well for a different usage scenario: ripping a DVD.

      The first step would be getting the data off of the DVD. Clearly that wouldn't be parallelizable unless you had multiple copies of the DVD (or some sort of hideous DVD multicast SAN).

      The second step (decryption) would probably be parallelizable. I don't know much about how CSS works but I imagine that it would be a fairly easy task to split the ciphertext into blocks for distribution and decryption.

      The third step (recompression) could be split up for parallelization by scene.

      The fourth step would be serial - reassembling the compressed movie into a single file, or maybe a few files if a maximum size (say, 700MB) for each single file were desired.

      Ideally, the second and third steps could be combined. Empirically it's clear that it's possible to jump to a scene without decrypting the entire title up to that point, so it should be possible to split the encrypted scene data out, pass it to a node, and get back that same scene in recompressed form.

      Splitting things up this way would probably also overcome the problem the parent post describes (that of not wanting every frame to be a key frame). Starting each scene as a key frame would add trivial overhead at worst; at best this is what a serial encoding process would do anyway since the frame content probably changes drastically with each scene change anyway.

  11. Re:Why bother? by oudzeeman · · Score: 2, Interesting

    OpenPBS is shit. It needs about 25 3rd party patches(many of which have to be applied in order) to be halfway decent, (Altair Engineering doesn't actively develop it anymore so these patches won't be integrated into OpenPBS). Also it often falls flat on its face. I wouldn't want to use OpenPBS unless I had a trivially small cluster.

    If you want something free, TORQUE is OK. It is a OpenPBS derivative (they started with the last OpenPBS version and added all the popular scalability and fault tolerance patches). TORQUE is actively developed under some DOE contracts, and even the company that has the DOE contracts to develop and support TORQUE will give other people some free support (I've had them on the phone helping to debug some of the code). You can get TORQUE from Supercluster.org.

    PBS Pro is very good, but costs a lot unless you are a degree granting department (then it is free).

  12. Re:Why another technology by Anonymous Coward · · Score: 2, Interesting

    "Mac" "embrased" open standards, and produced an easy-to-use solution with a reasonable GUI that actually has a chance of being adopted by end users. Have you actually tried setting up Globus yourself? It ain't easy, and it doesn't really do the same thing as Xgrid. As for OpenMosix -- on OS X???