MacResearch Introduces OpenMacGrid

Drew McCormack writes "MacResearch.org has just introduced OpenMacGrid. It is a distributed computing grid similar to SETI@home, but unlike other networks, it is built up entirely of Macs utilizing Xgrid, and access is unrestricted. Anyone with Mac OS X 10.4 can donate cycles, and any scientist with a reasonable project can burn cycles."

Comments

Movie studios and CGI

[Yoozer]

Y'know, I imagine stuff like this would be nice to speed up the rendering farms in movie studios. Either make 'm pay for the access or give every contributor with enough cycles a free ticket ;).

Re:Movie studios and CGI

[Max Romantschuk]

Y'know, I imagine stuff like this would be nice to speed up the rendering farms in movie studios. Either make 'm pay for the access or give every contributor with enough cycles a free ticket ;).

This only works in a LAN. Every single frame of a modern movie requires gigabytes of texture data etc. etc... It's not something you can send over the Internet.

Re:Movie studios and CGI

[Fist! Of! Death!]

But I was told the Internet 2.0 is the new LAN. Damn that Marketeer!

Re:Movie studios and CGI

[MidnightBrewer]

Actually, people can, and do, send render jobs over the internet all the time, and there are companies that make money off of renting out render farms (such as RenderCore, Inc.). As a freelance animator in Japan, I am actually considering using such a service in the near future, and have heard good things from other people about the service. The data only has to be sent once, and if it's cached remotely, the payoff is well worth it. After all, gigabytes of data might get sent one time, but subsequent transfers such as texture updates are going to be incremental, not completely from scratch.

As for the movie studios, a lot of them have the budget to get decent high-speed data lines, which is how a lot of movie dailies are shipped nowadays (think Lord of the Rings.)

Re:Movie studios and CGI

[Anonymous Coward]

With an iPod as backup. LINK

Re:Movie studios and CGI

[tolldog]

Not the same.

That service has data locality to their farm. Rendering over generic nodes on the internet won't give you that locality of data. Or you will have hundreds of clients streaming 20 GB of data to their machine to do 4 hours worth of work. It just isn't worth it.

Terminology

[John Nowak]

How about we call each node in the OpenMacGrid a MacGriddle?

What constitutes 'reasonable'?

[pickyouupatnine]

Who'll the judge? The community?

Re:What constitutes 'reasonable'?

[__aaclcg7560]

It's just commonsense. Modeling for global warming is OK. Modeling for taking over the world is NOT OK.

Re:What constitutes 'reasonable'?

[Funkcikle]

From my experience of the online Apple-using community, this entire thing will be used purely to predict when Apple will be releasing new shiny things to buy. Forget about global warming - good God, man! There could be a minor iPod update next Tuesday!

Re:What constitutes 'reasonable'?

[poopdeville]

iPod update on Tuesday?

Damn, I was gonna buy one tomorrow, but I'll wait. Thank you!

Re:What constitutes 'reasonable'?

[mwvdlee]

1. Accept all but the most blatent crap.
2. Present every client with a short one-liner describing the new project.
3. Let the user of the client decide whether to donate CPU on a project-by-project basis.

What I don't get is why this is Mac-only. Are Windows/Linux truely less able to perform these tasks or is it just a Mac promotional campaing under the guise of "research"? Seems illogical to me to keep out 95%+ of available CPU time if all you intend is to provide the research community with CPU time.

Re:What constitutes 'reasonable'?

[geekboybt]

Most likely because it relies on Xgrid, methinks.

Re:What constitutes 'reasonable'?

[Angostura]

It's Mac only, I guess quite simply that it uses the Xgrid software that is bundled in as standard with OS X. Details here. I seem to recall that there was a Java version of the Xgrid agent that ran on Windows, but I can't find it at the moment.

Re:What constitutes 'reasonable'?

[metalcup]

What I don't get is why this is Mac-only. Are Windows/Linux truely less able to perform these tasks or is it just a Mac promotional campaing under the guise of "research"? Because, X-grid is available only for Macs http://www.apple.com/server/macosx/features/xgrid. html, and all you need to do to set it up (i.e. allow your mac to be a part of the grid) is click on a few options in the system preferences panel - the end user does not need to work with scheduling and other details - the OS takes care of all that with a few options. It really is damn convinient to use for many types of clustering applications. (and I have setup Linux clusters etc). To that end, yeah, it is a bit of a promotional campaign, but only because no other OS can do it out of the box the way Mac can!!

Re:What constitutes 'reasonable'?

[Khabok]

if (motive != "TAKE OVER THE WORLD"){
giantShinyBeowulfCluster.task = motive;
giantShinyBeowulfCluster.mainScreen.turnOn (1);
}

else{
giantShinyBeowulfCluster.dDos (sender);
}

Re:What constitutes 'reasonable'?

[blake3737]

There could be a minor iPod update next Tuesday!
Who are you, think secret?

Re:What constitutes 'reasonable'?

[Hes Nikke]

iPod update on Tuesday?

Damn, I was gonna buy one tomorrow, but I'll wait. Thank you!

tomorrow IS tuesd...ohhhh!

Wow, Ronald is really diversifying the menu!

[Anonymous Coward]

Good for him! But I still won't eat there.

Trojans?

[Max Romantschuk]

From the disclaimer:
http://www.macresearch.org/contribute_to_openmacgr id

By contributing an agent to OpenMacGrid you recognize that third parties will be running software applications on your system. MacResearch.org, will make every attempt to ensure that third party applications are safe for execution on contributed systems (agents), but in no way will MacResearch.org or its affiliates be held liable for any damage to your system resulting from participation in OpenMacGrid. It is the responsibility of the person contributing the agent to ensure that they have permission to use the system in such a fashion.

So, Xgrid-experts, what kind of permissions does an application like this have? Is it sandboxed somehow?

Re:Trojans?

[dr.badass]

So, Xgrid-experts, what kind of permissions does an application like this have? Is it sandboxed somehow?

Xgrid jobs run as user 'nobody', which is decently safe, with process limits so it can't forkbomb you to death. A rogue job could fill up /tmp or ~/Public/Drop Box or whatever with garbage until you run out of disk, or some other annoying things. I won't say "nothing major", because that depends on what you've got that's readable or writable by others. I'm also not wearing my expert hat, so it's entirely possible that I'm unaware of some way that Xgrid jobs could 0wnz0r you.

You still need to trust OpenMacGrid to keep these bad jobs off the grid.

Re:Trojans?

[MichaelSmith]

You still need to trust OpenMacGrid to keep these bad jobs off the grid.

Sounds like an ideal candidate for chrooting

Re:Trojans?

[fatphil]

Indeed, but you'd also need to enable hard disk quotas to stop the filling of the hard disk.

Re:Trojans?

[JoeMerchant]

I'm thinking they're paranoid about system damage from overheating.

Re:Trojans?

[ATMD]

Do Macs support loopback devices?

Just make a file of whatever size you want to limit the jobs to, format it HFS+, mount it and chroot to it. Problem solved?

Re:Trojans?

[Jimithing DMB]

Do Macs support loopback devices?

Do they ever. The disk image mounting in OS X makes Linux's loopback devices look like crap. While you can make an image containing only a filesystem you can also make one containing a full disk image (including MBR and everything). When mounted it will for example show up as /dev/disk2 (whole disk), /dev/disk2s1 (first partition), /dev/disk2s2 (second partition) and so on. Makes disk recovery of an imaged disk a shitload easier because you don't have to go through the laborious task of calculating offsets based off the partition table. The kernel does it for you like it would with any other "disk".

Re:Trojans?

[Anonymous Coward]

Xgrid jobs run as user 'nobody', which is decently safe

Is it just me, or does that sound like a really stupid idea? The user 'nobody' is supposed to be... nobody. The whole point is that it's not a real user. It shouldn't be possible to log in as nobody; nobody should not be running any processes. It's the /dev/null of user accounts. If I set a file's group to nobody, then only I should be able to touch it. If nobody owns a file, then only the superuser should be able to touch it.

In the situation you're describing, 'nobody' seems to mean 'anybody'. And not just 'anybody on this machine', but 'anybody at all, anywhere on the Internet'.

Re:Trojans?

[profplump]

Traditionally the "nobody" user has been used by various daemons that need only read-only disk access (and often which can accept strict ulimits). It's generally not possible to log in as nobody, and nobody usually does not own any files, but it's still useful as daemon account.

It's becoming more common to assign each daemon its own user, but so long as your process doesn't write any files there's not much security benefit to having your own user, and there is a convience aspect to having fewer users to manage. In particular, it can become a hassle to ensure that each of 25 different daemon users has an account that prevents logins and owns no files; having fewer such accounts makes security verification simplier.

Slashdotted

[John Nowak]

I do hope their website isn't representative of their grid's performance...

Re:Slashdotted

[Anonymous Coward]

Their website is probably run by a grid of PPC macs...

*ducks*

(and yes, I used PPC macs for several years)

Re:Slashdotted

[Ash-Fox]

Their website is probably run by a grid of PPC macs...

Netcraft confirms they're running on Macs!

Re:Slashdotted

[Anonymous Coward]

Last I heard Xgrid does all the architecture transition for you. So PPC + x86 all the way.

Re:Slashdotted

[BWJones]

Naw, it's not the Macs, but the database that is limiting the connections and likely their bandwidth restrictions. I have a fair amount of experience getting Slashdotted on another website hosted exclusively on a Mac and I've yet to see it have a problem, even though it is a graphics intensive site.

Re:Slashdotted

[John Nowak]

I certainly meant nothing ill by it -- I posted it from my new iMac Core 2 Duo. :-)

Re:Slashdotted

[elrous0]

Impossible. That smug kid in those commercials said Mac's "just work."

-Eric

Re:Slashdotted

[Ash-Fox]

Impossible. That smug kid in those commercials said Mac's "just work."

But... He also claimed he could do business things too, which conflicts with the "just work" rule.

Re:Slashdotted

[Anonymous Coward]

So you mean its cool, quiet and fairly efficient with awesome vector abilities per clock?

Those poor bastards. The death of consumer PPC is BAD for the industry ingeneral, and is nothing to be celebrated.

Re:Slashdotted

[Anonymous Coward]

I agree, but my PPC Mac was soooo damn slow (running at "only" 1420MHz). My current 2.4GHz P4 is SOOO much faster, really. And that's not just Java.

Ok, maybe the Mach-O format, the Mac's memory manager and other inefficiencies like that are to fault, but then all the more reason for Apple to switch to a Linux / more modern kernel!

Re:Slashdotted

[marklar1]

Good point, but you've missed the biggest culprit... System Bus Speed.

Alot of the problem was the 1.4 GHz mac you are referring to was a PPC G4.... crippled by a 133 or 167 Mhz system bus. Were the PPC chips allowded to go one gen further on the laptop side (desktop remedied at onset G5 w/ sys bus at 1/2 CPU clock) then this problem would've ceased to be.

Macs never really saw the full benefit of the G4 due to the system bus...sad. Motorola had them ready to go in a highly efficient laptop form, I think w/ 667 Mhz busses, and doubled cache...but Apple made the jump to Intel. I was dissapointed, but Moto had failed so many times to keep up with Intel, or even their own predictions, that in a way it was a relief.

Re:Slashdotted

[Anonymous Coward]

Yes, it's interesting that everybody blames the G4's bus.

Back when DDR RAM was being introduced, I read about measurements that the bandwidth increase from 133 to 266 resulted in a whopping 3% performance increase for real applications.

That made me highly doubt any of those claims that a faster bus means anything. I'd like to see more up-to-date measurements of how a fast bus translates to higher performance, including the faster P4 bus, the Core, and the Athlon 64.

OTOH, that Mac experience confirmed it a bit (though CPU usage over time was also quite often on the high side, but that might have included time waiting for memory).

imagine....

[dino303]

with a beowulf cluster of these... they might even handle the rush from slashdot.

Re:imagine....

[MattPat]

with a beowulf cluster of these... they might even handle the rush from slashdot.

Well then, it's quite clear that they don't have one: Warning: mysql_connect(): Too many connections. :)

They'd better not use the same MySQL database for storing grid results... I can picture the headlines now: "Cure for Cancer Lost Because of Traffic from Geek Website."

Curious indeed

[agent dero]

I went ahead and signed up (what can I say, I'm a sucker for science) but I'm really hoping they make it clear what will be running on the agents.

One thing quite curious, the "introduction" images are almost direct yanks from xgrid@stanfard including the Dashboard widget the push as their own from the xgrid widget SDK linked with the xgrid@stanford project as well.

Should be interesting how this shapes up. 91 total agents right now, 0 working :-P

Re:Curious indeed

[rockrat]

Well, Charles Parnot who is the creator and manager of xgrid@stanford (and a major contributer to the Apple Xgrid community) is also a major player in this project. So, it's not really that curious.

Obligatory one...

[quigonn]

Hot grids down your pants!

Usefulness?

[wlan0]

How useful can it be to be locked into one OS? How hard is it to make a commandline program and then a Cocoa interface, that way you can get everyone and still have a pretty window and widget for OS X users.

Re:Usefulness?

[dr.badass]

How useful can it be to be locked into one OS? How hard is it to make a commandline program and then a Cocoa interface, that way you can get everyone and still have a pretty window and widget for OS X users.

OpenMacGrid uses Xgrid, which is Mac-only. It isn't something new they've made: it's built-in to Mac OS X. You ask "how hard is it...", and the answer is "A lot harder than just using what's already available."

Also, the Xgrid agent doesn't have a pretty window. It's a background daemon.

Re:Usefulness?

[fatphil]

In your excitement you seem to have forgotten that the command line was already available too.

I organise cross-platform distributed computing projects, and without exception the actual number-crunching code is an console application, plain C - not even any #ifs, and the only thing that is platform specific is a tiny wrapper script. The wrapper script I use is a Perl one that sits silently in a text console, or hung up, or started from init or cron. There's nothing to stop someone writing a GUI one in Tcl/Tk, or whatever scripting language runs on OSX. It would be a half hour job for someone familiar with how to use sockets in their scripting language of choice to 'port' my application to a new platform. Being tied to XGrid is an truly abysmal design in comparison.

Re:Usefulness?

[peragrin]

what do you think open grid is? it is just a gui wrapper around Xgrid.

And Xgrid only works on macs. sine it is just a daemon for the client there isn't any reason why it couldn't be ported to any other *nix, but that is up to apple to do. A simple google search would show you that someone has already duplicated xgrid's functionality with the help of apple. Tying at least Linux boxes into the client network. It looks like it hasn't taken off, but then again neither has xgrid. maybe this project will help both.

Re:Usefulness?

[dr.badass]

without exception the actual number-crunching code is an console application, plain C - not even any #ifs, and the only thing that is platform specific is a tiny wrapper script.

Without exception? Are you sure nobody ever does any vectorization outside of what the compiler does? Ever? Nobody ever links to platform-specific frameworks for any reason? Ever? Nobody ever writes code to run on a homogenous cluster? Ever?

Being tied to XGrid is an truly abysmal design in comparison.

In your case perhaps it would be. That doesn't mean that there aren't plenty of other people for whom it is more ideal than your setup. Those people include scientists that already run Mac OS X, and perhaps link the Accelerate framework, and perhaps already have Xgrid-ready jobs that can run on OpenMacGrid without modification. Or maybe they just don't want to concern themselves with the details of distributing jobs and returning results with a Perl script or something.

Re:Usefulness?

[Anonymous Coward]

How useful can it be to be locked into one OS?

Even worse than that, it is tied to 10.4. Anyone who hasn't upgraded can't participate. Last time I played with Xgrid, it transmitted the application along with the unit of work. Sure, it makes it easy to just build a universal binary and load it onto the controller, but why not have multiple builds on the controller and have the agent indicate the OS level when it requests work? The controller could then send the appropriate build down.

Re:Usefulness?

[hawaiian717]

It's been done: http://www.novajo.ca/xgridagent/. The problem is that Xgrid ships a binary to the target system to the client system to execute. When I played around with it I was able to get the agent to run on Gentoo x86, but naturally the job built for Mac OS X on PowerPC wouldn't run. There are potential ways around it; using an interpreted language like Python or Java might work (but be detrimental to performance) or using a wrapper script, but that's not something I had much success with.

Re:Usefulness?

[fatphil]

Did you miss the bit where I mentioned that it's wrappers around platform-independent portable things are useful. Wrappers around entirely non-portable things are next to useless.

But probably very pretty. Enjoy your eye-candy.

Re:Usefulness?

[fatphil]

Don't be a plonker.

What I wrote was
"""
I organise cross-platform distributed computing projects, and without exception the actual number-crunching code is an console application, plain C
"""

And you intepret that as something like
"all distributed projects without exception ..."

Please leave the 'reply' button alone until you learn to read.

Re:Usefulness?

[Anonymous Coward]

Dude, chill. As if you weren't implying your experience was generalizable (otherwise it's not particularly relevant here). Please leave the "reply" button alone until you learn to recognize your mistakes and admit them graciously.

Re:Usefulness?

[dr.badass]

What I wrote was...

If you weren't trying to generalize about distributed projects, then what the hell was your point?

Re:Usefulness?

[fatphil]

My point was that if you write portable code you can run it on many platforms without modification, and that this is a far superiour way of doing things than restricting yourself to one single minority platform.

Which should be obvious to anyone with a brain.

Re:Usefulness?

[dr.badass]

My point was that if you write portable code you can run it on many platforms without modification

Wow, that's a whopper. No wonder I missed it. Portable code is portable. Dang. You should get an award.

and that this is a far superiour way of doing things than restricting yourself to one single minority platform.

It is not superior in all cases. In fact, it is quite inferior for the target audience who (as I have already stated) may be part of an all-Mac lab already, may be using the Accelerate framework, may be using 64-bit, may already be using Xgrid jobs on private grids, etc. There are any number of reasons why people would choose to "restrict" themselves to the platform. Writing for portability means losing every advantage that the platform has in favor of a potentially larger and less efficient grid, which you will then have to find some cross-platform way to distribute jobs to and return results from. Not everyone is into that kind of masochism -- some scientists actually prefer to do science.

Re:Usefulness?

[fatphil]

You're one for whoppers yourself it seems - if you're prepared to restrict your target audience then it doesn't matter if you restrict your target audience.

Note, however, that I didn't need you to point that out twice, unlike my self-evident truth which you needed to have explained to you.

If you worked things out first time, you'd optimise your brain use by 50%!

Re:Usefulness?

[dr.badass]

what do you think open grid is? it is just a gui wrapper around Xgrid.

Um, no it isn't. OpenMacGrid is just an Xgrid controller. There is no GUI. You enter the address into the built-in Mac OS X Sharing Preferences pane, check a box, and you're done.

it hasn't taken off, but then again neither has xgrid

I'm not sure what your criteria are for "taking off", but Xgrid has been pretty successful. I think it doesn't get much press because the majority of grids are not public.

Re:Usefulness?

[dr.badass]

Your first comment in reply to mine contained the assertion that depending on Xgrid was an "abysmal design in comparison" to the "far superiour way" of limiting yourself to portable C. I have twice given you reasons why this is not always the case, and all you have done is repeat how "obvious" and "self-evident" your assertion is instead of making any attempt to address those reasons. Just let it go if that's all you're going to do.

/. effect good?

[ljaguar]

I just configured my ibook following the simple instructions here.

Dead simple. The process is still running at 0.0% so i guess i haven't been assigned anything yet.

First time that /. effect is beneficial to those involved!

Re:/. effect good?

[bill_mcgonigle]

First time that /. effect is beneficial to those involved!

People kill for the kind of exposure Slashdot brings. That's why we get Astroturfed occasionally.

Re-Enactment

[DrRevotron]

Re-enactment of the creation of the OpenMacGrid...

Person 1: "Hey, I've got an idea!"
Person 2: "Yeah?"
Person 1: "Yeah! Let's make a compute grid... except, it won't be like those other compute grids. Except, it kinda will. But it won't. But that's not the point. People will be able to submit their own projects!"
Person 2: "Oh, you mean like BOINC, GPU, The World Community Grid, distributed.net, Leiden, Grid.org, OurGrid..."
Person 1: "Well, uh... yeah... I guess... except, um... let's run it on a Mac!"
Person 2: "Hey, yeah, that's a totally original and cool plan, as opposed to actually devoting processor time to worthwhile and established projects like Folding@Home and SETI!"

Thought: Maybe, instead of everybody making their own little grid system... we could all make things go ALOT faster by devoting our processors to more than simulating chess games (Yes, I'm talking to you, Chess960) and focus it where it really counts, like finding a cure to debilitating diseases or searching for intelligent life. (Not a whole lot of it on Earth.)

Re:Re-Enactment

[gardyloo]

Thought: Maybe, instead of everybody making their own little grid system... we could all make things go ALOT faster by devoting our processors to more than simulating chess games (Yes, I'm talking to you, Chess960) and focus it where it really counts, like finding a cure to debilitating diseases or searching for intelligent life. (Not a whole lot of it on Earth.)

    Papa Bear would say you hate freedom. Do you hate freedom, Doctor?

Re:Re-Enactment

[Anonymous Coward]

Apple ships a generic grid framework called XGrid (generally for building private grids). This is just building on that framework, so they're not really building a new grid system.

The reason its only on a Mac is because... well... XGrid is only on a Mac.

Re:Re-Enactment

[Psiven]

Mod parent under-rated. The user raises a ligitmate point.

Re:Re-Enactment

[Anonymous Coward]

Hey, yeah, that's a totally original and cool plan, as opposed to actually devoting processor time to worthwhile and established projects like Folding@Home and SETI!

I hate to have to be the one to break it to you, but farting at home is both more worthwhile and established than Searching for Extra-Terrestrial Intelligence @ home.

Re:Re-Enactment

[dr.badass]

Maybe, instead of everybody making their own little grid system...

I don't think you fully understand what you're talking about.

For starters, BOINC is not a separate grid. It's a framework and client for many grids. BOINC users can (and do) contribute to many different projects including your "established projects" like Folding@Home and SETI@Home, and including many of the other grids you've listed. Many of the others you listed do exactly the kind of jobs you're calling for, like disease research.

Also, you seem to think that all grid computing projects are interchangeable, and that just isn't so. They may work with different data, or using different methods; they may not have the same requirements for job submission; they may operate on vastly different scales. Basically, they're suited for different research needs. A nice thing about OpenMacGrid, for example, is that researchers can take the same Xgrid job they've been using on their tiny network and send it to a public grid without much, if any modification.

Re:Re-Enactment

[Paradise Pete]

focus it where it really counts, like finding a cure to debilitating diseases or searching for intelligent life.

I agree with your sentiments, but folding@home is a solid investment, seti is like buying tickets to a million-ball lottery.

Re:Re-Enactment

[Billy the Impaler]

Though Folding put out a Folding@BOINC beta, I think the project is dead now. As such your assertion that Folding runs on BOINC is incorrect. Folding runs on its own client and network, just as the SETI Classic client did.

Re:Re-Enactment

[bill_mcgonigle]

Though Folding put out a Folding@BOINC beta, I think the project is dead now. As such your assertion that Folding runs on BOINC is incorrect. Folding runs on its own client and network, just as the SETI Classic client did.

That's true - and the reason I'm not donating any cycles to Folding, though I otherwise would if they were on BOINC. I like that the BOINC projects get that I don't want to sysadmin more than one distributed computing client.

Re:Re-Enactment

[bill_mcgonigle]

I agree with your sentiments, but folding@home is a solid investment, seti is like buying tickets to a million-ball lottery.

Spoken like a good engineer. Folding@Home has the potential to change how life is lived on Earth - a very good investment. SETI@Home has the potential to change what we think life is - a philosopher's game.

I wish Folding would get back to their BOINC client so I can easily donate some cycles to their project.

Re:Re-Enactment

[dead_one]

Except there are probably plenty of scientists out there with far less ambitious projects which would none-the-less benefit from a few days worth of spare cycles provided accessing them was less hassle than just running the project locally. I've just started a number theory phd, and sometimes find myself staring at processes that'll take three days to finish on my machine. Not worth setting up a global distributed computing project for, but if there's one just sitting around that I can easily offload it to, run something else requiring more interaction and still have my answer in three hours instead, then that'd speed things up considerably for me.

Apologies if some of the projects you mention do just that (and the only novelty here is the use of Macs), as I haven't looked at distributed computing efforts in years and am assuming they're all assigned to long-range goals (factoring, protein folding, SETI etc) rather than being generally open to researchers.

Re:Re-Enactment

[dreamer-of-rules]

What with the whole 'submit a proposal' process to go through first, I doubt this would work for your three day projects. Also, IFAIK, OS X does have local exploits, which means that you still need to trust the software being run by the XGrid on your system.

If the proposal review doesn't insist on reviewing and archiving source code first, this could be the start of a nasty bot-net. Am I right?

Re:Re-Enactment

[thejoelpatrol]

Person 1: "Well, uh... yeah... I guess... except, um... let's run it on a Mac!" Xgrid only runs on Macs. It is, as I understand it, fairly easy to design for and set up. There is a reason they did this. I've run BOINC. It supports some good projects. But at the time I tried it, it was kind of a pain to manage, update, etc. Something like this based on a built-in component of the OS should be pretty simple. That's the idea.

Re:Re-Enactment

[Stewie241]

So I guess the question is why did Apple pull a Microsoft and create its own new standard rather than building in something that would work with something that is already there? Based on discussions here (I know little of it), there is no shortage of distributed models. What makes Apples special? Have they innovated in any special way? Or did they create a new standard for the sake of creating a new standard?

Re:Re-Enactment ..obligatory Python reference

[Anonymous Coward]

(Monty that is), inspired by your last line, of course...
http://www.gecdsb.on.ca/d&g/astro/music/Galaxy_Son g.html

...and access is unrestricted.

[EatingSteak]

"...and access is unrestricted."

Well, kind of. Except for the fact that you need a proprietary OS to access it. And proprietary hardware to go with it. It seems if you do not have the correct hardware and try to run this, Apple will sue the shit out of you. Why don't they make this compatable with all versions of FreeBSD, then call it unrestricted?

Re:...and access is unrestricted.

[poopdeville]

Because they're running Xgrid, probably in the hopes of using unused cycles on Xeon-based PowerMacs.

Re:...and access is unrestricted.

[mr_matticus]

Isn't that like expecting that a public airstrip claiming "unrestricted access" be accessible to submarines, too?

Practical limitations may apply without something violating a notion of "unrestricted." Sort of like how unrestricted Internet access in your home still requires you to have a computer or other suitable device; you can't just plug the Internet into your arm.

Re:...and access is unrestricted.

[Shashvat]

you can't just plug the Internet into your arm Now there's a good research subject to throw CPU cycles at.

Re:...and access is unrestricted.

[WrecklessSandwich]

you can't just plug the Internet into your arm.

I beg to differ.

Re:...and access is unrestricted.

[suv4x4]

you can't just plug the Internet into your arm.

Oooooh :((

Pitty, that would really put an interesting spin on the "donating cycles" phrase.

Re:...and access is unrestricted.

[Lars T.]

"...and access is unrestricted."

Well, kind of. Except for the fact that you need a proprietary OS to access it. And proprietary hardware to go with it. It seems if you do not have the correct hardware and try to run this, Apple will sue the shit out of you . Why don't they make this compatable with all versions of FreeBSD, then call it unrestricted?
But it would also require somebody to be able to read for comprehension. Since you didn't understand that article, that rules you out anyway.

Java Web Start

[DamonHD]

I've always thought that the Java sandbox is well-enough understood and fast enough to be a natural for this sort of thing, since it's properties are fairly well tested and understood. And 'HotSpot' JVMs can give very good performance too.

For example, I just put up this Java Web Start http://master.gallery.hd.org/_AI/remote.jsp project to enable people to help along my little AI project. It is entirely in the sandbox, and is careful with bandwidth and memory and CPU usage. But you don't have to trust me, you only have to trust Sun.

Rgds

Damon

Macs and research?

[kunalthakar]

I don't think scientists who require computational power work on macs. Why would they require macs when there are beowulf clusters around?

Re:Macs and research?

[ananke]

Ever heard of the 'workstation' concept?

Re:Macs and research?

[VaderPi]

I guess you have never heard of SystemX at Virginia Tech. It is a supercomputer originally built with off the shelf Mac G5s. It was even written about in Slashdot when it made the #3 spot in the list of the fastest supercomputers.

Re:Macs and research?

[WiiVault]

Yeah I never meet scientists who use a Mac. Go troll somewhere else Mr. Gates.

Kind of makes me hungry...

[rogtioko]

...Bad for future foodility bills :(

From my general vegan perspective: could instead call each node an apple. Get it? apples on tree branch grid. Dynamic with key word Macgrid.

Not to mention

[eklitzke]

Y'know, I imagine stuff like this would be nice to speed up the rendering farms in movie studios. Either make 'm pay for the access or give every contributor with enough cycles a free ticket ;).

This only works in a LAN. Every single frame of a modern movie requires gigabytes of texture data etc. etc... It's not something you can send over the Internet.

Not to mention I'm sure they'd be thrilled to have basically the entire movie contents floating around the Internet on random people's computers.

Re:Not to mention

[Yoozer]

Solve the texture problem with a good centralized library of textures - or use procedural ones - not that uncommon. Just imagine that a node will only render a small square of the scene (or even spend its time calculating the light sources from a single render pass which are then put in the pipeline) and nobody's getting the entire movie.

Of course, there's the problem if you manage not only to catch every single piece, decrypt the bitmaps, -and- manage to put 'm in the right order. However, the amount of work you need for that is way more than bribing someone for a pre-release version.

Re:Not to mention

[onemorechip]

...or someone who cracks the code and figures how to splice in a random frame, a la Tyler Durden in Fight Club.

Missing the point

[Anthonares]

The comments so far have (mostly) overlooked the main point of just why the OpenMacGrid is different: it's *open*. That means that scientists, even PhD students like myself that want to run jobs using far greater numbers of nodes than the clusters (beowolf or otherwise) at our home institutions will now have a means to do so. Most such projects have neither the resources nor the capabilities to create their own custom cross-platform clients like those mentioned from other distributed computing projects.

OpenMacGrid (or just OMG, I guess) uses XGrid, which is built-in to every OSX 10.4 distribution and acts just like any other job queue manager, except it's even easier. So, the whole process of writing a distributed computing project becomes far simpler as well.

Finally, the OMG it doesn't matter if the OMG is cross-platform running on proprietary hardware: so is every other cluster that I as a scientist have ever had access to. The SGI cluster is proprietary, and to an extent so is the Linux machine at our High-Performance Computing lab on campus. And, if you're thinking about it being non-cross-platform from the client side, well, you're probably not thinking differently anyway, so just go download Folding@Home.

Re:Missing the point

[rogtioko]

I think dr.badass said why OpenMacGrid is innovative. namely that it uses Xgrid whereas for other distributed computing projects a user would have to adapt code to a certain format.

Re:Missing the point

[bcg]

I totally agree with you.

I see the difficulty of starting a distributed computing project as being a serious problem.

One solution I am proposing is borrowing some of the techniques from BitTorrent and using them in distributed computing. So far, the results have been very encouraging.

So here is a tidbit of my PhD research (an abstract from a paper hopefully being published soon):

"This paper describes the operational characteristics of "CompTorrent", a general purpose distributed computing platform that provides a low entry cost to creating new distributed computing projects. An algorithm is embedded into a metadata file along with data set details which are then published on the Internet. Potential nodes discover and download metadata files for projects they wish to participate in, extract the algorithm and data set descriptors, and join other participants in maintaining a swarm. This swarm then cooperatively shares the raw data set in pieces between nodes and applies the algorithm to produce a computed data set. This computed data set is also shared and distributed amongst participating nodes. CompTorrent allows a "simple home-brewed" solution for small or individual distributed computing projects. Testing and experimentation have shown CompTorrent to be an effective system that provides similar benefits for distributed computing to those BitTorrent provides for large file distribution."

If anyone else is interested in my shameless self promotion: http://www.comp.utas.edu.au/users/bcg/

Re:Missing the point

[Anthonares]

bcg,
This looks really interesting, and using BitTorrent in tandem with a distributed computing queue scheduler (like XGrid, in the case of OMG) could provide not only the ability to effectively distribute coarse-grained tasks, but also efficiently share large input datasets for each process.

Thanks!

hehe

[Anonymous Coward]

seems they are running there website on the grid

sql error too many connections

Re:hehe

[Imaria]

There is a quiet satisfaction in seeing a distributed network website be slashdotted; you would think they'd be prepared for a lot of connections.

two ways to get more nodes on board

[v1]

Instead of blindly contributing my cycles to whatever project some group of people in california (or wherever) decide is the project of the day, I would like it if I was given the option as a node to pick which project(s) my cycles were used for. People feel better about helping others and contributing/donating when they have a better knowledge of what exactly they are helping. I would be more likely to donate my cycles if I was able to pick which project I was most interested in loaning my hardware to.

It would also be to their benefit to introduce some competition. Contests like RC64 encouraged teamwork, and there were daily ranking boards where you could go see whose teams were knocking out the most units that day. There is no better motivator to encourage donation of resources than competition and bragging rights. Many of those teams were group oriented, there were things like TeamUnix, TeamMacinotosh, TeamUCLA, etc, and again that gives the nodes in each team a feeling of belonging to a group of people they can relate to, even if they have little in common.

Re:two ways to get more nodes on board

[bill_mcgonigle]

I would be more likely to donate my cycles if I was able to pick which project I was most interested in loaning my hardware to.

You're looking for BOINC.

Fan burnout a result?

[Oshawapilot]

With all the stories I've heard of the cooling fans on the MacBook's going wonky if run at too high of a speed for too long, I'm reluctant to offer my spare cycles to Xgrid.

The last thing I want to happen as a result of being a participant is seeing my fan spooled up to 6000+ RPM day in and day out while my Mac crunches numbers, only to result in the fan itself crapping out a few months later.

Re:Fan burnout a result?

[King_TJ]

IMHO, it's silly to contribute a portable computer's CPU cycles to these types of projects anyway. They usually run when your machine is *idle*. Portables are usually shut off or put into a sleep mode and disconnected from the Internet when they're not actually being used.

However, I will very likely set up my Mac Pro for this project when I get home tonight. It runs 24 hours/7 days since I have a small ftp server on it. It may as well be doing something else useful for other people while it's using the electricity anyway.

Re:Fan burnout a result?

[swebster]

...except that CPUs use more power when loaded (see here for instance).

botnet

[pikine]

"nobody" user can still listen for and establish connections over network, so an OpenMacGrid node can participate in DDoS attack and spam delivery.

Grid computing is essentially botnet, trying to use that concept for good scientific purpose.

No...

[denzacar]

It is more like expecting McDonald's saying they will give 50c from each purchase to a fund to end the world hunger, but ONLY if you order your BigMacs wearing at least 1500$ worth of Armani clothing and apparel.

And an iPod.

Re:No...

[mr_matticus]

No it's not. You've got your analogy backwards. McDonald's serves multiple groups of people and you've singled out a subset, whereas the situation at hand serves only a subset.

This situation is exactly like going to Wendy's and collecting 50c from each purchase, but letting McDonald's, Burger King, and Carl's Jr. sit it out.

Lean, mean, MacGriddle...

[ThomsonsPier]

You'll have to stave off George Foreman, or he'll try to put his name on it.

What about proprietary research?

[ubrgeek]

Big Pharm, for example, could sneak in research that is truly useful, but then gets pumped into its proprietary medicinal development? That takes it out of the realm of Open Source research, doesn't it?

Re:What about proprietary research?

[drc1]

Big Pharma have more than enough internal resources to do this sort of stuff, added to which I'd be very suprised if they wanted details of what would be a very early stage project potentially leaking into the public domain.

Is this Apple-only?

[Anonymous Coward]

If this is not an exclusive Mac-only technology then I am not interested. Why would I want to share my computing cycles with users of inferior platforms?

Patents.

[Anonymous Coward]

How many patents this project will generate per year? I am happy to help these smart guys to make money.

Question ...

[ScrewMaster]

" ... and any scientist with a reasonable project can burn cycles."

Define "reasonable".

Site is now back up

[drc1]

After being hit by slashdot the site appears to be up and running fine now.

What utter crap.

[Umrick]

Signed up, and checked in tonight. What the hell is it running?

nobody 5606 100.0 -3.4 103580 70916 ?? RN 8:00PM 130:53.62 setiathome_5.13_powerpc-apple-darwin
nobody 5598 0.0 -0.3 28208 5508 ?? SNs 7:57PM 0:02.37 /var/xgrid/agent/tasks/4nOFfwTN/executables/boinc -no_gui_rpc -attach_project http://setiathome/

For a serious grid, sure I'd donate. But for what sure seems like an attempt to inflate some jack-off's seti at home score? Don't think so.

Re:What utter crap.

[Anonymous Coward]

Just to put you straight on this: We are integrating BOINC into OpenMacGrid, and are currently testing it. The seti job you saw was a test job, nothing more.
Why are we integrating BOINC into OpenMacGrid? At times, OpenMacGrid may not be completely filled by submitted projects. It will depend on how many projects are running at the time. If the grid is reasonably empty, we will probably run BOINC projects, not to 'inflate some jack-off's seti at home score', as you suggest, but just to make best use of the valuable resource that OpenMacGrid is. It would be a shame to let it sit idle when there is an ebb in projects.
It is also important to point out that SUBMITTED PROJECTS WILL ALWAYS GET PRIORITY over BOINC jobs, or anything else; BOINC will only be used as a top-up.
We haven't determined which BOINC projects will be run on OMG yet, but it will probably be a mix.

Drew McCormack
MacResearch.org