World Wide Cluster

gwjc writes: "There is a pretty good Ian Foster article on Web-based computing clusters at the Nature site. The usual SETI@home, condor, and entropia mentions as well as a few that were news to me such as "Compute against Cancer" and "Fight Aids At Home" with links. I wonder how I go about declaring."

Comments

There are downsides to this..

[Dan Hayes]

After all, whilst the advantages of distributed computing are clear in that they can provide a way of harnessing a lot of computing power for a cheap cost, there are also downsides to this kind of project.

If people are so taken up with this sort of thing, imagine how easily it could be abused. People don't tend to be able to recognise and deal with email viruses, let alone a rogue distributed project that claims to do one thing whilst in actuality do something else. It sounds to me like a perfect opportunity for intelligence agencies to get their software on people's computers without anyone knowing!

How can you tell whether that client you run 24-7 on your home computer is actually helping calculate the next prime number or whether it is scanning all of your net connections and sending the information to a giant government database to be perused at their leisure? Police states like Britain already want to keep records of everything you do, this seems like a damn good way of doing it on the sly.

Personally, I'd be very wary of any piece of software that sits on your PC and has a constant connection to the internet. Unfortunately, most people are too trusting when it comes to their security online...

Re:There are downsides to this..

[epaulson]

Look, It's not hard to build sandboxes to protect yourself against foreign code. The way to do this has been well-documented - it can be as simple as a chroot() call or as full-out as the virtualization tricks that VMWare and plex86 pull. Just because Sun and Microsoft haven't managed to get it right yet doesn't mean it's not possible.

And if you're really worried about abuse, let's take a quick paranoid-look at things.

Evil Groups:

1. Microsoft

2. NSA/CIA

3. Telecoms

I'd say that if the intelligence community wanted it's software on computers it's already got plenty of opportunities.

-Erik

This is quite scary.

[Lover's Arrival, The]

I mean, could this be the next stage of the Internet we are seeing? Could the internet end up as one big supercoumputing cluster, and when we use it we timeshare? It does seem to be the ultimate long term direction that the Web is heading in, doesn't it?

The thing that scares me is the possibility of said cluster being abused, and hackers using it for ill purpose. Also, what are the implications for privacy? Look at .Net and the like, this is the nest step, and all my private files will be spread all over the Web! Ultimately, the superdupercluster could become conscious and ruin us all! ;-) I would like to see these technologies more strictly controlled. Sharing of data is one thing, but sharing of processing power seems a bit on the dangerous side, don't you agree?

Thanks for reading.

Re:This is quite scary.

[drinkypoo]

I mean, could this be the next stage of the Internet we are seeing? Could the internet end up as one big supercoumputing cluster, and when we use it we timeshare? It does seem to be the ultimate long term direction that the Web is heading in, doesn't it?

I sure hope so.

I was just talking about this very concept with a coworker just the other day, about how to fairly share processor time.

The concept is something like this: Every N instructions executed on your system (probably in some sort of virtual machine) constitutes some sort of computing "unit". CPU-hours and such are meaningless when people have different processors, so I think that this is the best way to measure it. (Any resource can be shared this way, but let's just talk about processing power for now.) You should be able to either sell blocks of time on your systems (which opens up the possibility for companies which make their money by selling compute time) or you should be able to simply trade it.

We would need a central system to manage the block counts. This system must be free. Who should run such a system? I have no idea. In any case, when you send out your jobs, they get picked up (peer to peer) by whoever is idle, and processed. The doling out of jobs would be handled by the block count system.

So now let's say my system has had the cooperative network plugin sitting on it for a couple of months and I've built up, oh, a thousand blocks of processing. I now send out 500 blocks worth. My priority should be weighted positively because I have not been using my credit. Therefore, someone who has 500 credits and sends out the same amount of work should have to wait longer (and/or get less concurrent jobs) if my request alone will make the system busy.

Now, if the system is largely idle, and I have 1000 credits and send out 1500 blocks' worth of work, then my jobs still get processed immediately. However, I will then be at -500 credits. If I then submit another 500 block job and the system is busy, I will have a very low priority.

However, you might decide that you want certain people to have priority on your system even if they have a low number of credits. It's your system, it's your right. You should be able to add (or remove!) priority from a user/project ID, disallow all use by an ID, or allow use only by a certain ID. You should also be able to specify times during which only you have access, no one has access, a certain group is prioritized higher, or what have you. Again, it's your machine.

And finally, the source to the server code should be freely available so that anyone can run a distributed processing network of their own, for public or private use. You should also be able to merge networks and later split them again if a group wants to share their private resources with others. Key management should be peer to peer but brokered through the server (or at least signed by the server) for security purposes. Only people with a current (and verified) key should be able to use your resources, if you so specify.

Analysis of Distributed Projects

[�nubis]

Being a long time distributed computing advocate, I've used (and crunched many blocks for) GIMPS, distributed.net, Seti@Home, Folding@Home, and United Devices. I'm currently using all my spare cycles for United Devices. Why? Well here's a brief explanation/analysis of the projects I've used:

GIMPS - They have a good, clean client, but the critical problem is that the project has no conceivable benefit.

distributed.net - Probably the best client/site out there, and definitely the largest pre-Seti@home project. However, the cracking of encrypted messages has next to no scientific benefit (it is quite easy to calculate the chance per try of cracking any of their ciphertexts). Recently they've been doing some work with OGRs. Finding new OGRs looks like something that at least has a marginal benefit. On a side note, distributed.net has partnered up with United Devices.

Seti@home - It seems like everyone and their mom is running seti@home. However, reportedly seti@home actually has more clock cycles than they need. (they can only get so much radio info per day to analyze)

Folding@Home - Definitely a lesser known project which is being run by some researchers from Stanford, they analyze proteins. The project definitely has scientific merit, however they're experiencing some growing pains due to their recent popularity. Also their client is definitely beta-esque.

United Devices - This is the project that I'm currently contributing to. (so of course I'm bias) I chose them because they're doing something useful (working on cancer stuff with some researchers from the University of Oxford), have a fairly good client, and have a 'rewards' program for their users. (btw, GIMPS and distributed.net users also have the chance of winning a large cash prize) In addition, UD has partnered up with distributed.net, so it looks like UD just might be the commercial corporation to win the Internet-based distributed computing market.