Grid Computes 420 Years Worth of Data in 4 Months

Da Massive writes with a ComputerWorld article about a grid computing approach to the malaria disease. By running the problem across 5,000 computer for a total of four months, the WISDOM project analyzed some 80,000 drug compounds every hour. The search for new drug compounds is normally a time-intensive process, but the grid approach did the work of 420 years of computation in just 16 weeks. Individuals in over 25 countries participated. " All computers ran open source grid software, gLite, which allowed them to access central grid storage elements which were installed on Linux machines located in several countries worldwide. Besides being collected and saved in storage elements, data was also analyzed separately with meaningful results stored in a relational database. The database was installed on a separate Linux machine, to allow scientists to more easily analyze and select useful compounds." Are there any other 'big picture' problems out there you think would benefit from the grid approach?

Comments

Excellent

[President_Camacho]

The search for new drug compounds is normally a time-intensive process, but the grid approach did the work of 420 years of computation in just 16 weeks.

Cue the stoners in 5, 4, 3, 2....

Ok, how does this apply to patents?

[zappepcs]

If the grid solution finds THE cure for H5N1, will it be patentable? If not, who pays for the R&D to implement it? Who gets the patent? Do the thousands of people who allowed their PCs to be used get anything? Will big drug companies be able to use this and keep the prices low for the final product?

Re:Ok, how does this apply to patents?

[Dr. Spork]

These are all good questions, and every user who volunteers their computer for something like this should find answers to them. I'm quite sure that the stuff discovered by distributed networks does not automatically enter the public domain, but in cases like SETI and protein folding, the organizers explicitly state that it will. But it wouldn't be illegal for a drug company to use volunteers' computers just for corporate profit. You have to judge the merit of each of these projects on a case-by-case basis. Remember also that there is a cost to participating: you have to run your computer at peak power, and this will add several hundred dollars to your utility bills each year while polluting the planet with extra coal smoke and CO2.

years of computation?

[convolvatron]

sorry, i missed that definition. what is that in library of congresses per human hair?

Here's one

[realmolo]

Would it be possible to use all that computing power to make an electronic voting machine that works?

Oh wait! How about a voting machine based on "quantum computing"! Then we wouldn't even have to vote, the machine would already know who won.

Goddamn liberal qubits! Bunch of flip-floppers!

Stupid conservative qubits! They think that there is ONE and ONLY ONE answer for everything!

Re:Here's one

[iminplaya]

Then we wouldn't even have to vote, the machine would already know who won.

That's a no-go. Reading the result will change them. Kinda like what happened in Florida :-) Proof that you can do quantum processing with pencil and paper without all these electronical contraptions.

From the Article

[imstanny]

Up to 5,000 computers were used at any one time, generating a total of 2,000GB of useful data.

Based on the size of useful data GRID collected from 5,000+ machines and the quantity of pornography on my computer, they are claiming that: porn != useful.
...GRID computing; you disappoint me.

Re:Malaria?

[Soporific]

According to:

http://archive.idrc.ca/books/reports/1996/01-07e.h tml

Malaria kills quite a few people every year so I don't think it's a waste.

~S

Lots of things still out there

[Raul654]

(Preface - My research group specializes in parallel computing) There are classes of problems so computationally intensive that the computers that can do them in a reasonable amount of time won't be invented for decades. Almost all of these are simulations of physical reactions (invitro drug simulation, climate simulation, biomolecular engineering sims, physics sims, 'etc). As a general rule, these problems scale weakly (meaning that as you add more computers, you can simulate more datapoints, and get more accurate results). If memory serves, the hardest problem I can recall involved hydrogen fusion simulations, requiring computers 10-1000 times faster than the best in the world today.

Re:Wow, 25% scalability! Amazing!

[nick_davison]

Worse...

It's over 4 months, not a fraction of a second.

If I have a task that takes 100 seconds to run and I want it completed in under a second, scalability becomes a challenge... I have to figure out how to break it in to at least 100 distinct parts and deal with all of the communication lags associated. To have any kind of fault tolerance, I probably want to break it in to at least 1,000 tasks so that if one processor is running fast, it can get fed more and if one processor corrupts its process, I don't find out right at the end of the second, with no room to compensate, that I have to run re-run that full second's worth of processing elsewhere to make up for it. That's where the challenge comes in.

If I have a task that takes 100 seconds to run and all I'm trying to do is run it a lot of times over a period of time that's many times greater, I can run it 864 times a day per system with absolutely no scalability issues whatsoever and simply send the relatively small complete result sets back. With 100 systems, if each one can run a distinct task from start to finish, I'd be expecting pretty much dead on 100 times the total number crunching as there are absolutely no issues with task division, synchronization or network lag.

In this case, they ran 5,000 computers over 4 months. Assuming a single task is solvable in under 4 months by a single system, they should have had no difficult task division problems to solve, absolutely minimal synchronization issues and next to no lag issues to address. In short, even a pretty inefficient programmer should be able to approach 1:1 scalability in that easy of a scenario.

Efficiency of algorithms is a challenge when you want a single result fast. When you want many results and are prepared to wait so long as you're getting very many of them, that's an incredibly easy distributed computing problem.

Re:Malaria?

[Anonymous Coward]

Yes, I'm concerned about global warming too. But I think you're off base about malaria. Sure, there is a chance that malaria will be more prevalent. However, I think our real resources should be put to combating the animal predators from the North who will want to eat us for lunch.

First, when global warming happens, all the polar bears will come South looking for something to eat. We are probably on the top of their list. First, the bears will be real angry at us because we melted their front yard. And secondly, we happen to be the fattest creatures around--there is a lot of meat on our bones. And don't even get me started on what will happen to our shrubbery when the reindeer head this way.

I think instead of wasting CPU cycles on malaria, we instead should be using those computing resources searching for a safe but effective polar bear repellent. That way we have our priorities straight. Just my 2 cents.

Re:Malaria?

[Praseodymn]

Worrying about the environment is a luxury. Being able to do something to stop what will probably kill you is a luxury. Living anywhere because you want to is a luxury. Having a choice to take the lucid dream inducing malaria drugs or not is a luxury.

Where malaria flourishes, luxuries are scarce.

Travel as much as you can in your life, preferably to the poorer countries. They are often the happier ones.