Slashdot Mirror
Grid Computing Coming Of Age
ravenousbugblatter writes "The New York Times online has an article discussing grid computing and recent advances made by Dr. Ian Foster, among others. The article compares the state of grid computing over the internet to where the internet was in 1994, which was soon after the development of the software for the use of URL's, HTML, and HTTP. Predictions are made in the article that in the near future the massive power of grid computing will be available to anyone with an internet connection, not just to big companies that can afford to hire HP and Sun to run a grid project for them."
As a coder who works with things like md5 cracking programs (like the thingy in my sig) and various assundry other programs, I can honestly say: the crackers do NOT need any more processing power!
====
Crudely Drawn Games
We did study on size of infrastructure of Internet for demographic studies and thanks to partners at Stanford, found that Google setup now is bigger than the whole Internet in 1995 in terms of machines and total bandwidths. Grid computing definitely works..
-- Dr. Fu Ling-Yu, Internal Technology Consult; Tongji University, People Republic of China.
Grid computing isn't meant to be used for home users. It's meant to be used for computing tasks that would otherwise be run on super computers - Modelling molecular flow patterns and tectonic plate movements, to name but two. The implication that I read was not home users, but mobile users - Scientists and engineers who're out of the office and need an answer fast.
There are companies out there that would love to be able to run computationally intensive modelling, but can't afford the systems they need to get it done in a reasonable amount of time.
Stop thinking in terms of things that you would use it for, and start thinking big but not enormous. There's plenty of stuff out there.
"God, root, what is difference?" - Pitr, userfriendly
The computation performed by Seti@Home is what Grid researchers refer to as an "embarassingly parallel".
Among many other things, Grid folks hope to solve problems that aren't quite so amenable to divide-and-conquer. But then they had to go base their protocols on the bloated Web services stack, implying a relatively high granularity per compute unit. So we'll see how well that works out!
If you wanted to do this right now, you could cut a deal with a mid-range ISP. Buy an account on every server for use only during off-peak periods, run standard clustering software, and crunch all night. Run on a server farm with large numbers of identical machines interconnected with massive bandwidth. A true Beowulf cluster application.
Nobody does this. That's an indication there's no market for commercial "grid computing". Clusters, yes; reselling computer time, no.
Remember "push technology"? "Micropayments"? "Grid computing" will go the same way.
As for "peer to peer" systems, bear in mind that without copyright problems, music distribution would be trivial and cheap. Just put each new song out on Netnews. Netnews is far more efficient than any of the peer-to-peer systems. The music industry only generates a few tens of megabytes of new data per day, after all.
I've been working in the Grid Computing area for the last two and a half years, and would like to make a stand for all of us who aren't just worried about bigger supercomputers.
Supercomputers are great, but the number of big computing problems that can handle being run on distributed groups of supercomputers is small. That's why things such as the Earth Simulator and the ASCI programme still exist - sometimes it's just better to build a bigger box!
Where Grid Computing might take off in the science and business mainstream is collaboration and sharing of resources. In particular, I work on producing middleware to try and share and unify data resources. In the astronomy community for instance, they have spent many years standardising the naming schemes for their databases and as a result, projects such as Skyserver and SkyQuery are becoming possible. Now consider the bioinformatics field: hundreds of competing standards for naming things as simple as gene expression ids. Grid computing should provide some of the tools to make knowledge extraction from the many disparate scientific databases possible.
This has applications in business, and it's something we're already seeing in the uptake of Web Services. One recent Grid Computing initiative - Grid Services - is pushing the boundaries of Web Services, and extending them to standardise functionality such as state and lifetime management which should make them more useful for the kinds of collaborative problems which are cropping up in both business and science.
For instance: a car manufacturer has an agreement with different suppliers of airbags - obviously information exchange must take place to ensure safety of the passengers, but both the car manufacturer and airbag supplier will not necessarily want the other to be able to see all data for their parts, just use it. As suppliers change, the manufacturer must ensure that data is properly traced and expired. This is not much different from scientific collaborations, financial collaborations or even network gaming where we have a huge number of swiftly changing, transient resources.
It is these problems of dynamic collaboration and maintenance of resources that Grid Computing may eventually solve.