Slashdot Mirror
Grid Computing Coming Of Age
ravenousbugblatter writes "The New York Times online has an article discussing grid computing and recent advances made by Dr. Ian Foster, among others. The article compares the state of grid computing over the internet to where the internet was in 1994, which was soon after the development of the software for the use of URL's, HTML, and HTTP. Predictions are made in the article that in the near future the massive power of grid computing will be available to anyone with an internet connection, not just to big companies that can afford to hire HP and Sun to run a grid project for them."
As a coder who works with things like md5 cracking programs (like the thingy in my sig) and various assundry other programs, I can honestly say: the crackers do NOT need any more processing power!
====
Crudely Drawn Games
We did study on size of infrastructure of Internet for demographic studies and thanks to partners at Stanford, found that Google setup now is bigger than the whole Internet in 1995 in terms of machines and total bandwidths. Grid computing definitely works..
-- Dr. Fu Ling-Yu, Internal Technology Consult; Tongji University, People Republic of China.
Yeah I know, people who read slashdot aren't Joe Schmoe.
Google runs a pretty big server farm yes, it's true. I'm sure grid computing helps them immensely. I guess my point was, this won't make a public impact on Joe Schmoe.
Also I was serious, how will this affect network security?
Grid computing isn't meant to be used for home users. It's meant to be used for computing tasks that would otherwise be run on super computers - Modelling molecular flow patterns and tectonic plate movements, to name but two. The implication that I read was not home users, but mobile users - Scientists and engineers who're out of the office and need an answer fast.
There are companies out there that would love to be able to run computationally intensive modelling, but can't afford the systems they need to get it done in a reasonable amount of time.
Stop thinking in terms of things that you would use it for, and start thinking big but not enormous. There's plenty of stuff out there.
"God, root, what is difference?" - Pitr, userfriendly
The computation performed by Seti@Home is what Grid researchers refer to as an "embarassingly parallel".
Among many other things, Grid folks hope to solve problems that aren't quite so amenable to divide-and-conquer. But then they had to go base their protocols on the bloated Web services stack, implying a relatively high granularity per compute unit. So we'll see how well that works out!
I just recently heard that here at Virginia Tech we are getting a (massive?) grid comprised of some of the first dual-processor G5s rolling off the assembly line at Apple. The number of machines? I believe it was in the 1100 range.
Thus, if you like grid computing and want to do some research as a grad student or whatever, this might be the place for you.
If you wanted to do this right now, you could cut a deal with a mid-range ISP. Buy an account on every server for use only during off-peak periods, run standard clustering software, and crunch all night. Run on a server farm with large numbers of identical machines interconnected with massive bandwidth. A true Beowulf cluster application.
Nobody does this. That's an indication there's no market for commercial "grid computing". Clusters, yes; reselling computer time, no.
Remember "push technology"? "Micropayments"? "Grid computing" will go the same way.
As for "peer to peer" systems, bear in mind that without copyright problems, music distribution would be trivial and cheap. Just put each new song out on Netnews. Netnews is far more efficient than any of the peer-to-peer systems. The music industry only generates a few tens of megabytes of new data per day, after all.
I have been watching the developement of one such application: Gled , "a hierarchic server-proxy-client-viewer model written in C++ and offering a mixture of object oriented framework and toolkit" (says the project homepage) and I can say that it looks a lot more like a Quake window to a programmable scene made of very complex object collections, running on multiple systems (and with multiple users) with a GUI to its underlying cluster systems, than a Seti@Home screensaver.
My personal favourites are the autogenerated code, the autogenerated GUI and the object brokering facilities over the clusters.
The trend of Grid and Grid-like cluster computing is, IMHO, going in the direction of better viewing facilities, more interactive software and higher-level interfaces, where the underlying grid can be thought of as a piece of iron, a strange dynamic multiprocessor arhitecture with impossible latencies.
Links:
-Kvorg
I think grid computing is way overdue. I am not disparaging the current researchers in the field. Resource sharing and management has been with computers from day one (one of the quotes from the talk is from 1969). Ever since I discovered PovRay (it was DKB or something back then) I have been waiting for fast computers available for my use without muss or fuss. I'm still waiting.
I've been working in the Grid Computing area for the last two and a half years, and would like to make a stand for all of us who aren't just worried about bigger supercomputers.
Supercomputers are great, but the number of big computing problems that can handle being run on distributed groups of supercomputers is small. That's why things such as the Earth Simulator and the ASCI programme still exist - sometimes it's just better to build a bigger box!
Where Grid Computing might take off in the science and business mainstream is collaboration and sharing of resources. In particular, I work on producing middleware to try and share and unify data resources. In the astronomy community for instance, they have spent many years standardising the naming schemes for their databases and as a result, projects such as Skyserver and SkyQuery are becoming possible. Now consider the bioinformatics field: hundreds of competing standards for naming things as simple as gene expression ids. Grid computing should provide some of the tools to make knowledge extraction from the many disparate scientific databases possible.
This has applications in business, and it's something we're already seeing in the uptake of Web Services. One recent Grid Computing initiative - Grid Services - is pushing the boundaries of Web Services, and extending them to standardise functionality such as state and lifetime management which should make them more useful for the kinds of collaborative problems which are cropping up in both business and science.
For instance: a car manufacturer has an agreement with different suppliers of airbags - obviously information exchange must take place to ensure safety of the passengers, but both the car manufacturer and airbag supplier will not necessarily want the other to be able to see all data for their parts, just use it. As suppliers change, the manufacturer must ensure that data is properly traced and expired. This is not much different from scientific collaborations, financial collaborations or even network gaming where we have a huge number of swiftly changing, transient resources.
It is these problems of dynamic collaboration and maintenance of resources that Grid Computing may eventually solve.