Slashdot Mirror
Grid Computing: Conceptual Flyover For Developers
An anonymous reader writes "This article relates many Grid computing concepts to known quantities for developers, such as object-oriented programming, XML, and Web services. The author offers a reading list of white papers, articles, and books where you can find out more about Grid computing."
Recently I saw a similar design for a network and some "old timers" said it was no good to do it this way. It wouldn't satisfy the needs.
One thing I have noticed is that for many "old timers" there is the feeling of we have always done it the old way, why change. Any thoughts of how we drag that old donkey into the new methods when they don't want to go?
Evolution or ID?
As part of a university group that adopted Grid computing about a year ago, the Grid is mostly over-hyped material that isn't ready for prime time. The basic idea (see e.g. Legion) worked more than a decade ago, but what I've seen of today's Grid software is fragile, overcoupled, underdocumented, and doesn't yet deliver on all the promises.
:)
We were taught that the test of research software is whether a full professor (or corporate executive or other obscenely busy person worth >> $100/hour) finds it useful enough that they take time to learn it - the uses I've seen for the Grid don't pass that threshold yet.
There are some exceptions: tightly-integrated applications put together in a couple of the hard sciences that really just do supercomputing with a friendlier face. There's enough payoff there for a physicist to be happy with the software.
For a geek, however, even there, most "grid UI research" is simplistic, derivative, and uninspired.
Apologies to my first-ever-advisor who is now a Grid bigwig.
I have had some experience with grids and the overwhelming difficulties I've come across have been in the areas of security.
First and foremost, grids are designed to run in a distributed environment which makes security design and administration that much more complex.
Second, grids are currently in their infancy and there is little prior art to the types of attacks and problems that will affect them. Despite this, they are very juicy targets with the kind of storage and bandwidth that would make even a hard-bitten cracker weep for joy. (i apologise for the imagery)
Third, in my book security has to be a top-down approach - i.e. the guys on top lead the way and then everyone else follows. Grids have no tops or bottoms which makes this a bit tough to apply. In short there is no security hierarchy in a default grid environment. Responsibility HAS to be established explicitly. A simple example is who is responsible for the data held on one of the nodes? Is it the person who wrote the application, the person who owns the application, the person who owns the hardware?
Grids are fascinating in their security requirements (and those who think these are solved by web services have another thing coming! People are a huge aspect of the security of a system, and distributed system like grids have a very complex task of ensuring that people behave the way they should).
Keep going:
Consider this: most IT departments are being forced to do more with less. Budgets are tight, resources are thin, and skilled human resources can be scarce or expensive. To top it off, most corporate managers know that they have a super-abundance of idle computing power. It's well known in industry circles that most desktop machines only use 5% to 10% of their capacity, and most servers barely peak out at 20%. No surprise then that many of the big money people in corporate America balk at the thought of purchasing more equipment to get the job done. What these companies need is not more horsepower, but more efficient use of existing horsepower. They need a way to tie all of these idle machines together into a pool of potential labor, manage those resources, and provide secure and reliable access to the number-crunching muscle. Imagine if a corporation or organization could use all of its idle desktop PCs at night to run memory- and processor-intensive tasks? They would get more work done faster, possibly get to market faster, and at the same time cut down their IT expenses.
The idea seems to be to turn the whole network into a cluster. "Why buy more servers when you can gove some of the load to your desktops?" is a short summary.
Actually I always thought the name came from the concept of power grids. i.e. plug in your application to the computing grid, get it to run your computations, and get the results without having to worry about how the computing power got to your home...
Kind of similar to a power grid no? plug toaster, insert bread, get toast - no need to worry about coal/oil/nuclear fuel burning, transfomers or megawatts...
As an "boots on the ground" IT professional it would be nice to have a consumer grade "grid computing" solution to offer some small business customers as an alternative to buying a server farm for the two days a month they actually put strain on it.
If there were an easy way to cluster their workstations they wouldnt need to invest in an underutilized server farm. They could just schedule their processor/disk intensive reports and processes for off hours or rely on grid load balancing to take the extra cycles from the computer of the CSO (Chief Solitaire Officer) so that the impact would be imperceptible to the average user.
The current problem with the concept of grid computing is the lack of an easy way to deploy it in a standard business environment. What the article and its links are driving at is coming up with a cheap and easily implimented mechanism to turn every office, and chain of offices into a grid.
In theory, you could sell your unused processor cycles the same way people who generate their own power sell power back to their power companies. You ISP could actually, someday become a processor cycle reseller and you could operate on a minimal set of hardare in the typical office enviroment becuase you can always pick up extra cycles from your ISP when you need them.
Ah, the pipe dream.
Hosting companies have large numbers of identical machines with high bandwidth interconnects. That's just what you want for "grid computing". They're already set up to allow customers to run applications on their machines, and are able to deal with the security problems. Load is very low during off-peak hours. The machines stay up; they don't suddenly get disconnected from the net because somebody turned their desktop off. They're all loaded with the same base software. It's the ideal situation for commercial "grid computing".
So why is nobody selling this? Because there's no market for it. There's no real commercial market for supercomputer time, distributed or otherwise. Once upon a time, from about 1960 to 1980, there were engineering computer service centers, where you bought time-sharing service on big mainframes. Control Data and UNIVAC were the preferred machines for this. But that business is dead. CPU time became too cheap.
A well-known commercial grid was Gateway Processing on Demand, announced in late 2002 with great fanfare. Gateway offered "grid computing" on thousands of Gateway-owned machines. They quietly dropped that service some time last spring. Their former CEO admitted that it generated "not a lot" of revenue. Basically, it was an attempt to generate some revenue from Gateway's unsold inventory of machines.
Grid computing is one of those schemes where all the interest is on the sell side. Nobody wants to buy it. "Micropayments" and "portals" are like that. They didn't sell either.
How is GRID different from these methods?
Grid is a set of capabilities which must satisfy requirements at a much higher conceptual level. Whereas we might say that DCE and CORBA define the atomic structure, Grid provides the biochemistry.