Grid Computing and IBM

cozimek writes: "I just read this article from the NY Times that discusses a plan by IBM to leverage their support of the Linux platform to build grid computing. IBM has already won support of grid projects for supercomputing in England and the Netherlands, and now seems ready to take on the Internet. Of course, the article says it could be many years before we see any fruits of this bounty." This has been submitted many times, so we're posting it. But somehow I resent the fact that it's just a vaporous press release generating this hype, taking advantage of a well-known idea that many are already working on and was forecast many, many years ago.

Comments

Moore's Law versus Grid Computing

[Doug Merritt]

Grid computing was hyped back in the 1960s (Multics was targeted towards that, IIRC). It never happened because of Moore's Law. Cheap processors means it's been more cost effective in most cases to buy your own cpu than to lease it from a grid.

A science fiction novel I read recently (Permutation City by Greg Egan), however, reminded me that this may eventually change, if and when Moore's Law stops working.

If compute power hits a stable plateau in 10, 20, 100 years, whatever, then the cost of compute power will also roughly become a constant number of dollars per clock cycle (or peta-clock cycle).

In that case, as Egan presents it, compute power from a global grid may indeed be the only way to get larger amounts of compute power than your local processor can give you, and therefore, as a commodity, it may go to the highest bidder at any given moment.

(Hopefully not so badly as with California's power grid bidding, but we'll see.)

P.S. the advent of nanotechnology computers, or quantum computers, or purely optical computing, etc, wouldn't dispel the above scenario, it would just delay it. It's not clear that even Vinge's Singularity would literally prevent Moore's law from going away. (I don't believe that the Singularity will do away with the laws of physics.)

Re:Moore's Law versus Grid Computing

[dpilot]

> If compute power hits a stable plateau in 10, 20, 100 years, whatever, then the cost of compute power will also roughly become a constant number
>of dollars per clock cycle (or peta-clock cycle).

IMHO, we're very close to this point, if not there already. But in a different way. Consider this an economic limit, not a technological one. We can keep shrinking chips, but it keeps getting more and more expensive to do so.

The first hint came with the sub-$1000 computer. Prior to that, a top-end PC was about $2000-$3000, with a lower priced PC about $1500-$2000. We kept buying all the power we could afford. But with the sub-$1000 computer a class of users began buying all the power they NEEDED, and let the cost ride down. More expensive PCs became the tools of gamers and technical use, and Microsoft was the only force pushing basic compute power upward.

I'd like to upgrade to a 1.5 GHz Palomino this Fall, about my normal schedule, but times are tight, so I'm probably going to pass for another year. (Maybe a Hammer, then!) And to look seriously at it, my K6-3 does just about everything I ask of it. Star Trek Elite Force runs great, RealMyst was lackluster, though.

Who does what?

[rho]

I'm not sure I understand -- who provides this "grid"? Are they built and maintained by IBM around the world? I don't think IBM would be thrilled to discover that Compaq is using the IBM grid to advance Compaq's bottom line. I like IBM, don't get me wrong -- but I doubt they're such humanitarians.

Is the "grid" made up of PCs on the Internet? First, most of those PCs are on dial-up connections, making things very complicated (and the PCs themselves not very useful). Second, who compensates the people who own the PCs? Is it strictly voluntary, like SETI@home? If so, how will anti-nuke activists prevent Los Alamos from running simulation calculations on their PowerMac?

I think the idea is fantastic, but I'd like a few more details..

Re:Who does what?

[hyrdra]

I'm not sure I understand -- who provides this "grid"? Are they built and maintained by IBM around the world? I don't think IBM would be thrilled to discover that Compaq is using the IBM grid to advance Compaq's bottom line. I like IBM, don't get me wrong -- but I doubt they're such humanitarians.

Think Internet. Right now, we're paying for bandwidth, because the Internet is largely an information-only medium. However, in the future, we will also be able to have a certain amount of processing power, shared by everyone, used by everyone. IBM is just providing the structure (and at first the systems for the demo) to access mass computational resources. Soon, you will be able to access network wide applications which are processed on many machines across the network in a distributed way.

Right now we have an enormous processing surplus. Most machines sit unused for hours. Check your load averages if you don't believe me. Even a personal desktop used 8+ hours a day will barely break a few percent. Now imagine if we had some infrastructure, which is what IBM is aiming to do, to harness and unite all this power for general use? We would have an enormous amount of processing power available.

Is the "grid" made up of PCs on the Internet? First, most of those PCs are on dial-up connections, making things very complicated (and the PCs themselves not very useful). Second, who compensates the people who own the PCs? Is it strictly voluntary, like SETI@home? If so, how will anti-nuke activists prevent Los Alamos from running simulation calculations on their PowerMac?

Bandwidth will come in time. Even so, imagine having all of AOL's dialup connections available for processing. 56k isn't that much, but imagine millions of connections at once. As soon as we get lots of bandwidth and always-on connections wide-spread, this will be much easier. It's an upgrade path too. We can still start now and as people get faster connections and faster machines, the overall system power will increase.

As far as compensation, this is a public thing. We all use each other's resources, and we all contribute to the available processing resources. The sum of the parts of something are greater than one part alone, working alone. Similar to how Gnutella users each contribute and take, and why it works so well. Just translate the information into processing power. You can take as much or as little as you want, most people falling somewhere in between (this is how it always is and is a regular pattern).

I'm sure there are going to be leeches. But many people will want to share because they realize how the system works. Distributed systems like Gnutella do work (albeit a few leeches here and there), and this is proof that a processing system will also work.

Yes, read the article (and its references)

[FreeUser]

The intro posted is not correct. The article says that Grid software infrastructure is being developed on the "open source model," it does not say that it incoporates Linux (although I'm sure Linux will be a major OS used with it). MS has also contributed $1 million to the effort, and hopes to tie in .NET

The intro is absolutely correct, which if you'd done any digging whatsoever *cough*google*cough* you would have found for yourself:

It will be based on Globus and Linux software which uses the internet as an underlying communication system. [Atkinson, 2001]

It really can't be stated much more clearly than that.
--

BBC Article

[Maran]

Here's a link to the BBC Article. Maran

Sun is already there!

[Arethan]

Enter Sun Grid Engine

And yep, it's free!

Vaporous, but still gives it exposure...

[baptiste]

I agree its a vanilla corporate release, but its good news. A lot of people don't even know what grid computing is. This can help spread the word of yet another excellent OSS project

I had heard of grid computing before, but hadn't read much about it. Google turned up lots of resources this mornign - worth teh read. The article was right - the software to manage a grid will be super complex and the security implications are daunting.