Slashdot Mirror
Grid Computing Coming Of Age
ravenousbugblatter writes "The New York Times online has an article discussing grid computing and recent advances made by Dr. Ian Foster, among others. The article compares the state of grid computing over the internet to where the internet was in 1994, which was soon after the development of the software for the use of URL's, HTML, and HTTP. Predictions are made in the article that in the near future the massive power of grid computing will be available to anyone with an internet connection, not just to big companies that can afford to hire HP and Sun to run a grid project for them."
Its the 4th one, and getting better every year.
please consider setting up Grid Computing section! ...so that I can finally filter it!
Thanks in advance.
Welley Corporation - SLM Scammers
Snap to Grid2003! We Rule!
The article from NYTimes requires (free) registration.
Here is the registration free URL
Please use news.google.com for finding article links.
As a coder who works with things like md5 cracking programs (like the thingy in my sig) and various assundry other programs, I can honestly say: the crackers do NOT need any more processing power!
====
Crudely Drawn Games
Been there. Done that. Nothing more to prove.
>> Imagine a Beowulf cluster of these!
Wow! Imagine your karma going down!
Can anyone see another player apart from Microsoft having the market penetration required to make themselves the defacto distributed computing platform??
Go Google I say - let microsoft get someone else to beta test their software.
Q.
Insert Signature Here
Sure this is great if you're doing simulations or animating/rendering stuff . But for Joe Schmoe who surfs the web and reads his e-mail, what's the big deal? How will this affect network security?
Shouldn't the above post be labelled Redundant?
...that bastard Scott is right?
The network really is the computer?
Where'd I put that mousepad......
Would anyone who *has* registered care to post the text of the story?
"It was the Woodstock of the grid -- everyone not sleeping for three days, running around naked and shagging in a kind of scientific performance art," said Dr. Larry Smarr, director of the California Institute for Telecommunications and Information Technology, who was the program chairman for the conference.
no wonder it took so long to develop.
I am a leaf on the wind
It's about time grid computing become of age. I've been waiting to hit that for years.
Sincerely,
A dirty old mainframe
Consensual sex is boring.
We did study on size of infrastructure of Internet for demographic studies and thanks to partners at Stanford, found that Google setup now is bigger than the whole Internet in 1995 in terms of machines and total bandwidths. Grid computing definitely works..
-- Dr. Fu Ling-Yu, Internal Technology Consult; Tongji University, People Republic of China.
What would be the difference between Grid Computing and then a "psduedo-cluster" networked together with VPN Tunnels?
From the article they seem to be basically the same thing. Or am I wrong?
Error 407 - No creative sig found
no more upgrades every 3 years !
This is great if you think it's great. Grid computing is a technology without a cause right now. It's preposterous to think that the average joe, or even the average joe company, will have any use for grid computing in the forseeable future. Most of us can't keep our load average above 0.1 (that's 10% for you Windows-users) doing anything useful as it is!
Heck, look back over the grid computing stories we've seen here on /. Whose name keeps popping up?
SKYNET LIVES!!!!
Tsk tsk...what a blatant example of a violation of DMCA right here on /. This so misrepresents the law-abiding nature of the /. population ;-)
h-t-t-p-colon-slash-slash-slash-dot-dot-org
Imagine a beowulf cluster of thes^H^H^H^H^H^H^H^H...
For more info on porgramming for grid computing try MPI or LSF/NQE/PBSPro
Rus
Cheap UK and US VPS
With the upcoming PS3 carrying out grid computing, there's no stopping for this technology reaching out to the masses, even those who don't know it!
h-t-t-p-colon-slash-slash-slash-dot-dot-org
I prefer the term distributed computing, why did distributed computing turn in to grid computing?
I just recently heard that here at Virginia Tech we are getting a (massive?) grid comprised of some of the first dual-processor G5s rolling off the assembly line at Apple. The number of machines? I believe it was in the 1100 range.
Thus, if you like grid computing and want to do some research as a grad student or whatever, this might be the place for you.
I thought 1960s timesharing was nothing more than job submission and process load balancing run on supercomputers. Grid computing can involve 1000s of distributed nodes with millisecond (ethernet) latencies between them, that's not timesharing as far I know. The wheel isn't being reinvented here.
If you wanted to do this right now, you could cut a deal with a mid-range ISP. Buy an account on every server for use only during off-peak periods, run standard clustering software, and crunch all night. Run on a server farm with large numbers of identical machines interconnected with massive bandwidth. A true Beowulf cluster application.
Nobody does this. That's an indication there's no market for commercial "grid computing". Clusters, yes; reselling computer time, no.
Remember "push technology"? "Micropayments"? "Grid computing" will go the same way.
As for "peer to peer" systems, bear in mind that without copyright problems, music distribution would be trivial and cheap. Just put each new song out on Netnews. Netnews is far more efficient than any of the peer-to-peer systems. The music industry only generates a few tens of megabytes of new data per day, after all.
Timesharing is multiple people/jobs connecting to one mainframe or computer and "sharing" the useage of that single computer. Grid computing (aka distributed computing) is sharing one job over multiple computers. Totally different concept, completely different target audience.
now how did it go again? Ah, thats right....
*points and laughs* hahahahahahahahahahahaha
(I have mod points, but there is no -1 WRONG.... so I decided to take the piss instead
IBM, as well as other companies and organizations, is working on the Globus project and there are different scenarios out there. One of which is online gaming (butterfly.net). There are others, but right now, it's mainly scientific based. For the people who know, the Globus toolkit just reached version 3. This is important to know because this version is OGSA/OGSI (in draft) compliant which is an open standard describing the communication between the grid nodes (WSDL, WSDD, XML, etc). The grid is different from clusters and the grid is different from p2p computing. One view on the grid is use of remote resources. For instance, you can use en electron microscope remotely. Perhaps even with the DaVinci surgical machine, it would be possible to perform (minor) surgery remotely. The advantages of this are obvious. A specialist can help more people since it cuts on travel time. In my view, the grid cannot be applied to a certain solution meaning the grid isn't supposed to be for a certain problem, but rather, a new avenue to do things. With this in mind, the grid will grow according to how we think of using it. The Internet is an example of this type of growth. Furthermore, the grid is probably geared towards businesses and other backend operations. Later, perhaps, it'll become more of on online service directory in which you can find resources to do you work; printing facility, specialized resource use (super/quantum computers), and other things. Again, the grid will grow will become what we need it to be (even if we cannot predict it).
All aside, it's exciting technology, not to mention that the Globus toolkit was named on the of the top 10 techs that will change the world.
For those interested in security, the Globus toolkit involves an asynchronous certificate signing method initially and then move onto a synchronous method for better performance. The Globus books and papres call this a PKI scneario. (I'm not a security guy)
Also Globus is not the only grid tech out there. Seti at home is one (and their derivatives). There are also distributed storage methods in which you can send data onto the grid and it'll be there (somewhere safely tucked away).
fun stuff!
Also, Globus is open source and the services are written in Java. I just love this stuff.
I really think the people complaining about not personally having a use for grid computing are completely missing the point. As long as enough people have a use for it, it will be useful. Having done a good number of calculations on a few different supercomputers, I can think of nothing that the grid currently offers to me...but I'm sure the people who run many-hundred processor jobs on a regular basis have a different perspective. For a while, the grid might be the plaything of big scientific and industrial computational projects, but has any technological advancement like this ever not caught on. Eventually, someone will figure out a new idea, only possible on a grid, which involves porn, gaming, or the ability to transfer media files in a manner of questionable legality, and soon kids will be asking what life was like without it back in the dark ages. A little patience, people, give the geniuses and madmen (not necessarily mutually exclusive) a little time to work...
I have been watching the developement of one such application: Gled , "a hierarchic server-proxy-client-viewer model written in C++ and offering a mixture of object oriented framework and toolkit" (says the project homepage) and I can say that it looks a lot more like a Quake window to a programmable scene made of very complex object collections, running on multiple systems (and with multiple users) with a GUI to its underlying cluster systems, than a Seti@Home screensaver.
My personal favourites are the autogenerated code, the autogenerated GUI and the object brokering facilities over the clusters.
The trend of Grid and Grid-like cluster computing is, IMHO, going in the direction of better viewing facilities, more interactive software and higher-level interfaces, where the underlying grid can be thought of as a piece of iron, a strange dynamic multiprocessor arhitecture with impossible latencies.
Links:
-Kvorg
I am subject of racisms on Slashdot. I can not tell you who I am but parent poster is nonsense speaker. He has obsession with my name!
Clustering is to Grid (or distributed)
as
LAN is to WAN
or am I just fooling myself?
He tried to kill me with a forklift!
"Grid" computing has been around for decades. People have been doing useful work with arrays of systems for a long time, both in the commercial and academic areas.
Government of the people, by corporate executives, for corporate profits.
I've been working in the Grid Computing area for the last two and a half years, and would like to make a stand for all of us who aren't just worried about bigger supercomputers.
Supercomputers are great, but the number of big computing problems that can handle being run on distributed groups of supercomputers is small. That's why things such as the Earth Simulator and the ASCI programme still exist - sometimes it's just better to build a bigger box!
Where Grid Computing might take off in the science and business mainstream is collaboration and sharing of resources. In particular, I work on producing middleware to try and share and unify data resources. In the astronomy community for instance, they have spent many years standardising the naming schemes for their databases and as a result, projects such as Skyserver and SkyQuery are becoming possible. Now consider the bioinformatics field: hundreds of competing standards for naming things as simple as gene expression ids. Grid computing should provide some of the tools to make knowledge extraction from the many disparate scientific databases possible.
This has applications in business, and it's something we're already seeing in the uptake of Web Services. One recent Grid Computing initiative - Grid Services - is pushing the boundaries of Web Services, and extending them to standardise functionality such as state and lifetime management which should make them more useful for the kinds of collaborative problems which are cropping up in both business and science.
For instance: a car manufacturer has an agreement with different suppliers of airbags - obviously information exchange must take place to ensure safety of the passengers, but both the car manufacturer and airbag supplier will not necessarily want the other to be able to see all data for their parts, just use it. As suppliers change, the manufacturer must ensure that data is properly traced and expired. This is not much different from scientific collaborations, financial collaborations or even network gaming where we have a huge number of swiftly changing, transient resources.
It is these problems of dynamic collaboration and maintenance of resources that Grid Computing may eventually solve.
Use those unused CPU cycles in the search for disappearing African Uranium!
when is bandwidth going to be so available/cheap that this is going to matter?
Just in case someone's not familiar with parallel programming terminology, embarassingly parallel means that each operation is independant of any other, so it can scale virtually perfect. The classic example:
1 women * 9 months = 1 baby
9 women * 9 months = 9 babies
9 women * 1 month != 1 baby
Putting 9 women on the baby-making task for 9 months is scales the baby-making operation, but since it's embarassingly parallel, it doesn't speed up any one operation (each baby still takes 9 months, but in those 9 months, you can now get 9 babies).
Another example (I made up the 10 factor):
1 worker * 10 days = 1 car
10 workers * 10 days = 10 cars
10 workers * 1 day = 1 car, because they can work on seperate things at the same time.
Of course, in reality, they can't quite work on things at the same time (steering wheel can't be installed before the body is welded together), so you have operations that block, leading to the steering wheel guy sitting around doing nothing, but you get the idea. This, of course, is where pipelining comes in...
-- Is "Sig" copyrighted by www.sig.com?
I fully agree with you 100 %. And tell me about it. And what do you think of this site . Random NYTimes.com Registration Generator http://www.majcher.com/nytview.html
You go here and it fills the NYTimes registration page with random characters. Maybe it might come handy some day, but, oh god, you must be pretty angry about it. Yeah. And tell me about it. I agree with you fully 100%.
To see a world in a grain of sand, and then to step back and see the beach where the sand lies
As a student I have my disk space regulated, am I soon to have my CPU time regulated (this used to happen). I don't know whether I would end up with more CPU time available to me for a project or less?
Mmm, your example is incomplete, no ammount of woman can produce babies by themselves...
/.'er * 9 months = 1 harrasment suit
1 woman + 1 man * 9 months = 1 baby
1 woman + 1
etc.
"Semper in excretum set alta variant"
The "Grid" prescibes a standard way of connecting distributed computers for large-scale computing. Sort of like the InterNet didnt really take off until more than 20 years after its inception, when http and browsers became standards.
.. hook it up to the grid and have the power of 500 petaflops.
Yes, the Grid can provide true parallelism, but it doesn't just do that by itself; the programmer has to divide the job up into tasks and distribute them. What the Grid provides is the framework to make that possible, and the security infrastructure to make it more likely that the owners of compute servers will allow other people to use their resources.
The computing model really is the mainframe one: you submit jobs to a queue and they get run and come back to you. You don't care or know when or where they get run. This model may be ancient, but it's how much of "business computing" is done.
Are grids and clusters the same thing? From what I've read/heard they seem very similar in that both involve large networks of computers working together and parallel processing, but are there any major differences?
Yes yes it's off topic, but after having seen T3 it has been bugging the hell out of me that at the end of the movie John said that there was no central computer for Skynet, but instead it had spread into thousands of machines on the internet. Isn't that fantastic!
And then I thought, well, if Skynet is actually a distributed computing app running on computers on the internet, then, when Skynet decided to blow up all of the cities, wouldn't it also be destroying the majority of its compute nodes and thus massively diminish its capacity. It would, after all, be living in the same cities as you and I. Not to mention that computers rely on the same infrastructure (power grids, telecom) as we do.
Ok, so I am anal, but a plot hole large enough to launch the shuttle through does catch my attention and drive me crazy for a while.
Mmm, your example is incomplete, no ammount of woman can produce babies by themselves...
1 woman + 1 man * 9 months = 1 baby
1 woman + 1
Ah, true. You see, very few people on
-- Is "Sig" copyrighted by www.sig.com?
Just to let you know: there was a lengthy (but very interesting) presentation by Rüdiger Berlich from the Forschungszentrum Karlsruhe about GNU/Linux' role in the GRID computing area.
:)
His speech was very informative for me since I did not know there are that many scientific interests connected with GRIDs...also he gave some nice insights on the why of GRID computing - ever tried to cope with an annual dataflow of one petabyte?
The paper from his speech was on the Linuxtag DVD, I think it's ok if I let you have it.
Presentation from Rüdiger Berlich about GRID Computing and Linux, given at the LinuxTag2003.
I don't get it...the man sits around for 9 months and then a woman comes along and poof a baby is born?
(1 woman + 1 man) * 9 months = 1 baby
Order of operations man! Gotta get those parenthesis right.
link
33 of those dead are since the war was 'over' on May 1st.
I'd have to remind you that statistically, 36 is a very small number for a strategist. Any military leader would consider it minimal acceptable losses.
That comes from the cold side of me. The half that occupies my heart says that while I agree with you that it is a tragedy, your trolling is amatuer and smallish. Reminding people of the circumstances and results of war does nothing to make anyone's grief easier and places no strain on the leaders that make such horrible decisions. Your words are better spent lobbying and writing for your congressman. Not on Slashdot with offtopic agendas.
It's rather disconcerting to see someone with good intent waste it on such obviously poor application and execution. Atleast when I'm off-topic, I try to elicit a discussion. But I do so love flamebaite. It taste so good in the morning.
"Last one in is a rotten goblin!" - Kepp
Yeah but Butterfly sucks - I saw their schtick at GDC. All smoke, no fire. If this is the future of gaming then we are all going back to our Nintendos.
While I do understand the baby making process quite well. I am not actively pursuing it as an option right now.
Thus far I have quite happilly stuck it out at step 1.
Sadly it seems that's about to end. My girlfriend of two years left me this morning....
life's a bitch.
"Semper in excretum set alta variant"
The car example is interesting, in that 1 worker will take
much much longer than 10 days to make a car, although
10 workers can easily make a car in 1 day, because no
one worker has the easy facility with all 10 of the distinct
production roles in producing a car. This is counter to
the argument of the skeptic of parallelism who claims that
Amdahl's law makes parallel computation inherently
inefficient, and points to what may be the single most
appealling aspect of the "Grid" paradigm for distributed
computing, to wit:
As opposed to clustering or even p2p, "Grid" is (albeit in a
vague and unfocussed manner) begining to amalgamate
diverse and disparate resources which can be used to
construct productive pipelines in which each contributing
system's peculiar strengths are more efficiently utilized
than is possible in a homogenous system, where the
imbalance in the resource uses of a particular application
almost *never* matches the imbalance in the resource
supply: The Grid is subsetted so that the global system
implementing a production pipeline is efficiently utilized,
or, to flip the coin, the intrinsic underutilization of
an inefficient pipeline construction does nothing to
harm the grid -- it only means that the remaining
unutilized capacity is available to other dynamic subsets,
to be utilized in other production pipelines.
-I like my women like I like my tea: green-
Man is optional.
-I like my women like I like my tea: green-
I don't get it...the man sits around for 9 months and then a woman comes along and poof a baby is born?
(1 woman + 1 man) * 9 months = 1 baby
Order of operations man! Gotta get those parenthesis right.
Not exactly, because you really only need the man for a few seconds (though I pity the guy who's that quick). So perhaps:
(1 woman + 1 man) * 30 seconds + (1 woman * 9 months) = 1 baby?
Who knew sex could be made so boring?
-- Is "Sig" copyrighted by www.sig.com?
You could have read Ian Foster writing about grid computing in April's Scientific American. If you can't find a copy, try sciam.com