Slashdot Mirror
TeraGrid v. Distributed Computing
Nevyan writes "After three years of development and nearly a hundred million dollars the TeraGrid has been running at or above most peoples expectations for such a daunting project. On January 23, 2004 the system came online and provided 4.5 teraflops of computing power to scientists across the country. However, the waiting list for TeraGrid is long, including a bidding process through the National Science Foundations (NSF's) Partnerships for Advanced Computational Infrastructure (PACI) and many scientists with little funding but bright ideas are being left behind. While the list of supercomputer sites and peak power is growing how is the world of Distributed Computing faring? "
The problem with using distributed computing for everything is that the number of people willing to let others use processing power on their computer is not infinite. It is a very large number, but eventually everyone who wants to/knows how to help out their favorite cause will have something already installed. In addition, the more useful endeavors that use distributed computing, the less users you will get for each, and only the 'interesting' projects will get many users. Who wants to use their computing power to analyze some boring old physics experiment when you could be finding aliens or curing cancer?
Distributed computing has its uses, but remeber: the public will only be willing to help you as long as they feel like they're contributing to something worthwhile.
Is for a different kind of distributed computing client, one that allows you to sign up for different kinds of research programs. For example, you could say "donate half my spare time to aids research, and 1/4 to math reserach, and 1/4 to seti research". Also integrate a method of possible payment for work units completed (and a checking process to remove cheaters) and I think you will have an increase in effeciency in the entire way that we treat computers. Maybe instead of everyone shelling out thousands for top of the line computers whose peak output they only need for 5% of the time, they shell out a lot less for a networked computer that buys time from other people's machines. Clearly this wouldn't work in all applications (particularly those requiring low latency) but with improving network connections I think this is a possible future.
I submitted this story last night, and it didn't get posted.
...some day there will be available computing solutions powerful enough be themselves not to need distribution of the computations...
There have been big projects like SETI@home, Great Internet Mersenne Prime Search, RC5-64 and many others.
There are some like the Casino-21 http://www.climate-dynamics.rl.ac.uk/ and Evolution-at-Home http://www.evolutionary-research.org/ too.
It's becoming easier to create the required code for distributed projects, and it most certanly has become easier to actaully get them distributed.
But will it run Longhorn?
--If the world didn't suck, we'd all fall off.
Important to remember that the Grid is a _kind_ of distributed computing. But the main thing about The Grid (like The Internet, The Grid is basically TeraGrid in the US + European Data Grid) is that it is suitable for handing off parallel jobs with high intercommunication needs to (i.e. MPI jobs). Not necessarily because these jobs can run across different nodes of the grid (though they can with MPI/Nexus or whatever it's called), but because each "node" in the Grid network is a HUGE MOFO LINUX CLUSTER or similar. The grid gives lots of physicists access to computing resources for parallel processing jobs that would otherwise be sitting idle.
/.ers generally mean by distributed computing is a bit different - most apps there are "embarrassingly parallel" ones you can just farm out. They don't need to chatter to eachother, just process some data and send it back to Central.
What
Google's distributed OS has been discussed a lot on Slashdot, but it is more than just a search algorithm on their own servers:
Google Compute is a feature of the Google Toolbar that enables your computer to help solve challenging scientific problems when it would otherwise be idle. When you enable Google Compute, your computer will download a small piece of a large research project and perform calculations on it that will then be included with the calculations performed by thousands of other computers doing the same thing. This process is known as distributed computing.
The first beneficiary of this effort is Folding@home, a non-profit academic research project at Stanford University that is trying to understand the structure of proteins so they can develop better treatments for a number of illnesses. In the future Google Compute may allow you to also donate your computing time to other carefully selected worthwhile endeavors, including projects to improve Google and its services.
- The Google Compute Project
Distributed computing has its uses, but remeber: the public will only be willing to help you as long as they feel like they're contributing to something worthwhile. Uh, I'm not sure what this has to do with the TeraGrid . . . The TeraGrid is a distributed computing system . . . but it does not use the "public's" computers. It uses university and computing center machines across the USA (e.g. NCSA, Argonne National Labs, Purdue, etc.) .
...Anonymous Cowards!
The idea of payment for work units is interesting. While it would certainly provide incentive for participating in distributed computing projects, I can see two problems with it already:
1) Getting the money to pay people. One advantage of distributed computing is that you don't have to pay for time on expensive cluster. That advantage disappears when you pay distributed computing users. Of course, it may still turn out to be cheaper, and there may be users willing to participate for free.
2) Botnets and profit. We all know of spammers using zombies to peddle goods, and of script kiddies using them to DDoS. What if some enterprising but immoral person decided to use the computing power of his zombies to profit off of the distributed computing payments? With enough zombies, he could easily make a good amount of money off of other people's computers.
If you can divide your problem into very many independent subproblems, clustering or distributed computing will work well. If not, your best bet is a true supercomputer.
So: SETI@Home splits up its scans into sections, each of which do not depend on any other; therefore, a distributed solution is efficient. However, the Earth Simulator deals with chaotic systems (or so I would assume), which do not independently parallelize; this is where having hundreds of processors and terabytes of RAM and using something like NUMA is greatly more efficient.
In short: use the right tool for the job.
The problem with large projects like TeraGrid, EarthSimulator and other supercomputer sites is that the underfunded _brilliant_ ideas are left behind by those who can afford to pay for or build these centers and sites.
While TeraGrid is a powerfool tool it is one that thousands of scientists and laboratories are standing in line to use. Meanwhile Distributed Computing is available, cheap and relatively quick.
While it may look good on your project to say you used a IBM BlueGENE or DeepComp 6800 is it really worth the extra cost and waiting in line for your chance to use?
True Distributed Computing is the way to go and shows positive results. Now we just need to tinker with it some more!
I don't understand why we are asking how a hammer is doing compared to a screwdriver? Both are varied computational models, and are at best architectural descriptions as titles; TeraGrid v. Distributed Computing. They have specific application domains and are used to solve different types of problems. One dealing with non-discrete data and experimental calculations (TeraGrid), the other focused on discrete chunks of data being filtered or rendered and are non-time nor message dependent (Distributed Computing; as defined by the Nevyan's reference). You have two tools in your tool chest. What makes one better than the other? They have completely different jobs that they tackle. They both will be successful. They need not be in competition.
4.5 Teraflops for $100 million? Surely not. That much compute power can be had for 1/20'th the price. What am I missing?
Charles Parnot of Stanford University is looking for your spare CPU cycles for his distributed XGrid@Stanford project.
Care to cite a source?
When you apply to the PACI program you get a grant of Service Units -- i.e. time on the computers. You don't need huge amounts of funding. The requirements state that you need to be a researcher at a U.S. institution. It also helps if you can show that you actually need and can use that kind of computing power.
And, please, distributed computing and supercomputing are not synonymous in terms of what problems they address. Distributed computing cannot replace supercomputers in every case. DC is good for a limited set of problems.
Lastly, an example of Teragrid research: Ketchup on the Grid with Joysticks.
It is based on Apple's XGrid, and uses volunteers from the Mac community here at NCSU, as well as some of the lab macs, and soon we will hopefully have official Linux and Windows clients, maybe even Solaris, to run on more of the computers around campus.
There is even a really nice web interface that shows the active nodes and their status, as well as the aggregate power of the two clusters.
Its really nice, anyone who is part of the grid can just fire up the controller and submit a job, I am a part of the lower power grid since my TiBook is only a 667, but I was able to connect up and do the Mandelbrot Set thing that comes with XGrid at a level equal to around 7 or 8 GHz.
There are some screenshots here
e to the pi i plus one equals zero
This is exactly why there's BOINC, distributed.net, Grid.org, etc. that have multiple projects served to one user-installed program: Instead of projects having to compete for real-time resources, they run work units one after the other, and users can pick and choose which projects are run. This should prevent said burnout for most users.
I think the Seti distributed thing is great and all, but I'd like to donate my free cycles to a project with more immediate (and likely) benefits to mankind. Any suggestions?
-Brien
" Just tell me, will you allow someone to use your computing power for a really _unknown_ purpose?"
Just think, for once redmond got it right! Windows has had that feature for years!
There's this one, it's probably what you want:
http://www.stanford.edu/group/pandegroup/folding/
But I'm quite selfish (and actually interested in primes abd or at least know more about them than I do about protiens), and there are entities offering big prize money for big primes, and if one of my machines finds one, I'll get big bucks:
http://mersenne.org
Tag lost or not installed.
Actually Mr Mod, its not Offtopic, Redundant maybe but Offtopic no.
Unfair! Let's all do some Distributed Metamoderating!
Tag lost or not installed.
It's too bad that whoever modded this Insightful doesn't know much about parallel applications.
DC is fine and very cost-effective for its niche of applications, which is those that are "embarassingly parallel." This is (somewhat circularly) defined as being very easy to parallelize on a DC machine. What characterizes these apps is very low communications between different tasks, which works for DC because the high network latency doesn't get in the way.
I've love to see you try to put Conjugate Gradiant (CG) on a distributed system. It involves large matrix-vector multiplies that inherently require lots of vector fragments passing between the processors. CG is one of the 8 NAS Parallel Benchmarks, and if you look at Beowulf papers that use NAS, you'll see that they often leave out CG because performance is so bad. If it's low on a Beowulf, where the network is presumed to be local and dedicated, it will totally suck on anything with a typical high-latency/low-bandwidth network.
download porn
Yes I like the folding@home project. But you may find something interesting from the projects listed here: http://www.aspenleaf.com/distributed/distrib-proje cts.html
yes I know my sig is coming up
Join Team Mozilla #38050 Folding@home
There really is no comparison.
We're not really just measuring flops; you are limited by bandwidth with distributed computing.
A distributed client can donate as much cpu as it wants but scientific problems requiring large datasets + processing + sending it back isnt going to compare to supercomputing nodes which have multigigabit network connections and distributed filesystems running at gigabytes/sec.
Not all problems can be broken into small computational chunks.
Actually, as an experimental and theoretical biophysicist-in-training who knows about proteins, I'd say the folding project is only marginally more useful than the prime number search. Most biology research projects, especially computational ones, has to be sold on the basis of potential benefits to human medicine. Such advertising does not actually mean that medical benefits exist.
While there's much to learn from studies of protein folding, there's very little medical importance to purely theoretical simulations. Since the delusion that we'll be able to replace laboratory research with really big computers is attractive to people who know nothing about biology, the impact of this type of research gets vastly overstated.
On the other hand, Folding@Home has already yielded far more interesting results (if not exactly "useful" outside of the world of biophysics) than SETI@Home probably ever will, so go for it.
in Japan!
Now the public can chose what problems that it wants solved
Jesus, there's a horrible thought. I've met the public (and seen it's choice in TV). I'd rather have monkeys choose.
Well the idea is that, you, as a member of said public, take responsibility more serious instead of just dissing it because others do.
With great power comes great electricity bills.
Since someone has already posted the Aspenleaf list of projects, I'd like to point out my personal favorite, Find-a-Drug. It has actually returned positive anti-cancer and anti-AIDS results that have been lab tested and verified by the National Institute of Health. If that doesn't have immediate benefit to mankind written all over it, I don't know what does.
"The "Grid" portion of the TeraGrid reflects the idea of harnessing and using distributed computers, data storage systems, networks, and other resources as if they were a single massive system." (from the TeraGrid FAQ)
It looks like TeraGrid is latching onto a catchword in order to boost awareness of their system. What they are describing here is not Grid computing at all. Grid computing was designed to take advantage of all the dead cycles that computers typically have. The idea is that someone might have a large group of computers that do not take full advantage fo their computational cycles (like a large lab for reading e-mails and browsing the Internet). With Grid computing you would take these computers (not some Itanium cluster like TeraGrid is doing) and distribute work accross these nodes that can be performed during otherwise dead cycles. (I have no sources immedeately available but check out Grid computing through the ACM or something and you'll see plenty of info on what Grid computing really is.)
This is what Seti@home does. It takes underutilized machines and runs computations on them. TeraGrid on the other hand, takes large clusters of otherwise unused machines and lays an abstraction over them that makes them look like one large supercomputer. This is nothing more than a distribution strategy. It looks like a nice distribution system that has the potential to scale well, but it's not Grid computing and it's nothing new.
Jesus, there's a horrible thought. I've met the public (and seen it's choice in TV). I'd rather have monkeys choose.
You might be right about those monkeys. In Holland, we have the Beursgorilla (http://www.beursgorilla.nl/). This gorilla decides what stock to buy or sell based on the bananas presented to him. He proves to be better at "advising" than most of the other "real" and expensive advisors.
For me, the DistributedComputingGorilla might decide what project will run on my computer.
It's an all *nix environment presently totalling around 4200 CPUs of which 96 ( in a single cluster)
is AIX 5.2, 3128 (WOW!!) is on Tru64 (in 2 clusters) and the rest, distributed in 5 clusters
are some form of Linux.
Two of the clusters have a second phase which together will add 316 CPUs on Linux.
As of October 1 of this year, 5 clusters at 3 sites will be added with the OS / CPU breakdown as follows:
Linux : 1800 CPUs in 3 clusters
AIX 5.1 : 320 in 1 cluster
Solaris 9 : 256 in 1 cluster
That's an awful lot of Unix and a buttload of Tru64 and Linux
Pain is merely failure leaving the body
To list only a few rants:
- vastly inconsistent spelling;
- multiple phonemes attached to one character (I think this is the way to properly note that one letter has lots of different sounds);
- a complete lack of a distinction between singlular and plural 'you' (a very common situation, solved in the Southern U.S. with "y'all");
- many homonyms; no representation (alphabetic characters) for sounds common in many other world languages including arabic 'Q'/'K', swahili tounge-click, and hebrew 'chai' (as in Chaunukka).
On the plus side, we do advantages of:- very few grammatical cases except possessive (word endings - for example, word Jack vs. word Jack's, although plural Jacks does confuse!);
- No silly male/female/neutral distinctions for everyday objects that have no inherent gender (since when should a table be presumed male or femaale?!?!);
- Large vocabulary - the number of words is very large compared to many other languages, which leads to fewer homonyms;
- since we have few grammatical cases, rhyming can be more difficult and therefore more beautiful;
- Verbs do not change endings when used on different subjects (he fell, she fell, we fell, they fell) (vs. french's je vais, tu va, il va, nous avons, etc.).
Just a rant, but maybe if we all decided to do something about this mother tounge of ours, we might change it for the better (being ease of use, distinctiveness, expressiveness, reduction of confusion, etc.).-- Kevin J. Rice
Unitarian Church: Freethinkers Congregate!
(Creutzfeldt-Jacob Disease [aka Mad Cow], et alia)
Grid computing is a crock of shit.
It's my job to try and make this crock of shit work.
Teragrid is one of several grids building on Globus. Globus is, at best, "research grade" software. Frequent re-writes. Critical bugs ignored ("we'll fix it in a later version"). The developers clearly only do the most simplistic of tests and then are surprised when the system doesn't scale into the thousands of concurrant jobs real users nead.
On top of all of this, no one is seriously attacking the real problem: cross-administrative domain issues. Want to run on a given grid? Well, you'll need to fill out different forms for different sites, obey different policies at different sites, and tweak your tasks to match each sites different configuration. The goal of simply sending your jobs off to run somewhere with minimal effort is still in the depressingly distant future.
Any Globus based grid that is doing real work at this time is only doing it because there is a small core of techies at the bottom, desperately bailing water. A grid might compose 10 or 20 sites, but a given research will typically only use one or two that they've finally gotten working.
The grid, the idea of being able to get (perhaps purchasing) compute time as easily as you get electricity, is a noble goal. I think that in the future it will be key to enabling compute-heavy research. But anyone telling you that today's grids are great and wonderful is just blowing smoke up your ass in the hope you'll give them more grant money. This sort of unrealistic happiness just gives us techies ulcers when people discover that everything is broken and that we're bailing water. "Hey, we were told everything was wonderful, what's taking so damn long." And we can't be honest and tell them "You were lied to. This is research grade software, strictly experimental and completely inappropriate for professional use."
Feh.
Anonymous for obvious reasons.
The only things which serve humanity are aliens. To say that science has such a purpose places scientists, who are the best and brightest, in the position of slaves. Worse, it enslaves them to the worst and most ignorant - the degree of their evil or ignorance dictates how much moral claim they can place on the scientists. It reminds me of a quote "You have the privilege of strength, but I have the right of weakness!"
Disgusting. Your Nickname says it all. My only hope is that after I'm long dead, you inherit the world that you're advocating.
I want to delete my account but Slashdot doesn't allow it.
Currently available projects are...
SETI@home
Predictor-Protein structure prediction
Coming soon....
climateprediction.net
Folding@home
Farther in the future (i.e. pending funding)...
Einstein@home -- a search for gravitational waves.
In the conceptual stage, since sometime last week...
neuralnet.net -- studies of the nature of intelligence using neural nets and genetic algorithms
Support SETI@home
The key word there is "probably." One positive, one single positive, from SETI will be arguably the greatest discovery in human history (certainly in the top 5).
Stop learning! Only you can prevent esoterrorism.
> solved in the Southern U.S. with "y'all"
Who would have guessed that ignorant hicks could improve the English language?
I suppose it makes sense, though, that someone with less education could come up with simple & obvious solutions (although I'd prefer using a word that didn't make you sound stupid).