Slashdot Mirror
Canada to Launch Countrywide Virtual SuperComputer
LadyCatra writes "A serious shortage of world-class computing power in Canada prompted University of Alberta scientists to create the next best thing -- a countrywide, virtual supercomputer.
On Nov. 4, thousands of computers from research centres across the country will be strung together by a U of A effort to create the most powerful computer in this country.
The full story is here"
Why didn't they just make a client program for distributed computing so the entire country/world could help out?
sig.
...on Nov. 5, someone will find a way to temporarily use all of this virtual power to play a round or two of half life....
This seems like a really good idea, I don't really understand why more places don't do this. I mean most of us work in offices where the computer power is amazing and largely untapped.
I think what this really needs is to be make easier for the mainstream, so anyone could do it. Perhaps bundle the tools (programming and deployment) with mainstream operating systems?
It's just an idea, my NeXT had Zilla (it's version of this) years ago - seems a shame that this hasn't caught on more widely. So come on Apple - let's see it, put it in the Darwin project and put a nice UI on it in Mac OS X.
Anyway, before activating It, make sure It doesn't have any access to a spare nuclear warhead on orbit around Earth.
"The Network is the Computer"
It would be nice to see a worldwide system. If this is going to work there must be some CPU time quota system, perhaps a quota that can be bought and sold. This could make it interesting for ordinary home users to join (earn quota, sell quota, make $$$). There are many projects in the academic world that could never make a SETI@home launch, since the research is to boring. Still, we need to use all that idle time buring away across the world.
SkyNet!
That's a fantastic idea ! If this works, we'll be able to use it for useful computation ! It might sound crazy, but with such a virtual computer, one could make computations to help SETI or to cure cancer skyrocket ! How did they come up with such a great idea ?
...all Beowulf posts under this thread, including (but not limited to):
- standard Beowulf trolls mixed with standard Canadian accent lexicon ("eh?", "aboot")
- posts about how a Beowulf cluster could perhaps help Canada out with a stereotypical Canadian "problem" (lousy beer, socialized medicine)
- jokes combining the word Beowulf with the name of the mentioned U of A chemist Wolfgang Jaeger
Thank you.
"The computers will be linked by the Internet, but involve a simple networking system, Lu said. Keeping the linkage as simple as possible was the goal."
Based on the article I would assume that they have made a custom tailored system (if not kludge) for one specific purpose ("for calculating energy shifts as two molecules are manipulated around 3-D space") - and not a platform which could be easily tailored and managed to solve different kinds of tasks with different kinds of relationships between the tasks.
Ohh, I could also link my grid computing links.
"A language that doesn't affect the way you think about programming, is not worth knowing" - Alan Perlis
The computers will work jointly on a molecular chemistry research question that would take a single computer as long as six years to complete. Jonathan Schaeffer and Paul Lu, professors in the U of A's department of computer science, expect their virtual supercomputer will do the work in one -- one day, that is.
So how is this different from DC or SETI?
Live for the present, learn from the past, and dream of the future!
The article isn't very specific on the kind of problems they will try to solve. The 'search' problems, where you have a big search space than can be easily divided into smaller chunks are easy. Unfortunately some problems cannot be easily split into many independent parts - simulations generally fall into this category. Weather simulations, nuclear explosion simulations, well, simulations in general :-). You can just assign each computer a square mile of terrain, do the computations for the whole simulations, then merge the results - the neighboring squares interact, so computers have to communicate after each time slice. This is where communication will probably slow your 'network supercomputer' down. No matter how fat the pipes are, they will be several orders of magnitude slower than an internal supercomputer bus in terms of latency. To put it short: this might be of some use, but they better start gathering money for a real supercomputer.
step 1:build the largest virtual supercomputer in canada
step 2: ???
step 3: global domination!
From the article Gerald Oakham and his fellow physicists have a problem. In the hunt for the most elusive speck of matter known to science, they are about to generate more data than any computer on the planet can analyse.
My school, in conjunction with the Université de Sherbrooke (mostly the U de S) are setting up a world-class beowulf cluster for general scientific work. A physics professor at my University, who also happens to be a world class astronomer (Dr. Lorne Nelson) has a research grant that he is using to help with the funding for this cluster.
rather than joining a currently existing project? I'm a student at the University of Virginia and we have a project like this that's been going on for 5 years now: http://legion.virginia.edu/
(I think these are a little out of date. There's a bunch of rack-mount machines in there now too)
They talk about how they feel that Canada should be pursuing its own supercomputing, but why not join up with other universities that have been pursuing similar projects and give Canada access to the computing power of other countries as well? Isn't the goal here for people to work together for mutual benefit? I don't understand why they feel the need to isolate their Canadian initiative, rather than giving Canada the access to computing power far greater than they can acheive on their own.
Check out photos of UVA's branch of Legion: http://legion.virginia.edu/centurion/Photos.html
This room has big glass walls, and everytime I walk by it I wish I had a room like it.
The article quoted that the computers "will be linked by the Internet, but involve a simple networking system". How many of you are willing to bet that someone is already gleefully planning a DDOS party?
The Wknd Sessions - Malaysian and South East Asia independent music
So, why is this news? Is there some new technology they are using?
You mean like Popular Power tried (and failed)to do? Check their old site to see what they used to propose.
Looks like selling CPU cycles is not a lucrative business...
I code, therefore I am.
I was a little surprised myself. I expected to get modded down Redundant, but Offtopic and Troll were surprising.
-- Cheers!
I clipped this out of reuters....
Today, the Canadian Ministry for computing announced their initial tests of the Canada-wide massive computer project..
Computer Scientist Thom Serveaux had this to say," when we switched it on every command was answered with the word "eh?" and it kept calling us "knobs" and was asking for "back bacon" we are trying to see if there is any problems in northern nodes that were like the Quebec nodes that started a fight with the other nodes demanding every command to be repeated in french."
Updates will be posted on their progress..
Do not look at laser with remaining good eye.
Isn't William Shatner from Canada? Maybe this is an attempt to develop a more powerful 'Priceline SuperComputer'....
A supercomputer capable of creating more convincing commercials, perhaps?
Good judgment comes from experience.
Experience comes from bad judgment.
Saving up for a "real" supercomputer is a pipe dream. Supercomputers cost several million dollars a year in upkeep, and that's the killer. You might easily get grants to allow a project to use 'x' dollars worth of computing, but nobody is going to approve a capital grant that requires millions each year.
When the University of Toronto did purchase a Cray in the mid-eighties, there was a massive fight. Many felt that the resources to support the Cray were sucking money desperately needed everywhere else. (although, boy, we in meteorology a happy bunch...)
While lower profile and somewhat more painful to use, this is far more practical solution for the realities of academic computing today.
As many of the other posters have pointed out, this work isn't necessarily new, but it is news.
There are other tools out there which do this: Legion, Avaki, Sun Grid Engine, Globus, to name a few but the goal is to create a network of (mostly) supercomputers which doesn't require a lot of reconfiguration at each site. What differentiates this work from many other approaches is that it is transparent to the system administrator.
For those who ask "why can't you just do something let seti@home" the answer is that not all problems in science and business can be easily decomposed into small chunks. Bandwidth requirements and latency may also be a problem. A lot of scientific programmers have to worry about communications much more than about processing power (although this tradeoff has been seesawing backwards and forwards with new advances in both technologies).
There's a worldwide effort through both business and academia to create a number of good, interoperating frameworks for doing this sort of transient, virtualised supercomputer.
Have a look at the Global Grid Forum (which is becoming the focus for Grid computing standards) for more information.
Looks similar to the Grid Computing project from India, announced sometime back ...
I would imagine this distributed computing effort will be conducted using Ca*NET, which isn't really what I'd consider a public network. As such, I don't think they'd really have to worry about hackers and such.
The 'Canadian Internetworked Scientific Supercomputer (CISS)' website is located here: http://www.c3.ca/ce/ciss_t.html
It seems that November 4th they will be doing a full 'production' test. Cool.
-- bartman
Looks like they based their protocol on ssh.
.NET... odd, I thought MS was in the market for Universities. :>
No MS Passport or
-- bartman
Search for the elusive beer molecule. Eh.
It's 10 PM. Do you know if you're un-American?
The new supercomputer will be used to determine when the Maple Leafs will win the Stanley Cup next.
Virtual Laboratory of Eastern Ontario.
The High Performance Computing Virtual Laboratory (HPCVL) was formed by a consortium of four universities located in Eastern Ontario (Carleton University, Queen's University, The Royal Military College of Canada, and the University of Ottawa).
http://www.hpcvl.org/
It's also in the Top 500 supersomputer list, so it must be half-decent. So if four universities can have a dencent computer in Canada, others probably do too.
I believe they will use high speed networks of Linux based Beowulf clusters (actually clusters of clusters of clusters). Ontario has already established SHARCNET between a number of Universities with a total of over 500 COMPAQ Alphas (mostly four-processor, 833Mhz, Alpha SMPs) and some Pentiums, all running Linux. A press release from last year gives a good overview of the project, already first in Canada and the 11th most powerful academic computing system in North America. I believe the Canada wide project will essentially form a cluster of these cluster of clusters.
SHARCNET has been up and running for a while and last year accounted for about 27% of supercomputing power in Canada (half of all supercomputing power in Canadian universities), with three sites on the Top 500 list and total power exceeding institutions like Cambridge, Princeton, Cornell and Caltech. There's loads of information available about the hardware and software used at each facility, as well as CPU load and usage statistics at members sites like these status charts from the most powerful individual site, at the University of Western Ontario. As for applications, a number of researchers are already using the system for a variety of projects across science, engineering, and economics.
My next sig will be ready soon, but friends can beat the rush!
The University of Alberta has over a dozen clusters. Their central computing facility (CNS) has two clusters, Physics has three or more, CS at least one, Chemistry has seven clusters (0.5 THz total cycles), MechEng at least one, EE at least one, ...
The U of A (U of Eh?) also participates in MACI (www.maci.ca) and houses three SGI Origin computers, and is involved with the WestGrid project (www.westgrid.ca).
Prof. Schaeffer's point isn't that we don't have "computrons", but that research is increasingly using simulations (see Jaeger's work) and other computational methods, and computational resources are becoming increasingly overloaded as budgets are not growing as quickly as research advances.
--altadel
This would be a cool way to get unrestricted high speed internet into homes...for example, if they were to say, "let us use your idle processes in return for free, unlimited, un-portblocked, static-IP internet that you can run a Domain off!" Sounds like a pretty decent trade to me.
In SOVIET RUSSIA... erm...NSA AMERICA, the Internet logs onto YOU!
Canadian Internetworked Scientific Supercomputer (CISS)
</joke>
taken! (by Davidleeroth) Thanks Bingo Foo!
The government has made all four of its 386s available for this venture, which brings, along with all private computers in Canada, the entire total up to seven.
However, the government is concerned that the supercomputer doesn't properly reflect Canadian values. For example, the color of the cases is a light beige, which is racist. So all computers as part of this net will be painted multi colors.
In addition, they must be used to do CANADIAN computation from a CANADIAN programmer at least one run time out of three.
And, to further show off Canadian innovation, they've developed a new language for this cluster, C eh eh.
Here's an example.
#define EH EH
#include "political_correctness.eh"
#include "Liberal_Party_Donation.eh"
#include "Kickback_via_golf_course.eh"
#include "Payoff_to_Bombardier.eh"
eh main (eh)
eh
printefe(FONT_DOUBLE_SIZE, "Bonjour, monde")eh
printf(FONT_HALF_SIZE, "Hello, world")eh
eh
--- Jump!! Fire!! Bullet time!! - Lego version of the Matrix
>how does an entire country have less research computing power than a university?
Easy - just keep putting the Liberals into office. Their buddies make out like bandits. The rest of us will be back to an abacus and tin cans & string in no time.
Actually, climate change has been a real problem for the ecosystem of the north coast, with a lot of ice melt and more open water than usual. One of the effects is that seals have more of the year, or in some places year-round, that they can find open water instead of making breathing holes in the ice. Polar bears and the traditional Inuit hunting methods both depend on catching seals at their breathing holes, so their hunting is much less effective.
On computer-related topics - laptop batteries really don't like background CPU-burners. I used to run the GIMPS Great Internet Mersenne Prime Search, and I used to commute by train, with about an hour of battery time each way. NiMH batteries don't have the same failure behavior as NiCD, and they're nowhere near as nasty a toxic waste disposal problem, but they really don't like this kind of treatment. To compound matters, for some of that time period, I was running Windows NT 3.51, which was much more stable than Win95, but it insisted on being a *server* operating system that didn't need laptop power management drivers, so when it got a hardware low-power shutdown signal, instead of going into hibernation mode (see, the polar bears *were* relevant), it would blue-screen and die. I had to stop running the prime search.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
On the other hand, their code basically jumped around in a large matrix, without much locality, because that matched what the real system they were modelling would do. They needed 12-14 MB of table space for the system they were modelling, and our VAX 11/780 only had 4MB of RAM, so I played with a number of virtual memory operating systems (4.1BSD, SVR2.0p, various tunings) to get something that would survive being thrashed to death, and helped them do a lot of work on checkpointing their code, because their standard run took a week, and even if something else didn't cause the machine to crash, we'd get power hits during summer thunderstorms. After about two years of this, the price of RAM dropped to the point that we could afford to upgrade the machine to 16MB, which made our run time drop to about an hour....
Meanwhile, that Cray-1 of yours was mostly similar in performance to a Pentium 133, and some of the recent graphics chips have really immense memory bandwidth, though they're mostly running fixed-point or at best single-precision floating point rather than double-precision or quadruple-precision, so they're not *quite* a Cray-replacement even though they're faster at many things.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
There are real benefits for Canadian research that can come from this project - certainly there are a number of problems that are numerical and parallelizable, so there can be a lot of future to it if they do enough coordination, but most of Canada's academic supercomputing is currently driven by SETI. Besides scientific research, the other traditional users of supercomputers are weather prediction, oil exploration, and sometimes financial modelling - Canada may have more total supercomputer-based supercomputing than anybody realizes, in addition to SETI. However, the June 2002 top500.org list doesn't show anything in Canada above #227.
Other results from the Top500.org list - SETI@Home is still about 7 times as large as the largest single machine on the list , Japan's NEC Earth Simulator, which is about 5 times as large as the #2 machine, LLNL's ASCII White.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Thanks for the interest in our project. We have composed a http://www.cs.ualberta.ca/~ciss/CISS/faq.html. I hope it is useful.