Slashdot Mirror
Supercomputing: Raw Power vs. Massive Storage
securitas writes "The NY Times reports that a pair of Microsoft researchers are challenging the federal policy on funding supercomputers. Gordon Bell and Jim Gray argue that the money would be better spent on massive storage instead of ultra-fast computers because they believe today's supercomputing centers will be tomorrow's superdata centers. They advocate building cheap Linux-based Beowulf clusters (PCs in parallel) instead of supercomputers." NYTimes free reg blah blah.
No Registration Required
Just use the google link!
Brings a tear to my eye... life is good.
My calendar says June 2nd. What does yours say?
GTRacer
- ? slooF lirpA
Defending IP by destroying access to it? That makes sense, RIAA/MPAA. Go to the corner until you can play nice!
Gordon Bell and Jim Gray are not just "a pair of Microsoft researchers". They are two of the biggest names in high-performance computing. Gordon Bell awards, anyone?
Just wait till Bill and Steve hear that their engineers are recommending Linux instead of Windows 2003 Server.
Its nice to see some MS researchers going against the perceived stereotype and being open in their suggestions like this.
And I think they have a good point about massive memory being a very important part of computing advancement right now.
Atheism is a religion to the same extent that not collecting stamps is a hobby.
In an earlier story Microsoft researches recommended a Linux cluster. That story has been corrected. The Microsoft researchers recommend a hundreds of un-clustered Windows-XP servers. They claim they were eating Lea-Nuts brand PEANUT clusters at the time of the interview and were misquoted.
- For the complete works of Shakespeare: cat
New York Times?
MSFT'ers recommending Linux?
I thought they fired that reporter who was making things up
Cluster computing really is the future. Supercomputers are expensive, run wierd OSes (sometimes), and have infrasructure requirements. A cluster (I prefer OpenMosix, but Beowulf if you like) just requires fast ethernet or fibre.
Plus, think of all the computers that go unused at night in places like school computer labs. All those free machines could, at night, join a cluster and do number crunching for researchers.
-- Bill "Houdini" Weiss
There are lots of reasons to have really good bulk storage technology. But what's the killer app that's going to get the $10^9/year in government spending? Can you say "Domestic Surviellance" boys and girls? I knew you could!
You could at least use partner=SLASHDOT
Look at the average Joe Schmoe, or even us uber-users, who really needs a 3+ GHz machine? Even some of the cornerstones of fast computing such as computational problem solving are being addressed by grid/cluster based solutions which typically don't use high end machines.
I'm perfectly happy with my P3 800MHz, but I run out of hard drive space everyday.
Cheap, YET RELIABLE high density storage solutions are still not readily available. I know we are now down to a $1 per Gig, but the average size of a user's file has increased now. Media (legal or otherwise), games, and other programs are chewing up hard drive space.
There needs to be more research into trustworthy, lowcost high volume storage mediums.
-"Those who fought today will die tommorow."-
As much as I hate conspiracy theories and Microsoft bashing, this may be an extremely clever move. As of now, mainframe and supercomputing worlds are still relatively safe from commiditization. Unlike Linux, which is still virtually ireelevant on the desktop, mainframes and supercomputers are much bigger a piece to swallow for Microsoft. By recommending Linux clusters, Microsoft may actually be trying to establish commodity hardware in the world of supercomputing. The keyword here is hardware. Once clusters become ubiquitous, Microsoft will start aggresively pushing Windows 200X Server Cluster Edition, fighting an enemy it has already much experience with.
I think they're advocating spending the big bucks on data storage rather than on big iron.
When they mention beowulfs, it's in the context that when researchers need the equivalent of a supercomputer, they can just build/use a beowulf cluster. What they can't do on their own is come up with petabyte storage facilities and the data in them.
So what they're really advocating is spending money on storage; it doesn't say in the article what form that storage should take.
The government may very well like this. They're going to need big data farms to support the TIA program. It takes a lot of space to remember what kind of toppings every person in the US likes on their pizza.
You have to wonder why, all things seriously being equal, they don't recommend a *BSD-based solution instead of a Linux-based one. Esp given the near-equivalent functionality of the *BSDs, and the fact that MS has publicly endorsed the BSD license in the past, citing it as an superior alternative to the GNU License.
From the MS site, the Bay Area Research Center is "... a small Microsoft Research group located in the San Francisco Bay Area. We've been working on two large projects with other universities, companies, other Microsoft Research groups, and with Microsoft product groups in Redmond and Cupertino. These projects are Scalable Servers and Media Presence. "
I can't see scalability involving commodity hardware with MS OSes. In spite of Microsoft's desktop domination strategies, and small business server dominance (arguably, at least for the moment) they know they won't be taken seriously about clustering Windows 2003 server, purely because there is no design AFAIK in the kernel for operating in clusters in the first place. This is supercomputing using commodity hardware, not supercrashing using commodity OSes. Linux is perfectly situated to be recommended by anyone because it is not a competitors product, per se.
The homepages of the two men can be seen here, if anyone is interested in some of the more interesting history of the two. Little of it has to do with Microsoft propaganda and the marketing machine:-
Gordon Bell
Jim Gray
Conversion Rate Optimisation French / English consultant
Raw speed will always be useful for problems that are hard to parallelize. Right now those problems (parts of crypto, some quantum physics calculations, etc.) are important scientifically, but away from the money.
Industry will spend R&D money on clustering for storage and reliability, without major government subsidy, because there's a crying need for it. How much government money went into Google/eBay/Amazon?
Government research is supposed to complement industry R&D - to be aimed at fields where the results are still important, but maybe not as profitable. This is why government should not abandon raw speed as a research goal.
To a Lisp hacker, XML is S-expressions in drag.
By rewriting existing scientific programs, they say, researchers will be able to get powerful computing from inexpensive clusters of personal computers that are running the free Linux software operating system.
"The supercomputer vendors are adamant that I am wrong," Dr. Bell said. "But the Beowulf is a Volkswagen and these people are selling trucks."
All the people who are responding saying they don't mention Linux didn't read the second page.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
I saw that it could be google too, but anyhow, I made a username/password for y'all:
slashdot124
slashdot
Be wary however, I registered as a North Korean military R&D official under high salary.
---
"The chances of a demonic possession spreading are remote -- relax."
All this for the price of a few supercomputers every year. And the market for supercomputers pushes several technologies; for example, high speed interconnect and gallium arsenide, and sets the bar for high performance silicon. Pretty good deal, doncha think?
But now the Moron-in-Chief wants to bring back nuclear testing. (pardon me, 'nookyuler.' Bush can't be wrong about something as simple as pronunciation, can he?). Farewell to deterrence. Farewell to common sense...
--- Often in error; never in doubt!
Research on building Mega beowulf clusters is a legit govt activity and so is building some. But the beauty of the beowulf cluster is that it is affordable to bussinesses, acadmeics and govt, plus its very adaptable to budgets and interconnection schema (fast, slow, grid, scavenger).
but beowulf clusters wont replace the need for super fast, super scalable, computers with well architected interconnects. there are lots of problems in this class, mostly physics simulation, that just cant be done well on beowulf clusters.
I should probably note that my own work involves large computer clusters. However my probelms (in biology) are in fact well suited for beowulf clsters. thus I'm happy to hear of more money for beowulf computing. but frankly I think that this should be in addition to the fast computers.
the flip side here is that it might be the case that money for fast computer resources is not being well spent as it could be at present. there seems to be too much emphasis on "landing the contract" for the computer center than on building a good design. congress via DOE tends to doll these things out in a political fashion making sure each big client gets funding for a center rather than letting the best center get the most contracts. as a result some of the so-called super computers may be just glofied too-expensive-per-cpu unscalable systems already that could be eclipsed by a comparable low cost beowulf system.
but that being said its still an area that the gov needs to fund since it wont drive itself commercially but its needed for lots of science and simulation.
Some drink at the fountain of knowledge. Others just gargle.
He also talked about CERN generating 10 PetaBytes a year when their new collider comes on line
Supercomputers are sexy, but are losing the technology war. If you start designing a new one today it will be years before it is ready. During those years Intel and AMD will crank up their clock speeds and negate much if not all of the CPU speed advantage you get from your fancy design. Why not go for parallelism from cheap machines?
No electrons were harmed creating this post, though some may have been subjected to electrical and/or magnetic fields.
Imagine what a cluster of 700 to 1,000 blade servers running the latest Intel Xeon CPU's can do now! =)
Actually, it would be a very crappily-performing cluster. Blade servers are designed with two major goals - CHEAP and SMALL. Blade servers are engineered for high availability applications (think webserver farm).
Just because you CAN do something doesn't mean it's the optimal solution. It amazes me when I see vendors selling blade server clusters.
(Disclaimer: I work as an engineer with a company with builds Linux based clusters for universities and labs)