Slashdot Mirror
Supercomputing: Raw Power vs. Massive Storage
securitas writes "The NY Times reports that a pair of Microsoft researchers are challenging the federal policy on funding supercomputers. Gordon Bell and Jim Gray argue that the money would be better spent on massive storage instead of ultra-fast computers because they believe today's supercomputing centers will be tomorrow's superdata centers. They advocate building cheap Linux-based Beowulf clusters (PCs in parallel) instead of supercomputers." NYTimes free reg blah blah.
Cluster computing really is the future. Supercomputers are expensive, run wierd OSes (sometimes), and have infrasructure requirements. A cluster (I prefer OpenMosix, but Beowulf if you like) just requires fast ethernet or fibre.
Plus, think of all the computers that go unused at night in places like school computer labs. All those free machines could, at night, join a cluster and do number crunching for researchers.
-- Bill "Houdini" Weiss
There are lots of reasons to have really good bulk storage technology. But what's the killer app that's going to get the $10^9/year in government spending? Can you say "Domestic Surviellance" boys and girls? I knew you could!
Because getting rid of "big iron" will kill IBM and Sun, their competitors. They can't *really* think that a cluster of PCs is a one-size-fits-all solution.
Trolling is a art,
What company would like to supply database software worth a potential $1b per year?
Just waiting for the other shoe to drop...
Esteem isn't a zero sum game
Look at the average Joe Schmoe, or even us uber-users, who really needs a 3+ GHz machine? Even some of the cornerstones of fast computing such as computational problem solving are being addressed by grid/cluster based solutions which typically don't use high end machines.
I'm perfectly happy with my P3 800MHz, but I run out of hard drive space everyday.
Cheap, YET RELIABLE high density storage solutions are still not readily available. I know we are now down to a $1 per Gig, but the average size of a user's file has increased now. Media (legal or otherwise), games, and other programs are chewing up hard drive space.
There needs to be more research into trustworthy, lowcost high volume storage mediums.
-"Those who fought today will die tommorow."-
As much as I hate conspiracy theories and Microsoft bashing, this may be an extremely clever move. As of now, mainframe and supercomputing worlds are still relatively safe from commiditization. Unlike Linux, which is still virtually ireelevant on the desktop, mainframes and supercomputers are much bigger a piece to swallow for Microsoft. By recommending Linux clusters, Microsoft may actually be trying to establish commodity hardware in the world of supercomputing. The keyword here is hardware. Once clusters become ubiquitous, Microsoft will start aggresively pushing Windows 200X Server Cluster Edition, fighting an enemy it has already much experience with.
It's not Microsoft recommending anything. This is two independant researchers - leaders in the field - who happen to usually work out of Microsofts Bay Area research center.
They dont work for Microsoft, Microsoft simply provides the grants that fund their research.
If anything their report would tell those who are on the MS payroll to get to work on a cluster offering.
I don't need no instructions to know how to rock!!!!
I think they're advocating spending the big bucks on data storage rather than on big iron.
When they mention beowulfs, it's in the context that when researchers need the equivalent of a supercomputer, they can just build/use a beowulf cluster. What they can't do on their own is come up with petabyte storage facilities and the data in them.
So what they're really advocating is spending money on storage; it doesn't say in the article what form that storage should take.
The government may very well like this. They're going to need big data farms to support the TIA program. It takes a lot of space to remember what kind of toppings every person in the US likes on their pizza.
The article isn't about the fact that the researchers were Microsoft researchers, the article is about the fact that they suggest spending in data storage instead of processor power.
In my opninion, those two are inseperable. What use is running a giant experiment about modeling the globe's climate when you don't have a huge source of data to base your experiments on? And when you generate the model, what good is it if you can't store the results for future reference?
I do find that processors evolve very rapidly, still respecting Moore's law, but the data-storage field could really do with a scientific breakthrough to increase storage capacity alot.
Perhaps putting more money in the envelope for research in this field would satisfy the current need for cheap and reliable data-storage. Perhaps the need alone is enough to make people come up with better storage facilities.
In a word, Bullshit.
These are not MS evangelists we're talking about here. Gordon Bell and Jim Gray know a 'thing or two' about high-performance computing.
If these guys weren't able to speak their minds on technical matters entirely without retribution from Bill and Steve, they wouldn't be at MS at all. They don't have to be. They CERTAINLY don't have to tow the party line and recommend the flavour of the week, because it messes with the latest Sun/IBM/HP/Linux/Mac threat.
Now if you actually look at the statement these guys are making and examine it based on their history, they've got a very good point. They're not talking even remotely about 'one-size-fits-all' systems--they're talking about the future of cutting-edge research.
"People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban
Research on building Mega beowulf clusters is a legit govt activity and so is building some. But the beauty of the beowulf cluster is that it is affordable to bussinesses, acadmeics and govt, plus its very adaptable to budgets and interconnection schema (fast, slow, grid, scavenger).
but beowulf clusters wont replace the need for super fast, super scalable, computers with well architected interconnects. there are lots of problems in this class, mostly physics simulation, that just cant be done well on beowulf clusters.
I should probably note that my own work involves large computer clusters. However my probelms (in biology) are in fact well suited for beowulf clsters. thus I'm happy to hear of more money for beowulf computing. but frankly I think that this should be in addition to the fast computers.
the flip side here is that it might be the case that money for fast computer resources is not being well spent as it could be at present. there seems to be too much emphasis on "landing the contract" for the computer center than on building a good design. congress via DOE tends to doll these things out in a political fashion making sure each big client gets funding for a center rather than letting the best center get the most contracts. as a result some of the so-called super computers may be just glofied too-expensive-per-cpu unscalable systems already that could be eclipsed by a comparable low cost beowulf system.
but that being said its still an area that the gov needs to fund since it wont drive itself commercially but its needed for lots of science and simulation.
Some drink at the fountain of knowledge. Others just gargle.
He also talked about CERN generating 10 PetaBytes a year when their new collider comes on line
Supercomputers are sexy, but are losing the technology war. If you start designing a new one today it will be years before it is ready. During those years Intel and AMD will crank up their clock speeds and negate much if not all of the CPU speed advantage you get from your fancy design. Why not go for parallelism from cheap machines?
No electrons were harmed creating this post, though some may have been subjected to electrical and/or magnetic fields.
they're making the case that clusters should be pursued over supercomputers for the data-intensive number crunching activities like nuclear explosion modeling, etc.
I doubt it. You can only use a "cluster" like a Beowulf if your problem can neatly be divided into small, completely independent work units. If you want to render a movie, then so long as you have all the scene data, each frame can be rendered completely independently of any other, then stitched together at the end.
Try using a Beowulf-style cluster for a CFD problem, and watch as all computation grinds to a halt as your processors and interconnects devote all their capacity to inter-node coherency and synchronization. You need a traditional supercomputer like an SGI Origin for jobs like that, because of its massive internal bandwidth.
There is absolutely no danger of Beowulfs killing off the supercomputer in the near future. In fact, the supercomputer market is looking pretty healthy.
Don't confuse Microsoft research with the rest of Microsoft. The research branch has the same atmosphere as a university. In fact, Microsoft has bought a number of university research groups wholesale. Quite a few famous people are now working for them (e.g. Tony Hoare, Erik Meyer, and the guys in the original article).
I've heard presentations from them, and talked to them in private, and I can assure you they are far from following the party line. I'm sure that any pressure from above to do so would cause massive protest.
Microsoft is very wise to run the research branch this way. Research is not the province of yes-men.
Massive data storage doesn't mean a thing to people like me who do computational physics work. We need better supercomputers to simulate larger systems... or simulate them faster. Sure, we can simulate a system of 300,000 particles within a few hours, but there could be great value in simulating systems of millions of particles. Maybe there is some effect that we miss... or something.
Anyway, data storage is not a problem in MY field -- and I would think that government interests in supercomputing lie in places OTHER than fast database servers or whatever.
Mmmm......sacrelicious.
According to this a "beowulf" is a cluster of cheap computers, NOT a cluster of cheap LINUX computers. I don't think Microsoft is advocating Linux, as much as I/you/we wish they were... http://www.phy.duke.edu/brahma/beowulf_online_book /node61.html
> Would someone please mod down parent as "presumptuous" and "whiney"
Will someone ever shut the hell up and not spew crap just because they feel like pissing someone else off?
Maybe the editors, like some of us readers, want to play by the rules. Sure a non reg link exists, but how long will it exist when every news site starts abusing the system?
Im happy enough signing up once for a website, using a spam catching email address, and using those login detials to read whatever the editors post.
If you dont like registering for an article, then dont read it, but dont bitch about it.
No, I just actually work in the industry.
Where can I get Gigabit routing hardware that is comparable in price to 100BaseT equipment?
A Pirate and a Puritan look the same on a balance sheet.