Slashdot Mirror
Truly Off-The -Shelf PCs Make A Top-500 Cluster
SLiDERPiMP writes: "Yahoo! News is reporting that HP created an 'off-the-shelf' supercomputer, using 256 e-pc's (blech!). What they ended up with is the 'I-Cluster,' a Mandrake Linux-powered [Mandrake, baby ;) ] cluster of 225 PCs that has benchmarked its way into the list of the top 500 most powerful computers in the world. Go over there to check out the full article. It's a good read. Should I worry that practically anyone can now build a supercomputer? Speaking of which, anyone wanna loan me $210,000?" Clusters may be old hat nowadays, but the interesting thing about this one is the degreee of customization that HP and France's National Institute for Research in Computer Science did to each machine to make this cluster -- namely, none.
Can you imagine a Beowulf cluster of these... erm... clusters?
-Berj
Was when the HP-powered cluster started assimilating some of the Compaq multi-Alpha machines as it's own.
How about $0 Baldric a student run beowulf at the University of Western Ontario built one on hardware dontations. It's not exactly top 500 but it still kicks ass.
Should I worry that practically anyone can now build a supercomputer?
Unless "practically anyone" has the funds, the storage room, and the manpower to maintain this monstrosity, there is nothing to worry about.
And even if anyone could build a supercomputer, what's there to worry about? We don't live in the "War Games" world where supercomputers play chess, tic-tac-toe, and start nuclear wars for fun.
Never underestimate the bandwidth of a 747 filled with CD-ROMs.
Well, it seems like super clusters are becoming very easy to build hardware-wise. If you throw enough commodity at a problem, it becomes easier. I would think the biggest problem with supercomputers is no longer the hardware itself, but networking, and the programming to take advantage of the hardware. These computers still only really work for something that distributes easily. The biggest factors are now the ability to distribute, and schedule work for each node. The more nodes you engage, the more you hope your problem is CPU bound, so it will scale more.
:-)
Data transfer and message passing are such a big issue I belive the most important developments are in the networking topologies and hardware for these environments.
That said, I still want one in my basement
Now all we need are ways of getting local connections significantly faster (Did someone say Gig Ethernet) to allow faster communication between the nodes and we will be able to scale beyond several hundred and break the top 100. I hear 1gig NICs will be falling in price to under $100 US retail soon...
How fast do you connect to your cluster ?
Should I worry that practically anyone can now build a supercomputer?
Yes, you should probably worry that practically anyone can build a supercomputer. But you could mitigate all that fear with the fact that not practically anyone can whip up software that takes full advantage of it.
Thank god there isn't any off the shelf "missile trajectory" software in the CDW catalog. you would hope that any society that can whip together motivated coders to write such code already has access to some pretty spiffy kit.
(yeah i said "kit"... and I'm from Chicago... I feel like such a wanker.)
A Beowulf cluster of E-machines?
I dunno. It's kinda lacking when you compare it to all the other Beowulf clusters we've considered.
The next Slashdot story will be ready soon, but subscribers can beat the rush and slashdot the links early!
shouln't that be Yahoo! Serious News now?
Sez the cost was $210k US w/o cabling...why the qualification? What *would* cabling for 225-odd boxen cost?
Carousel is a lie!
The cluster is at #385
You know this Beowulf business is getting to be pretty staid and routine by now.
In fact, I'd almost say it would be newsworthy if there were any organization (university, company, govt lab) that had not yet built "a supercomputer from the COTS components".
What I'd like to see now is more metrics (some of which the article does, admittedly, reveal).
- hardware cost per FLOP (everyone already tells you this)
- FLOPS per human time to build
- FLOPS per sysadmin time to maintain
- FLOPS per kilowatt of electricity
- FLOPS per cubic foot of rack space
- can it run smoothly if Bad Andy goes behind the rack and unplugs a few network connections, a few power cords to some nodes?
Everyone knows that you can spend your own time scouring dumpsters for cast-off computers and coaxing them to life, bringing up an old 486 with an ISA 10bT card as a member of your cluster. Unless you're doing it for your own educational benefit, it's just not worth it.Don't get wrong. I love these clusters and want to use them. It's just that, in 2001, their mere existence is no longer as exciting as it was in the mid 1990s.
Now days, I care more about ease of use and ease of maintenance, taking the low cost of a Beowulf cluster as a given.
With the size of these clusters going up and the ratio of hardware cost to human time constantly decreasing, I'd be more impressed to see how a system with many hundreds of nodes was brought up in a short time, never rebooted for a year, even as 13 of the nodes developed variously problems and become unproductive members of the cluster.
"Provided by the management for your protection."
No, a beowulf cluster is the last thing that one would use for nuclear simulation.
While great at highly parallel tasks that require very little synchronization between threads (think code cracking), nuclear testing (and almost all other fluid dynamic problems) generally requires all of the cpu's to have high speed access to all of the memory. So one needs a huge shared memory system (think Cray or Sun StarCat).
And for this reason, I find the top 500 list to be a bit misleading in these days of massively parallel systems. Its great as a test of how many flops the system can crank out, but it does not take into account the memory bandwidth between the cpu's, and that is often more important than raw cpu horsepower.
Actually, the best MIPS/Wh is probably with the slower versions of the current laptop chips. Maybe portable G3/G4?
Also, I don't think you'd get much useful stuff done with early Pentiums and 486. Consider that a P4 2 GHz has 20 times the clock speed and probably does twice as much per cycle, so it's ~40X faster. Now, if you connect 40 P100 together, unless your problem is completly parallel (like breaking keys, as opposed to most linear algebra), you're going to lose at least a factor of 2 there. This means that in order to equal 1 P4 @ 2 GHz, you'll need almost 100 Pentium 100 Mhz. This means that 10 P4 would be like a thousand Pentiums. At these numbers, it's going to cost so much in networking and power...
I'd say (pure opinion here) the slower you'd want to have today is something like a Duron 1 GHz and the best MIPS/$ is probably with a bunch of dual Athlon 1.4 GHz (A dual is not cheaper that 2 single, but you get more done because of parallelism issues).
Opus: the Swiss army knife of audio codec
My experience doesn't suggest that the P4 does twice as much per cycle. I'm seeing P4s do a fair bit less than the P3 per cycle, and the P3, P2, and PPro cores didn't seem *that* much faster per clock than the original Pentiums. My gut tells me that the P4 doesn't do any more than the original Pentiums per clock cycle, and the only thing they have going for them is Intel's ability to manufacture them at high clock speeds.
If you really want a cpu that does a lot in a single cycle, look at the IBM POWER series. IIRC, on the floating point side, a 2xx MHz POWER III is darn not too far from an Alpha 21264 at 733 MHz. And now there are 1.1GHz and 1.@GHz POWER IV chips, in the new IBM p690 machines. I don't know how they compare to the POWER III per cycle, though, because the POWER IV opens a whole new (good) can of worms.
-Paul Komarek