Slashdot Mirror
Truly Off-The -Shelf PCs Make A Top-500 Cluster
SLiDERPiMP writes: "Yahoo! News is reporting that HP created an 'off-the-shelf' supercomputer, using 256 e-pc's (blech!). What they ended up with is the 'I-Cluster,' a Mandrake Linux-powered [Mandrake, baby ;) ] cluster of 225 PCs that has benchmarked its way into the list of the top 500 most powerful computers in the world. Go over there to check out the full article. It's a good read. Should I worry that practically anyone can now build a supercomputer? Speaking of which, anyone wanna loan me $210,000?" Clusters may be old hat nowadays, but the interesting thing about this one is the degreee of customization that HP and France's National Institute for Research in Computer Science did to each machine to make this cluster -- namely, none.
Can you imagine a Beowulf cluster of these... erm... clusters?
-Berj
Was when the HP-powered cluster started assimilating some of the Compaq multi-Alpha machines as it's own.
How about $0 Baldric a student run beowulf at the University of Western Ontario built one on hardware dontations. It's not exactly top 500 but it still kicks ass.
Should I worry that practically anyone can now build a supercomputer?
Unless "practically anyone" has the funds, the storage room, and the manpower to maintain this monstrosity, there is nothing to worry about.
And even if anyone could build a supercomputer, what's there to worry about? We don't live in the "War Games" world where supercomputers play chess, tic-tac-toe, and start nuclear wars for fun.
Never underestimate the bandwidth of a 747 filled with CD-ROMs.
Well, it seems like super clusters are becoming very easy to build hardware-wise. If you throw enough commodity at a problem, it becomes easier. I would think the biggest problem with supercomputers is no longer the hardware itself, but networking, and the programming to take advantage of the hardware. These computers still only really work for something that distributes easily. The biggest factors are now the ability to distribute, and schedule work for each node. The more nodes you engage, the more you hope your problem is CPU bound, so it will scale more.
:-)
Data transfer and message passing are such a big issue I belive the most important developments are in the networking topologies and hardware for these environments.
That said, I still want one in my basement
Now all we need are ways of getting local connections significantly faster (Did someone say Gig Ethernet) to allow faster communication between the nodes and we will be able to scale beyond several hundred and break the top 100. I hear 1gig NICs will be falling in price to under $100 US retail soon...
How fast do you connect to your cluster ?
Should I worry that practically anyone can now build a supercomputer?
Yes, you should probably worry that practically anyone can build a supercomputer. But you could mitigate all that fear with the fact that not practically anyone can whip up software that takes full advantage of it.
Thank god there isn't any off the shelf "missile trajectory" software in the CDW catalog. you would hope that any society that can whip together motivated coders to write such code already has access to some pretty spiffy kit.
(yeah i said "kit"... and I'm from Chicago... I feel like such a wanker.)
The ultimate Linux selling tool, every linux box in your company is a node in a cluster, add a few servers for extra speed, add a few computers to provide file I/O and backup capability, and you have one of the fastest supercomputers available to your company without having to spend an extra dime (everyone needs a desktop anyway). Can you imagine the extra cycles available for simulation, whatever when people start going home at 5 PM.
A Beowulf cluster of E-machines?
I dunno. It's kinda lacking when you compare it to all the other Beowulf clusters we've considered.
The next Slashdot story will be ready soon, but subscribers can beat the rush and slashdot the links early!
shouln't that be Yahoo! Serious News now?
They mention in the end that they are working with Microsoft to support this approach. They also suggest using spare cycles. Unlike SETI@home, where you download some stuff, work on it, send it back, this appears to be a system where the power scales linearly with nodes.
Windows support makes a difference. Take a large company (10,000+ in a single location) that has some intensive projects. In this case, they could just drop the $210,000 (call it $750,000 with installation, support, etc.) and put it in a room.
However, a smaller shop, say, 50-250 employees, being able to install this software on the staff's machines. They rarely use their computers to capcacity, and can probably contribute 90% of the CPU 90% of the time. This approach could let people doing giant calculations do so cheaply.
The real question, however, is who needs that kind of horse-power. For those that need the horse power, is the savings with off-the-shelf components meaningful.
Its a tremendous accomplishment, and I wonder how much of the changes were new (vs. Beowulf clusters that we always hear about). However, if this fills a need, congratulations, its an impressive accomplishment regardless.
Alex
i'll give you $210,000 so you can do exactly what with your new supercomputer?
also, who will pay your power bills?
i don't get this "drool factor" thing some people have for supercomputers... sure, they're cool and all, but they can do exactly nothing you would want or need to do on a day-to-day basis...
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"
Sez the cost was $210k US w/o cabling...why the qualification? What *would* cabling for 225-odd boxen cost?
Carousel is a lie!
The cluster is at #385
- A Beowulf cluster of E-machines?
Maybe so, but this cluster still made into the top 500 most powerful computers.I dunno. It's kinda lacking when you compare it to all the other Beowulf clusters we've considered.
Now, imagine a cluster of Athlon 1.4GHz machines doing the same thing ... now there's a drool factor, and probably cheaper to boot!!
What I'd like to see is a shot at a distributed supercomputer cluster utilizing the spare cpu cycles of computers on high-speed internet connections (cable or DSL). Since efficiency would be remarkably degraded by slow communication times and the fact that many of these computers would be running Office (ahem), you'd have to scale up at least one order of magnitude.
Technically I can't see why this wouldn't be feasible. It would be beyond SETI and protein folding in that the 'control center' could change what problem was being worked on at any time. It may not be incredibly practical compared to setting up specific machines in a single large room, but it would be free and have a potential user base in the hundreds of thousands or millions.
Imagine: instead of the same SETI screen output time and again, you'd get a message on your SS saying "would you like to see what your computer is working on right now? How about high-pressure fluid dynamics in environment x?"
Max
My god carries a hammer. Your god died nailed to a tree. Any questions?
Yes, we all saw the Apple ads for the G4 being capable of 1GFlop. What you didn't see, was that the Pentium III 500 was capable of ~2GFlop. Now that can run an 1GHz. You also didn't see that AMD's Athlon, having a superscalar FPU, is faster than a P3. And now they can run at 1.6GHz. The P4 has new instructions to speed up certain types of multimedia processing as well. By contrast, the G4 is only now approaching 1GHz. Go figure (as you Americans say.. :o)
An Apple is not a supercomputer.
RISC does not mean faster. It allows for simpler design which can lead to increased speed, but as we have seen, Apple have consistently failed to compete with Intel and AMD (not that they even make thier own chips...). CISC is actually a good idea, since with the huge speed differential between CPU and memory, and the introduction of cache, the bottleneck in any system is the memory bandwidth. Think for a moment : why did Intel add instructions to the x86 architecture in every iteration? Because its faster having one instruction doing something complex, than many simple ones, simply because of the reduced frequency of memory access. In todays computers, RISC doesn't mean anything, since memory, storage and network bandwidth is the bottleneck.
The moral of this story:
1: Don't believe Apple's advertising.
2: Don't believe what a Mac Zealot will tell you about RISC or some other claptrap.
3: Get ppc Mandrake if you're unfortunate enough to have actually bought a G4.
Yes, I use Macs. Daily. And I hate Apple. But my PHB is a Mac zealot. It frightens the hell out of me seeing all our company's work being stored on a Mac (OS9 (no pre-emption, memory protection, RAID, journalling, or anything you would want for a server...)).
"I think he was truly surprised at how little I cared about how big a market the Mac had" - Linus on Jobs
While I agree with what much of flegged said, his/her post implies that modern Intel/AMD CPUs -are- largely CISC devices. This simply isn't the case. Both (the AMD moreso though) make heavy use of RISC-type design and technique.
RISC does matter, or Intel and AMD wouldn't be using it.
- Turq - "That's TRON, he fights for the users."
I wasn't able to get hard facts about this, so I'm going to throw out the question for general "gee whiz" value.
I was pondering the computrons per watt of a cluster such as this versus a real honest-to-Bob supercomputer (Something from Cray/Terra/SGI, for example). we can assume that each machine in HPs cluster uses probably 60-80 watts (because they're sans monitor), so youre looking at about between 1.2 and 1.8 kilowatt hours to power this thing. I'm not sure what a Cray TSE uses, but I have to think it's nowhere near that because of all the redundancy that PC clusters use (one Power supply, chipset, etc per Core).
Though, I'm sure if you can afford either a Cray or 256 PCs, you can afford the power bills, too. If you have to ask how much it will cost you, you can't afford it. But while CIP (Cluster of Inexpensive PCs) is cheaper, is it as efficient?
.
Hilary Rosen's speech was about her love of money and her desire to roll around naked in a pile of money.
As we all know, "kit" is a british slang term for computer hardware. What many people may not know is that it is also the secret weapon in a British campaign of cultural assimilation.
Yes, you heard me right. Cultural assimilation. The brits are sick of seeing Mickey Mouse and Donald Duck and the sexy chick from Enterprise on TVs all over the world, and they're going to do something about it.
The British invented the English language, and in many circles certain British accents are percieved as more sophisticated or upper-class. They're capitalizing on that by inventing slang terms - "kit" being among the forerunners - that other English-speaking peoples appropriate. Thus is begins.
Soon, British TV will move off of PBS, where it belongs. British computer games and hardware will surpass American in popularity. And there is nothing - absolutely nothing - we can do about it.
(In case you hadn't realized it, yes this is a joke. And yes, I know it's offtopic and will be moderated as such. But this was fun to write.)
I'm the stranger...posting to
using a bunch of those 1U dual athlon rackmount boxes for this? seems like it would reduce the overall footprint by several orders of magnitude, as well as easily doubling (if not tripling) the power. comments, anyone?
I stole this sig.
Then, the US gets tired of bombing, and HP sells them new machines. Soon thereafter, we decide their new "good" dictator is just as bad as their old "bad" dictator, and the cycle begins again.
It's probably out of date because processors that speed are either already unavailable or will be shortly. They could presumably underclock, but it makes more sense to just tweak the model number slightly.
fencepost
just a little off
Hey remember all those completely and hopelessly out of work Russian PhD CS grads sitting around and starving and writing strong crypto software for the Russian Mafia? You might even have heard that the Russian Mafia is always looking to explore new business ideas and strategy.
Well hell wouldn't this be a great business opportunity for both of them?Call it RMBM (Russian Mafia Business Machines), and then build cheap super-clusters and turnkey code for "specialized" clients. The possibilities are endless.
This is where you get them now: Support. You sell them the machines at a 25% markup and then charge a ridiculous annual service agreement.
From the presentation:
"Using "borrowed" Post-CCCP Mi-8TV assault/commando choppers RMBM support staff can be deployed to your corner of the desert in a matter of hours! Lets see IBM match that! Not even Larry Ellison and his personal Mig can touch that! (canned laugh track)"
I don't know, maybe not.
I guess this is what you do with all of that extra inventory. Clusters coming from Gateway and Dell next.
Anyone who posts a comment containing the word "Beowulf" will be shot.
Including me.
Uh-oh.
As we all know, "kit" is a british slang term for computer hardware.
No it isn't. That's just the only context in which you've heard it used (translation: you read too much Slashdot, and should get out more often). "Kit" is the British equivalent of the American "rig" when used in this context. It is not used specifically to refer to computers.
Edith Keeler Must Die
i think i could build a better supercomputer
for less money with amd procs/mobos/etc
1gig tbird $100
decent cheap amd mobo w/integrated vid/snd/net $100
256meg ram $25
15gig ide $50
floppy drive (needed??) $15
cdrom (needed??) $25
decent nic $20
cheap case $40
total $375
subtract 10% (due to quantity purchase) gives less than $350 total each
pay a bunch of college kids $10/hour
they'd build 2 machines/hour
so 125hrs total to build comps is $1250
$350 x 250 machines is $87,500
add in (8) good quality 32 port switches @ $200 each and you're up another $2k
add in 250ish cat5 cables for another $1k (who wants to make them, buy for $3-4 ea)
your total cost is way under $100k
or even better
use the new SMP durons, 1gig each
not much more $$ since durons are cheap
add like $50 for the 2nd proc (total $150 for 2 duron 1gig smps, unsure if thats reasonable pricing) and another $50 to mobo cost for dual smp mobo
thats $450 ea box
250 x $450 gives us $112.5k for the boxes
add in networking stuff etc
less than $125k prob
man i want to do this
need someone with $$ =P
--wayne =)
E V E R Y T H I N G I W R I T E I S F A L S E
if they were easy to get, don't you think they would have used one?
The Kruger Dunning explains most post on
You know this Beowulf business is getting to be pretty staid and routine by now.
In fact, I'd almost say it would be newsworthy if there were any organization (university, company, govt lab) that had not yet built "a supercomputer from the COTS components".
What I'd like to see now is more metrics (some of which the article does, admittedly, reveal).
- hardware cost per FLOP (everyone already tells you this)
- FLOPS per human time to build
- FLOPS per sysadmin time to maintain
- FLOPS per kilowatt of electricity
- FLOPS per cubic foot of rack space
- can it run smoothly if Bad Andy goes behind the rack and unplugs a few network connections, a few power cords to some nodes?
Everyone knows that you can spend your own time scouring dumpsters for cast-off computers and coaxing them to life, bringing up an old 486 with an ISA 10bT card as a member of your cluster. Unless you're doing it for your own educational benefit, it's just not worth it.Don't get wrong. I love these clusters and want to use them. It's just that, in 2001, their mere existence is no longer as exciting as it was in the mid 1990s.
Now days, I care more about ease of use and ease of maintenance, taking the low cost of a Beowulf cluster as a given.
With the size of these clusters going up and the ratio of hardware cost to human time constantly decreasing, I'd be more impressed to see how a system with many hundreds of nodes was brought up in a short time, never rebooted for a year, even as 13 of the nodes developed variously problems and become unproductive members of the cluster.
"Provided by the management for your protection."
I fail to see what is impressive about this.
... 225 @ 733 mhz? That makes it to #325?
It looks like the wheel reinvented several times.
For cluster installs on several machines, use system imager .
For using and controlling a cluster of machines for various taskes, use LSF .
The number of machines is pathetic too
How sad. I need to bench mark our render farm (200+ boxes, 120 are dual 1ghz) and see what we can come up with. I know it is higher than that... and we have a smaller install for the industry.
I looked for info to spec our machines but I couldn't find any info.... any help?
-I just work here... how am I supposed to know?
No, it isn't. Yahoo! News is repeating a story which, if you'd bother to read the byline they wrote, was
The article on CNET's site should be getting the Slashdot treatment, don't you think?
Edith Keeler Must Die
I refer to "the british" meaning the residents of the island now known as England, not in the sense of citizens of the modern political entity. I probably shouldn't have done that, but hey - it wasn't meant to be accurate anyway.
I'm the stranger...posting to
You're right, I should get out more.
Of course, you know that makes the threat presented by the word even more insidious. If non-techies can use it well - I shudder to think of the potential for linguistic infiltration!
I'm the stranger...posting to
No, a beowulf cluster is the last thing that one would use for nuclear simulation.
While great at highly parallel tasks that require very little synchronization between threads (think code cracking), nuclear testing (and almost all other fluid dynamic problems) generally requires all of the cpu's to have high speed access to all of the memory. So one needs a huge shared memory system (think Cray or Sun StarCat).
And for this reason, I find the top 500 list to be a bit misleading in these days of massively parallel systems. Its great as a test of how many flops the system can crank out, but it does not take into account the memory bandwidth between the cpu's, and that is often more important than raw cpu horsepower.
Is there anything like a MIPS/Wh rating for CPUs? (Would thermodynamics dictate a certain minimum?)
With a seperate power supply and hard disk per CPU (i.e. complete box) I would imagine that old PCs generate a *lot* of heat per CPU cycle.
Has anybody done measurements/calculations on this?
bla
The United Kingdom consists of Wales, England, the Isles of Man, Scotland, and part of Ireland. So There!
Hail Brittania!
I'm the stranger...posting to
On the other hand, for a weather simulation, I would bet on the cluster.
No way. Weather sinulations involves lots of linear algebra on huge matrices. When you parallelize that, you need a lot of communication between the nodes. With a cluster of P100, communication will kill you right there (10x of so penalty). It's not that much the network bandwidth, but the latency. Weather simulation is one of the hardest problems to parallelize and that's why until recently, SMP was prefered to MPP (and of course clusters of small workstations).
As for the memory bandwidth, depending on the problem, sometimes the L1 is really effective.
Opus: the Swiss army knife of audio codec