Slashdot Mirror
How To Build a Homebrew PS3 Cluster Supercomputer
eldavojohn writes "UMass Dartmouth Physics Professor Gaurav Khanna and UMass Dartmouth Principal Investigator Chris Poulin have created a step-by-step guide designed to show you how to build your own supercomputer for about $4,000. They are also hoping that by publishing this guide they will bring about a new kind of software development targeting this architecture & grid (I know a few failed NLP projects of my own that could use some new hardware). If this catches on for research institutions it may increase Sony's sales, but they might not be seeing the corresponding sale of games spike (where they make the most profit)."
Couple issues with this as an alternative to the garden-variety x86 cluster connected with InfiniBand:
Slow network interconnect. For problems that are not trivially parallel, network latency is usually a big deal. Ethernet doesn't cut it.
Lack of RAM. 'Nuff said.
Have to care about Cell and PS3 architecture. The codes ("codes" has a slightly different meaning in the context of supercomputing) have to be modified to take advantage of this very specific architecture. Software always outlives hardware, so in the long run the effort may not be worth it.
That said, it's really cheap. If your application isn't held back too much by these issues then enjoy your insanely cheap cluster!
Why would you want to use PS3s for a homebrew supercomputing cluster if it means you have to write and optimize code for the SPEs to get benefit out of it? The PS3's linux environment doesn't let you utilize the GPU or all of the built-in SPEs and it doesn't have a lot of RAM available either.
Well, I'll bite; if the cell is the fastest processor for your workload, the PS3 is the cheapest way to get one, even at only six usable SPEs and no GPU. Doesn't the PS3 have GigE? That's plenty fast enough to shovel data in and out of the system.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
A) Although the cell is a pain to code for, it is much better than whatever PC you can get for ~$400 which will probably contain a mid-to-low-range dual core x86 CPU, whereas the PS3 gives you a Cell CPU which is much, much, faster than the x86 CPU.
B) PS3s are uniform. Other than HD differences, a PS3 built in 2008 will be the same PS3 built in 2012 (assuming the PS3 lasts that long) this allows for a uniform cluster without worrying about differing parts (for example, the Core i7 built in 2008 will not be the same as the Core i7 built in 2012 and getting a 2008 Core i7 is going to be a pain)
C) PS3s are the new fad. It isn't going to be hard to set up a supercomputer cluster with PS3s compared to using a mismatch of older computers because again, the PS3 is uniform.
Taxation is legalized theft, no more, no less.
I'm not trying to be a smartass, but why did he mention in TFA that his supercomputer cost $4000 if the 8 consoles were "Sony-donated"?
Oh come on, you are being pedantic. Clearly what he meant was "$4000 worth of consoles", never mind that they were donated. $X worth of consoles is a useful number if someone is considering buying PS3s and setting up a supercomputer; it's also a fun number to compare to the cost of renting time on some large supercomputer.
The original Wired article is informative:
http://www.wired.com/techbiz/it/news/2007/10/ps3_supercomputer
He asked for Sony to donate the PS3s because he didn't think the NSF would give him grant money to buy video game systems. Now that he has actually built the supercomputer and it does everything he hoped it would do, perhaps other researchers will be able to justify the money to set up their own clusters (without donations from Sony).
The numbers are a no-brainer: he used to spend $5000 to do a single simulation run using rented supercomputer time. For less than the cost of a single simulation run, you can set up your own supercomputer and make simulation runs whenever you feel like it.
ALso, like the iPod example at the top of the post, most research use of the technology won't come from actual iPods or consoles
Um, he is using actual PS3 consoles to do actual research.
If one wanted to build their own home "super" computer then why not just use CUDA and a few Nvidia cards?
If you think that is a good way to make a super computer, why don't you go ahead and do it, and make a web site explaining how it is done?
Meanwhile, he thought he had a good way to go with the PS3, and it did in fact work as he expected, so what's the problem?
Anyway, here's why he thought it was a good idea. From the above linked Wired article:
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Old CPU's have a much lower MIPS/Watt and a lower MIPS/interconnect so they have a higher cost. Many organizations have found it's cheaper to retire an old supercomputer and add a few nodes to the new one even if it is more capital outlay to get the same performance. Basically a Cell does many times as much useful work than a P4 at a fraction of the power budget.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
The number of PS3's sold will never be enough to hurt Sony's bottom line, but will boost the image of the console. Having credible scientists call your product a "supercomputer" is worth something. Does Ferrari's Forumla 1 racing team pay for itself? Nah, it's an investment to promote an image.
How is it useless, when the guy who built it, used it already for a month? And it has replaced 200 supercomputer nodes, for his purpose? I'd say that's very fucking useful.
But you know what, maybe you should send him an e-mail and try to convince him how his cluster is useless. Make it a nice, insightful and intelligent e-mail, like your post.
"The agriculture ministry is not in charge of Gundam" - Japanese ministry official.
Not quite. CUDA looks a lot like C in that it has C-family syntax but the biggest limitation it has is that there is no application stack - which means no recursion. CUDA also lacks the idea of a pointer, although you can bypass this by doing number to address translation (as in, the number 78 means look up tex2D(tex, 0.7, 0.8)). The GPU also has other shortcomings, in that most architectures like to have all their shaders running the same instruction at the same time. For this code
if (pixel.r < pixel.g){
//do stuff A
//do stuff B
//do stuff C
}else if (pixel.g < pixel.b){
}else{
}
The GPU will slow down a ton if the pixel color causes different pixels to branch in different directions. Basically, the three sets of shaders following different branches of that code will be inactive 2/3 of the time.
In the Cell, you really do just program in C with a number of extensions added onto it like the SPE SIMD intrinsics and the DMA transfer commands (check it out). The Cell really is 9 (10 logical) processors all working together in a single chip (except in PS3, where there are only 7 working SPEs). Furthermore, your 8 SPEs can be running completely different programs -- they're just little processors. Granted, you have to be smart when you program them to deal with race conditions and all the other crap you have to deal with for multithreaded programming. The Cell takes about 14 times longer to calculate a double precision floating point than a single (and there aren't SPE commands to do four at once like you can with singles).
So which is more powerful? It really depends what you're doing. If your task is ridiculously parallellizable and doesn't require the use of recursion, pointers or multiple branches, the GPU is most likely your best bet. If your program falls into any of those categories, use a Cell.
My UID is a prime number. Yeah, I planned that.
IBM is probably banking on the existence of people who want Cell processors in systems with more than 256megs of RAM. Other IBM value-adds would presumably include rack mountability, support for netbooting and other convenient management stuff, and so forth.
If your application leans almost entirely on the CPU with very little need for RAM, and you have an army of screwdriver monkeys(or grad students) to do all the legwork, the PS3 is an excellent deal. If you need something with RAM capacity that wasn't a joke in 2001, and/or management features that won't have you tearing your eyes out when you have 10,000 of them, then IBM smells opportunity.
sorry, but that's stupid -how many pop culture references from 1923 are relevant to TODAY's pop culture:
seeya snookums, me and the squeeze are the bees knees in our raccoon coats, we're gonna get jazzed up in our hupmobile on hootch and go check out Mary Astor's horse after we hit the blind pig.
I agree its unfortunate that this stuff is non free, but pre 1923 means that most talkies would be out of bounds as well -including stuff you can see on tv all the time.
I'm just sayin'