Slashdot Mirror


Eight PS3 'Supercomputer' Ponders Gravity Waves

Jamie found a story about a inexpensive supercomputer being used by an astrophysicist to research gravity waves. The interesting bit is that the system is built using 8 PS3s. Since nobody is actually playing games on the system, it makes sense to use them for research projects like this, but I really wonder now what is defining 'Supercomputer'... I mean, a hundred PS3s sure, but 8? I think we are de-valuing the meaning of the word 'super' :)

35 of 293 comments (clear)

  1. Inexpensive, eh? by EveryNickIsTaken · · Score: 5, Funny

    I mean, a hundred PS3s sure, but 8? I think we are de-valuing the meaning of the word 'super' :) I think we are de-valuing the meaning of the word 'inexpensive' :)
    1. Re:Inexpensive, eh? by The13thSin · · Score: 5, Insightful

      For a supercomputer that's pretty cheap. Also I find the statement in the summary that there are no games to be played on the PS3 a bit childish. The PS3 has not been out for a year yet and there are multiple great games to get for it right now and even more coming very soon. I expected more from the Taco.

      --
      "This should be fun, and by fun, I mean a wholly depressing insight into the cognitive ability of some grown adults."
    2. Re:Inexpensive, eh? by NeilTheStupidHead · · Score: 3, Insightful

      I think that the article imples that PS3s aren't flying off the shelves as fast as Sony might like and thus are sitting in a warehouse somewhere, otherwise going unused. Even the article claims that this was done mostly because of the open platform presented by Sony and the fact that this researcher was able to get the consoles free from Sony. This is great for Sony because a sold console is money in Sony's pocket regardless of who buys it and what they do with it. If they can convince researchers to buy PS3s then it's probably a better deal than selling them to gamers. Few gamers would buy the equivalent of 7 PS3s (about $2800) worth of games and accessories. Some will, but most won't; even when they do, it's spread over the life of the console. A researcher goes and gets 8 consoles, cash up front and there's $3200 for Sony (less taxes, mfg. costs, etc).

      Maybe it's jsut me, but that sounds like a pretty good deal from Sony

      --
      Lose: misplace or fail || Loose: not bound together
    3. Re:Inexpensive, eh? by smussman · · Score: 5, Interesting

      I think that the article imples that PS3s aren't flying off the shelves as fast as Sony might like and thus are sitting in a warehouse somewhere, otherwise going unused. Even the article claims that this was done mostly because of the open platform presented by Sony and the fact that this researcher was able to get the consoles free from Sony. This is great for Sony because a sold console is money in Sony's pocket regardless of who buys it and what they do with it. If they can convince researchers to buy PS3s then it's probably a better deal than selling them to gamers. Few gamers would buy the equivalent of 7 PS3s (about $2800) worth of games and accessories. Some will, but most won't; even when they do, it's spread over the life of the console. A researcher goes and gets 8 consoles, cash up front and there's $3200 for Sony (less taxes, mfg. costs, etc).

      Maybe it's jsut me, but that sounds like a pretty good deal from Sony IIRC, Sony sells consoles at a loss, and then gets that money back over the life of the console with license fees from games. So selling 8 consoles which will not generate money from game license fees, but still having to take the loss is not a good deal for Sony.
    4. Re:Inexpensive, eh? by Rogerborg · · Score: 4, Funny

      If you thought that was childish, then this should drive you into a perfect fit of apoplexy.

      --
      If you were blocking sigs, you wouldn't have to read this.
    5. Re:Inexpensive, eh? by zippthorne · · Score: 3, Insightful

      Yeah, but is that real money or is that subdivision expensing. In other words, does it lose $240 because Sony must use $800 of resources to produce a $600 product, or because sony-chipfab charges sony-board-assembly $60 for a part that cost $5 to produce?

      --
      Can you be Even More Awesome?!
    6. Re:Inexpensive, eh? by smussman · · Score: 5, Informative

      Yeah, but is that real money or is that subdivision expensing. In other words, does it lose $240 because Sony must use $800 of resources to produce a $600 product, or because sony-chipfab charges sony-board-assembly $60 for a part that cost $5 to produce? I found the original analysis of the PS3 cost here. At least 50% of the components are made by other manufacturers.
    7. Re:Inexpensive, eh? by SailorSpork · · Score: 5, Informative

      How much something "costs" to produce isn't as simple as the cost of parts. It may cost $5 for just the parts, it may also cost and extra $1-5 for the direct labor to put the board together, and it may cost and extra $1 to ship it. Then there's the cost of maintaining a chip fab, indirect labor and mangement costs and bribes to the chinese government to keep their cheap wage factory certified, divided over the 2-300 PS3's produced per month to keep up with worldwide demand, not to mention the cost of worldwide marketing, the cost of years of system & Blu-Ray R&D spread over each unit... keep in mind that the cost of making your PS3, Wii or software title is more than just more than the sum of the cost of the individual parts.
       
      Sorry, that was my Cost Accounding class talking, I'll stop now.

  2. 8 systems x 8 cores = by zifferent · · Score: 5, Insightful

    64 cpu's. That seems supercomputerish enough for me.

    --
    cat sig > /dev/null
    1. Re:8 systems x 8 cores = by andphi · · Score: 4, Funny

      Should be enough for anyone

    2. Re:8 systems x 8 cores = by Waffle+Iron · · Score: 4, Funny

      64 cpu's. That seems supercomputerish enough for me.

      It depends. For those problems that fit within the PS3's cramped memory, this is a supercomputer. For those problems that don't, this is a set of 8 matching doorstops.

    3. Re:8 systems x 8 cores = by Kupek · · Score: 5, Informative

      It's only six cores available to Linux per Cell processor on a PS3. One is reserved for the Game OS, and one is disabled to achieve a higher yield on fabrication. (The Game OS is always running, since Linux actually runs on top of a hypervisor.)

    4. Re:8 systems x 8 cores = by flaming-opus · · Score: 3, Insightful

      64 cores does not make a supercomputer. There are database servers with more cores than this, and have been for years. Technical computer, sure. Maybe even high performance computer. Definately NOT supercomputer. 8 systems, that's what? 4GB of RAM? There are laptops that can hold that much memory.

      If you went to a technical conference like, for example, Supercomputing '07, you would get laughed off the floor calling that a supercomputer. Supercomputer is a changing definition, but I don't think I'd call anything a supercomputer that didn't have at least 1TF of peak double-precission performance, and at least 200GB of RAM.

    5. Re:8 systems x 8 cores = by Ted+Stevens · · Score: 3, Funny

      5, Funny? Stop joking... You don't understand the gravity of the situation.

  3. Strange... by Anonymous Coward · · Score: 4, Funny

    Noone has wished for a Beowulf cluster of these yet. Has the world order changed and no notification issued?

  4. Supercomputer is a term that changes by szyzyg · · Score: 4, Interesting

    Just over a decade ago the fastest computers in the world were barely breaking the teraflop mark, today in theory the XB0x and Ps3 with their multitude of cpu cores and finely tuned graphics cards can top that. So 8 Ps3's - if you believe sony's hype could clock in a >10teraflops if the hardware was well utilised.

    I had a freiend who wrote a book 'Nemesis' which was a spy thriller involving a killer asteroid - it was published in the UK 1998, and back then he was talking about 'the teraflop box' as being the fastest computer in the world, unfortunaly it took 8 years to get the book released in the US and by that time a lot of the computer jargon had dated significantly, and you could get a teraflop box in the form of a turbocharged graphics card or cutting edge games console.

    1. Re:Supercomputer is a term that changes by Stefanwulf · · Score: 4, Informative

      Sony's estimate of 1 teraflop per unit is more than a little bit optimistic, and also counts all the operations performed by the graphics system, which the average researcher won't have access to. For a slightly more realistic estimate of what 8 ps3's running linux can do, I'd point you towards http://www.terrasoftsolutions.com/store/index.php?submit=software&submitimg%5Bhardware%5D%5Bsolutions%5D=1, who sell turnkey ps3 clusters. They are claiming that 8 units together break the theoretical teraflop mark, which seems more realistic to me. While that isn't among the fastest computers in the world by any stretch, it's still solidly in the realm of what I'd call a supercomputer.

      The cell is a fantastic piece of equipment - Dr. Dobb's has what I think is an excellent analysis of the kinds of performance benefits that it offers at http://www.ddj.com/hpc-high-performance-computing/197801624. I'm currently running one at home in a PS3 (for neural networks that drive an AIBO - I love Sony's tendency to dump hugely expensive hardware at mass-market prices), and I have every intention of picking up more used ones over the coming months to cluster together as the networks continue to grow. Even all by it's lonesome with code that's far from optimized, the one I have is running about 10 times faster than my main desktop for roughly equivalent computations.

      (Note that your mileage may vary - I just happen to like playing with systems that parallelize really well)

  5. Devalued super by Teese · · Score: 3, Insightful

    I think we are de-valuing the meaning of the word 'super'
    I'm pretty sure we devalued super when the PowerMac G4 was claimed as a supercomputer all by its lonesome.

    Super is a relative term, what was a super computer is now a computer that I hand-me-downed to my mom so she could check her email and browse the web.

    --
    "I'm a Genius!"*


    *Not an actual Genius
  6. Mystery solved by eaglesnax · · Score: 5, Funny

    Now we know who bought all the PS3s!

  7. devaluing super by mihalis · · Score: 3, Insightful

    Well the guy used to use a 200-node parallel supercomputer, but now he prefers to use 8 PS3s. That to me proves that 8 PS3s is like a supercomputer TO HIM.

    I'm sure there are faster setups available if had the money, but 100% of 8 PS3s indefinitely is preferable, from what he says, to the costly little slices of "real" supercomputers he tried to rent before.

    I wonder if Sony could offer a "HPC PSP3" which provided a stripped down processor board without the shiny case, graphics memory etc. It would be interesting if the Cell processor could get better economies of scale.

    1. Re:devaluing super by TargetBoy · · Score: 3, Insightful

      Wouldn't it rather be IBM that might offer this, since they actually make the cell?

    2. Re:devaluing super by mihalis · · Score: 3, Informative

      Wouldn't it rather be IBM that might offer this, since they actually make the cell?

      Yes, actually I think you are correct. If I recall correctly it's Sony, IBM and Toshiba in the cell consortium, and the most ovious vendor of a "compute-node Cell module" would indeed be IBM, not Sony, good point.

      By the way, I had a typo, it would not be an "HPC PSP3" of course, the Cell is way too hot and power hungry! Although ... of course with sufficient shrinks and price reductions the current Cell might well one day be in a portable game console. Then we could have another round of speculation on personal clusters. I love the "wheel of reincarnation" in digital technology!

    3. Re:devaluing super by wilsonjd · · Score: 3, Informative

      They already do, but it is a bit more expensive than 8 PS3s: http://www-03.ibm.com/systems/bladecenter/cell-based.html

  8. Re:Obligatory by threaded · · Score: 4, Interesting

    Not true. I know of at least one place where they have a Cray and can't afford the electric bill to switch it on. They cost a fortune just to sit and look pretty too: it's taking up room on campus that could be used for other things.

  9. Research grant? by threaded · · Score: 3, Funny

    Maybe his research grant doesn't stretch as far as heating the office. Win-win situation in that case.

  10. "We Report. We Decide." by Intellectual+Elitist · · Score: 3, Informative

    > Since nobody is actually playing games on the system

    You can always count on Slashdot for a fair and balanced presentation of information.

    Funny that I've bought 4 disc-based games and at least one downloadable game since the beginning of July, and have been using my PS3 almost exclusively for gaming since then. I'll be buying at least 4 more games before the end of the year, too.

    Sorry to burst your bubble, but the PS3 game drought has been over for a while now...

  11. Memory limitations by Kupek · · Score: 3, Interesting

    I'd like to know more details about his code, because a PS3 only has 256MB of RAM. That's a serious performance obstacle, since most high performance applications that do anything interesting need much more than that. I know it's a problem our group has had, and we've heard the same from others.

    1. Re:Memory limitations by chrysrobyn · · Score: 3, Insightful
      256k of local storage for each SPE is a problem, but you can code around it. (It's not trivial, but it can be done.) You can't code around having a small amount of RAM and still maintain high performance.

      Let me see if I get this straight, you can imagine a piece of code that doesn't mind churning on itself within 256KB, but you can't imagine having to keep 256MB of main memory fed from a network or disk? In my experience, any piece of code that can both benefit from extreme parallelism and fit both the code and enough data to be worth working on within 256KB can handle a few reads from a disk or the network once in a while. If it can't, then 256KB of memory isn't enough to keep the (sub)processor fed, and you need a machine with more on-die memory (many of which can be found).

      Cell is very good at integers and single precision floats for workloads that are parallelizable and fit within 256KB. If you stray from any of that, there are plenty of interesting competitors.

  12. G4 was a supercomputer ... at the time. by oneiros27 · · Score: 3, Informative

    It met the government's definition of super computer at the time. (1.5Gflops ... well, technically, 1500MTOPS).

    The designation is part of the "Dual-Use" restrictions on exports (basically, things which could be used for both military and non-military applications).

    The 1Gflop threshold was set as the necessary processing power to calculate balistic trajectories for missile systems.

    I can't find the documentation, but my understanding is that the current threshold is 190Gflop (since Jan 2002).

    --
    Build it, and they will come^Hplain.
  13. 1.2 TFlops by rockmuelle · · Score: 4, Informative

    8 PS3s gives you 1.2 teraflops of single-precision performance or a similar number if you stick to integer operations (6 SPUs/PS3 gives ~150 GFlops). 1.2 teraflops is a supercomputer in my book.

    Using Jack Dongerra's single-precision algorithms that do half the work in single and the other half in double precision, you can maintain a high level of performance and precision. And, the unique architecture of the Cell opens up some interesting algorithmic research issues, allowing scientists to publish twice for the same work: once for the science results, once for the computer science results. :)

    On the flip side, the Gigabit ethernet on the PS3s isn't really 1GB - the PPU can barely keep up. So, extra care must be taken around communication points. And, a similar Intel/AMD-based rack would run about $20k and is much easier to develop for, so if your labor is expensive (i.e., you're not in academia), PS3 clusters may not make much sense.

    -Chris

  14. 9 cores? by oblivion95 · · Score: 5, Funny
    They should have used 9 cores on the Cell. Then, they'd be using Seven of Nine.

    It's only six cores available to Linux per Cell processor on a PS3. One is reserved for the Game OS, and one is disabled to achieve a higher yield on fabrication.
  15. Re:Obligatory by Maller · · Score: 3, Insightful

    Please enlighten me. Who is stupid enough to by a million+ dollar computer without factoring in facility costs?

  16. Re:"We Report. We Decide." by Intellectual+Elitist · · Score: 4, Insightful

    > 8 games? Yikes, either you play *everything* or you've got some real crud in there. Care to elaborate?

    The four games I was referring to were Ninja Gaiden Sigma (88), Skate (85), Stuntman: Ignition (75), and Warhawk (84). The downloadable game was Super Stardust HD (84). None of those games are even remotely close to "crud".

    The four games I referred to having an interest in purchasing before the end of the year are Ratchet & Clank Future: Tools Of Destruction, Uncharted: Drake's Fortune, Rock Band, and the collector's edition of Stranglehold. I might also consider Army Of Two, Assassin's Creed, and Call Of Duty 4, depending on the reviews.

    > To give an idea, the top 8 games on PS3 get metacritic scores of 85 or more [...] Only one of those is over 90. To compare, the 360 has *27* games at 85 or more [...] 9 of which rate 90 or more.

    The original post had nothing to do with the 360 -- it was about the insinuation that no one uses the PS3 for gaming, which is ridiculous.

    You're also making an apples to oranges comparison, because the 360 has been out longer and has a much larger base of titles. But if you want to compare, as of October 13th Metacritic's aggregated ratings for the 360, PS3, and Wii show that the 360 has 264 rated games, the PS3 has 82, and the Wii has 87. Since the PS3 and Wii came out later than the 360 and around the same time as each other, this makes sense.

    If you look at the percentage of each console's library that has a metascore of 75 (out of 100) or higher, the PS3 leads with 54%, followed by the 360 at 44%, then the Wii with only 16%. If you go with a metascore of 80+, the PS3 has 34%, the 360 has 27%, and the Wii has only 8% above that level. At 90+ the Wii has 3%, the 360 has 3%, and the PS3 trails with only 1% of its library at that level.

    Going by percentages, the PS3 and 360 libraries are of roughly equivalent quality, while the Wii's lags far behind.

    > the general sentiment is the PS3 needs a killer app (like a halo, gears of war, or some other really good exclusive title) to make it worth getting.

    The general sentiment is also that Iraq was involved in 9/11 and that Britney Spears's personal life is somehow newsworthy. I'll think for myself, thanks.

    That said, every console gets a "killer app" eventually. I'm sure the inevitable God Of War III will fill that void if nothing else does beforehand.

  17. Re:Obligatory by adisakp · · Score: 5, Informative

    It would've been cheaper to just buy a Cray.

    If you read the article and followed the link to his PS3 Gravity Grid site, you'd know a couple things about the cost (FREE) for this computational power:

    #1) The total cost of purchasing an entire "PS3 Gravity Grid" supercomputer for yourself is less than the cost of a single simulation run on a BlueGene. In other words, you can buy the cow, the pasture, and a barn for the price of a gallon of milk.

    #2) Sony *DONATED* his 8-node cluster (albeit with 20GB PS3's which they were closing out at the time) so he actually got a "supercomputer" for nearly free.

    #3) The power of the 8-node PS3 cluster is roughly the same as a 200 node partition on a BlueGene SuperComputer (1 PS3 = 25 Blue Gene nodes). With 8 Cell CPUs, he has 56 SPU's running at ~3GHz to crank through his computations. This would mean a single CELL SPU is roughly 4X more powerful than a single BlueGene node which isn't unreasonable considering that it runs at a higher clockspeed (the supercomputer has to worry more about heat dissipation with hundreds or thousands of cores).

    #4) I believe that by the US Gov't's somewhat outdated standards, a PS2 qualifies as a supercomputer. The FPU power in a PS3 is on ther order of 200 times higher than that of the PS2 for single precision and considerably more for double precision (which is emulated in software on the PS2).

  18. Re:Obligatory by adisakp · · Score: 3, Informative

    Double precision floating point is emulated in software on the PS3 as well.

    As a professional programmer working in the games industry (on both XBOX 360 and PS3), I can tell you that's completely untrue. You can verify this easily with information available to the general public on the CELL microprocessor.

    The CELL supports Double Precision in hardware. However, the SPU vector instructions only run on Single Precision which allows for up to 8 SP ops (4 X Multiply+Add's) per cycle. Double Precision is scalar (non-vectored) and also not pipelined so the throughput is much slower since DP operations can cause stalls until they complete (there are rumors that IBM is working on a CELL that pipelines DP which will help immensely). Properly pipelined and vectorized Single Precision work can be 30-50 times faster than the scalar non-pipelined DP but the CELL still has true DP hardware which is much faster than emulation by orders of magnitude.