Slashdot Mirror


USAF Unveils Supercomputer Made of 1,760 PS3s

digitaldc writes with this excerpt from Gamasutra: "The Air Force Research Laboratory (AFRL) has connected 1,760 PlayStation 3 systems together to create what the organization is calling the fastest interactive computer in the entire Defense Department. The Condor Cluster, as the group of systems is known, also includes 168 separate graphical processing units and 84 coordinating servers in a parallel array capable of performing 500 trillion floating point operations per second (500 TFLOPS), according to AFRL Director of High Power Computing Mark Barnell."

37 of 163 comments (clear)

  1. Don't Update by PaddyM · · Score: 5, Informative

    Don't get the firmware update that gets rid of Linux. OOPS!

    1. Re:Don't Update by colordev · · Score: 3, Informative

      Good downgrades are available as Sony is loosing the PS3 jailbreak fight by a technical knockout

    2. Re:Don't Update by HazMat+79 · · Score: 2

      Well if ya just guess they paid $400 for each PS3 that would put it north of $700,000 for PS3s alone. Not counting the other GPUs and servers. Just wondering because I have never been in the market for a supercomputer, but how much would it cost for a "normal" supercomputer.

    3. Re:Don't Update by palegray.net · · Score: 2

      As of 2007, IBM's Blue Gene/P system cost $1.3M per rack, and the Blue Gene/L cost $800K (per a PCWorld story entitled http://www.pcworld.com/article/135334/ibm_drops_price_on_supercomputer.html). However, it should be noted that the hardware cost of such systems doesn't reflect the total configuration and operating cost. Many news outlets have reported on the favorable overall cost effectiveness of building supercomputing clusters with PS3s. Yellow Dog Linux has features specifically designed to support the Cell/B.E. CPU.

    4. Re:Don't Update by i_b_don · · Score: 3, Insightful

      I wouldn't say "no matter how you look at it".

      Super computing is also a lot about pushing those large quantities of data around and the programming that allows you to use that theoretical 500 TFLOPS of power. You could end up with something that can do significant calculations but just uses 100baseT to push data around. That's just isn't very efficient for many uses of super computers, and certainly not a world class number cruncher. Just to give you something to compare it against, super computers today are looking to have 10 Tbps switches on backplanes. That's 10 Tbps of information passing through the switch hooking up a rack of servers.

      IMHO, hardware super computer engineering hurdles are about four things: processing power, data pipelines, memory, and dissipating heat. You can't fail any one of those four if you want something usable. (Software engineering hurdles I'll leave to experts as I am not one.)

      d

      --
      all language nazi's will burne in heil!
    5. Re:Don't Update by hairyfeet · · Score: 3, Insightful

      Hi MR AC! I think what you and many of the other posters are missing is that having Linux on PS3 was BAD for Sony and here is why: Game consoles are traditionally sold using what is more commonly known as the "razor and blades" model, in that consoles are sold at a loss and they then make up for that loss PLUS make their profits on the blades, that is the licenses for games and peripherals

      Now as I'm sure someone will point out the PS3 is no longer sold at a loss (does anybody know what they make? $5? Just because it isn't sold as a loss doesn't mean it is making anything either) but since we know these are running Linux that means every single one of those 1760 PS3s cost Sony money that they will NEVER make back. Because I seriously doubt the USAF is gonna be picking up 1760 copies of Little Big Planet or Move controllers.

      So as you can see allowing Linux on the PS3, which allowed it to be used as a non gaming machines was a seriously BAD idea on Sony's part, and they were very right to kill it. Now I would agree that killing the support in older ones through an update was wrong, but not offering it on newer machines was the right idea. Imagine if they kept Linux and setups like the USAF took off: What good would selling hundreds of thousands of PS3s do Sony when not a single one of those machines will be buying any games or Blu Ray movies? Why would game publishers care about a system whose biggest selling point was lab work? So I'm sorry FOSS guys, but in the end it had less to do with Linux and everything to do with business. Sony is already in last place, having tons of machines locked away in labs don't help their bottom line any.

      --
      ACs don't waste your time replying, your posts are never seen by me.
    6. Re:Don't Update by rhyder128k · · Score: 3, Insightful

      The publicity is worth quite a lot to them. It gets the PS3 a few mentions in the press in a context that suggests that the hardware is still considered extremely powerful. The mystical computational capability of the Cell is a large part of how Sony has promoted the PS3.

      --
      Michael Reed, freelance tech writer.
  2. Don't upgrade the firmware. by longtailedhermit · · Score: 2

    I hope they don't accidentally update the firmware. (I think the latest updates will wipe the linux installation.)

    1. Re:Don't upgrade the firmware. by SleazyRidr · · Score: 3, Funny

      Pipped by one minute. That's gotta sting.

  3. Re:So... by Monkeedude1212 · · Score: 2

    Of course not...

    its running linux!

    I don't know if WINE does HD.

    This post started as a joke - I don't know what it is now.

  4. Re:Why? by wed128 · · Score: 3, Informative

    Cheaper then the IBM CELL Blades...

  5. Re:Why? by twoallbeefpatties · · Score: 3, Funny

    Because they had a ton of leftover trade-in value at Gamestop.

    --
    Libertarians somehow believe that private businesses should be stronger than governments but weaker than individuals.
  6. Re:Why? by Monkeedude1212 · · Score: 4, Interesting

    At the time of the PS3's release, it was very affordable for the Cell Architecture and performance it provided, and you could put your own operating system on it.

    You know how Sony lost money on every PS3 sold... but then made the costs back with like 10 dollars from every game?

    And you notice how the government bought 1,760 thousand of these things (or more) for a non-gaming purpose?

    Did you hear the firmware updates and new PS3s remove the "Other OS" option?

    Or did you think that those 3 incidents were entirely unrelated?

    Man I ask a lot of questions.

  7. This is their third try. by Minwee · · Score: 5, Funny

    Previous efforts included a supercomputer made up of 1760 Wiis, but the resulting cluster had less computing power than the researchers' mobile phones and had to be abandoned.

    A second attempt at putting together a cluster of 1760 X-Boxes was scuttled when investigations showed no less than 600 of them were showing Red Rings of Doom at any given time.

    The Air Force team is confident that using PS3s is a better idea due to their size, weight, and the fact that nobody can find any games that they want to play on them anyway.

    (Are there any fanboys I haven't offended with this? I'm trying to be thorough.)

    1. Re:This is their third try. by phyrexianshaw.ca · · Score: 2

      Wow, I play a good pile of games a year.. and I've never even heard of one of those. :P

    2. Re:This is their third try. by Linker3000 · · Score: 2

      Actually, the Wii cluster was quite powerful, but the operators couldn't lift the 1760 Wiimotes glued together in order to navigate the front end menu.

      --
      AT&ROFLMAO
    3. Re:This is their third try. by morari · · Score: 2

      Pft. The Dreamcast is easily the most impressive console to have existed thus far. Far ahead of its time.

      --
      "He who can destroy a thing, controls a thing." --Paul Atreides, Dune
    4. Re:This is their third try. by CODiNE · · Score: 2

      (Are there any fanboys I haven't offended with this? I'm trying to be thorough.)

      I have a Virtual Boy you insensitive clod!

      --
      Cwm, fjord-bank glyphs vext quiz
  8. Re:So much for the theory.... by Monkeedude1212 · · Score: 2

    Who in the private sector has 1,760 PS3s hooked up together running the non-Sony OS?

  9. Re:DMCA by Desler · · Score: 2

    What's to jailbreak? These are PS3s they bought a long while ago and they wouldn't be updating the firmware since playing the latest Blu-Rays and getting on PSN isn't a high priority for this lab.

  10. Re:Why? by Anonymous Coward · · Score: 3, Insightful

    At the time of the PS3's release, it was very affordable for the Cell Architecture and performance it provided, and you could put your own operating system on it.

    You know how Sony lost money on every PS3 sold... but then made the costs back with like 10 dollars from every game?

    And you notice how the government bought 1,760 thousand of these things (or more) for a non-gaming purpose?

    Did you hear the firmware updates and new PS3s remove the "Other OS" option?

    Or did you think that those 3 incidents were entirely unrelated?

    Man I ask a lot of questions.

    You fail to understand the economies of scale.

    While it is true in a sense that Sony spent more on each PS3 than they charged for them at the beginning of the cycle of the product, most of those costs are sunk costs in manufacturing. The more PS3s they sold, the less they were losing (rather than the other way around, which is the infantile economics view a lot of people claimed).

    Now they have sold so many PS3s that the sunk costs are more than paid for, but it's not sensible to say current PS3s are profitable and older ones were not. They paid for development and equipment, and each PS3 sold at any time was revenue Sony used to recoup losses and eventually make a profit.

    Think of it like this. You buy a $100 grill and $200 in meat to cook and sell hamburgers. You eventually sell 1000 burgers at $2. The combined cost of the first burger might seem like $300 or $100.20, but it was actually $0.21 the entire time if you think long term (which, of course, is how Sony saw it). the real fear at the beginning was that the PS3 would flop like the original XBOX did (imagine if you only sold 40 burgers). but it hasn't. It's a huge success and on track to sell almost as many units as the other two Playstations.

    No, Sony did not cancel other OS to stop the Air Force from building this supercomputer. they did it to prevent analysis of their security architecture that facilitated pirating games.

  11. Re:Why? by l0ungeb0y · · Score: 2

    What's even more interesting is that in 2005 IBM and Sony tried to sway Jobs to migrate to Cell after IBM essentially called it a day on consumer line of POWER processors (PowerPC). As I recall, Sony even offered to build a PS3 emulator for the OS X platform to sweeten the deal. However, Jobs was reluctant to have the Apple brand compared in anyway to a gaming console, Cell Blades in the server room be damned.

    Fast forward to 2010, Apple migrated to x86 chips, have had huge success with their mac-mini server, but have just killed their XServe line, largely because it lost it's excellent clustering features when they went x86. Apple could be wiping the floor in the grid and super computing market with low cost mac minis. Instead, they are off on the sideline still not able to play with the big boys.

  12. Why are Sony so horribly short-sighted by mewsenews · · Score: 4, Insightful

    As far as I know, they're the only console maker that has a branch of the American armed forces using their hardware for a literal supercomputer cluster, which is a stunning, resounding endorsement for the real world horsepower behind their hardware, and they've disabled the very "other OS" feature that allowed the air force to build the cluster in the first place.

    What the hell, Sony, you idiots.

    1. Re:Why are Sony so horribly short-sighted by blair1q · · Score: 2

      ITAR.

      If anyone could do this with Sony's currently-produced hardware, Sony would be breaking the law by shipping that hardware to anyone "International".

    2. Re:Why are Sony so horribly short-sighted by blair1q · · Score: 2

      The processors in PS3 were developed in the USA. I have little doubt that Sony has to adhere to ITAR when making and selling them.

    3. Re:Why are Sony so horribly short-sighted by Anonymous Coward · · Score: 2, Informative

      COTS products are generally not ITAR. An item can fall under ITAR when it is designed to specifically meet a US defense requirement or is built using technology or parts that are ITAR.

  13. Re:Why? by AK+Marc · · Score: 3, Interesting

    While it is true in a sense that Sony spent more on each PS3 than they charged for them at the beginning of the cycle of the product, most of those costs are sunk costs in manufacturing.

    No, they were losing actual money on each one. That is, they spent some amount of money to design and tool for it. Then, for each one they put out, they spent more on the materials and construction than they took in. They weren't making money but not paying off the initial cost. They were actually losing money on each one. They required game sales to make up the difference. And they did.

    Now they have sold so many PS3s that the sunk costs are more than paid for, but it's not sensible to say current PS3s are profitable and older ones were not.

    The hardware profits (if any) have still not reached the level of the hardware development and production costs. They have not now, or ever, made money on putting out PS3 hardware.

    Think of it like this. You buy a $100 grill and $200 in meat to cook and sell hamburgers. You eventually sell 1000 burgers at $2.

    You are wrong. It's like buying a $1000 grill and $1000 in meat to sell 100 hamburgers at $5 each. You spent $2000 to make $500. There is no way to buy another batch of $1000 meat and sell another set of burgers at $5 to make up the difference. However, Sony knows this. They sold it with fries and a drink. The cost for fries and a drink is $1 per order, and the combo is sold at $12. So if everyone who walks up buys only a burger, Sony would have gone out of business (provided they made enough burgers). However, almost everyone gets the combo, so Sony makes about a 10% profit overall on the burger, even though they are selling them at a loss.

    This isn't like car sales, which is how you described (except for the Volt, which loses money for each one sold with no way to ever make it up, so they are written off as R&D expenses or such). It takes over a billion dollars to design and tool for a new car, so the first one is either sold at a billion dollar loss, or all of them get some percentage of that cost attached to them causing the net profit to be called a loss until some volume is achieved. But the Volt and the PS3 were sold for an actual loss. The more volume sold, the greater the loss.

  14. Re:What would..... by mister_playboy · · Score: 3, Informative

    Distributed computing. Have a PS3 app that is installed if selected and when your not playing it runs distributed computing. Give the people whos PS3's are been used something in return like online credits for DLC.

    Anyone got any figures on how much processing could be available if done?

    Folding@Home did exactly what you are proposing. PS3s are a major contributor to the project:

    http://fah-web.stanford.edu/cgi-bin/main.py?qtype=osstats

    --
    Do what thou wilt shall be the whole of the Law ::: Love is the law, love under will
  15. Re:Why? by CronoCloud · · Score: 2

    They did, the Zego: http://en.wikipedia.org/wiki/Zego

    And the PS2 based GSCube before that:

    http://en.wikipedia.org/wiki/GScube

  16. Re:Why? by Ndkchk · · Score: 2

    According to a few different places, Sony started making a profit on PS3s in April this year. Only took them about four years to get there...

  17. Re:So... by stonewallred · · Score: 2

    So, has anyone told the Federal prosecutor in the region about the DMCA violations going on by the USAF?

  18. Re:This is Why Sony Disabled OtherOS by magarity · · Score: 2

    Sony is/was losing money on every sale, banking on making it up in licensing fees from the games purchased for each console sold

    Other people have already mentioned this, proving that none of you read the article and/or thought it through at all. It clearly states the USAF purchased them by working directly with Sony, not sending a few hundred privates out to get a few from the local Best Buy. When the USAF called to place an order for 1,700 units, do you seriously think Sony quoted them the retail price at which they lose a few bucks per unit? Furthermore the folks at Sony planned on the USAF to send legions of privates out to buy an average of 3 games each in order to make a profit on the software side of the deal? It took you 10 times longer to type your post than it should have to think of these minor details.

  19. Re:Why? by Narishma · · Score: 2

    Nobody said they got access to them.

    --
    Mada mada dane.
  20. How's the power consumption? by Kaenneth · · Score: 2

    Is the PS3 an energy efficent solution, or is the power bill higher than it would be for a 'traditional' supercomputer?

    Not just in direct power consumption, but air conditioning costs, etc?

    I used to have a pretty powerful setup at home, 6 multicore PC's stacked up on my desk, but my current hardware is not only more efficent, I don't have to run the AC even when there is snow outside.

    I know government in particular (I used to work in a govt. budget office) has lots of accounting tricks to use. For example, we had to go through a horrible bidding process for software purchases; but I found a nice loophole of buying books that came with the software, since books could be bought directly. (also, the book+software bundle was cheaper than just the software)

  21. Re:Why? by rubycodez · · Score: 3, Informative

    you must be one of those people who think the PC in PowerPC stands for Personal Computer.

    Performance Optimization With Enhanced RISC – Performance Computing

    Variants of the PowerPC architecture are in about half of all automobiles, most video game consoles, all IBM mainframes and Power7 servers and blades, some aircraft control systems......

  22. Re:WTF? by hardwarefreak · · Score: 2

    i cannot believe that IBM or other U.S. vendors instead of Sony would not have been capable of crafting such a system... quite telling, IMO

    They have, almost 5 years ago, actually. You are simply uninformed, haven't been paying attention:

    http://www-03.ibm.com/press/us/en/pressrelease/19198.wss
    http://www-03.ibm.com/systems/bladecenter/hardware/servers/qs22/index.html

    And the IBM Cell blade has a Cell chip, the PowerXCell 8i, with 1 extra SPE, 5 times the double precision floating point performance, and with the Infiniband HCA, over 50 times faster network communications.

    Where have you been these past 5 years? :-)

  23. Re:Why? by DrgnDancer · · Score: 2

    The military doesn't *typically* buy video games systems for soldiers. We briefly considered using some of our MWR (Morale, Welfare, and Recreation) funds in Iraq to buy a couple of game systems and some other stuff to put together a lounge type area for our troops, but it turned out to be nearly impossible to do. The rules surrounding the use of MWR funds make such a thing... difficult. Soldiers buy their own PS3s.

    --
    I don't need a million points of light, just two points of multi-mode fiber and a 10 Gig-E router.