Slashdot Mirror


First 96-Node Desktop Cluster Ships

Panaphonix writes "The Register reports that Orion Multisystems is shipping the first 96-node desktop cluster. 'With the new, larger system, customers get pretty much the most powerful computer around that can plug into a standard electrical socket.' According to the spec sheet, the DS-96 runs Fedora Core 2 and gets 110 GFlops sustained, 230 GFlops peak."

57 of 323 comments (clear)

  1. Imagine a Beowulf... by Anonymous Coward · · Score: 4, Funny
    Oh, never mind.

    I FAIL IT!

  2. mmm clusters by Worminater · · Score: 2, Funny

    My uni recently got some a 12 system dell cluster that came loaded with redhat. mmm; paralizing is fun:)

  3. dup by pg133 · · Score: 2, Informative
  4. Question by ErichTheWebGuy · · Score: 3, Interesting

    I am not familiar with the architecture of clusters, so I am a little surprised by the more than 100% difference between sustained and peak GFlops. I know what a GFlop is and all that, I just don't immediately see why there is such a huge difference.

    Can someone summarize why there is such a huge difference?

    --
    bash: rtfm: command not found
    1. Re:Question by bobbozzo · · Score: 4, Informative

      finite bandwidth between processors makes it impossible to sustain anywhere near peak performance for most real-world applications.

      Linpack is what is usually used to measure sustained performance on HPC systems.

      --
      Nothing to see here; Move along.
    2. Re:Question by katana · · Score: 4, Funny

      Sustained: Riding down the street.
      Peak: Taking it off some sweet jumps.

    3. Re:Question by Ruie · · Score: 2, Interesting
      You are right - on pure number crunching, with little I/O the cluster peak and sustained performance for the same program should be identical.

      It might be that their peak number is derived assuming code particularly favorable to the processor architecture in use - say using SSE to do the floating point math. This can easily produce the factor of 2 difference.

    4. Re:Question by brsmith4 · · Score: 5, Informative

      The theoretical max gives a rough estimate of the raw Floating-point power for all of the processors on the system. You pretty much add up the GFlops potental for each node (not exactly, but pretty much). The sustained and demonstrated GFlops of the cluster is based on the Linpack benchmark. The reason there is such a huge difference between the two numbers can be a result of a few factors. 1) The interconnect is GigE and Linpack tends to make use of Message passing comms which are affected adversely by the latencies produced by the GigE connections (myrinet would have been a good choice, but I suppose it was probably impossible to squeeze that into that case) 2) Memory speeds also are a factor as pushing floating point numbers around involves memory. This cluster isn't using anything fancy when it comes to the memory and I suspect this may be another cause for this.

      When they say that this line of clusters can "make or break" Orion, I am right now, leaning for broke. For the cost of this machine, one can get a real cluster with a lot more performance. I know this thing is nice because of the power requirements and the fact that you don't need a dedicated server room to store it, but for $100,000, you can get Microway to build you a pimptacular cluster with Dual-Opteron nodes, high-speed memory and a phat interconnect with either myrinet or infiniband. You will get a lot more work done for the same price.

    5. Re:Question by Daniel+Phillips · · Score: 4, Insightful

      I know this thing is nice because of the power requirements and the fact that you don't need a dedicated server room to store it, but for $100,000, you can get Microway to build you a pimptacular cluster with Dual-Opteron nodes, high-speed memory and a phat interconnect with either myrinet or infiniband. You will get a lot more work done for the same price.

      You forgot a couple of things:

      * HVAC costs
      * Realestate costs

      Remember, this is a deskside cluster. Try that with your dual-opteron cluster. And try adding up all the costs.

      --
      Have you got your LWN subscription yet?
    6. Re:Question by glesga_kiss · · Score: 4, Insightful

      He's also forgotten about hiring an extra 2 guys at least to maintain the extra machines. A selling point of this box will be that "it just works". Pay for a support contract and wammo, you've got a cheap low maintainence cluster. For people working on top-secret stuff (who else needs clusters? ;-), hiring people is a risk and the vetting process is expensive.

    7. Re:Question by Tim+C · · Score: 2, Insightful

      The vetting process isn't just expensive, it's time consuming too. I don't know about the States, but here in the UK getting security cleared (to "normal" level, I forget the correct term) takes about 3 months. That's not 3 months of filling in forms, of course, but it is three months during which the person can't work on the project while you wait.

    8. Re:Question by Anonymous Coward · · Score: 2, Insightful

      Space (e.g.real estate) and energy(electricity) is a premium in places such as Japan. Could be interesting to see how sales develop in such places.

    9. Re:Question by brsmith4 · · Score: 2, Insightful

      You don't need two extra guys. I manage 5 clusters myself on a regular basis. Granted, it would be a hell of a lot of work if I couldn't enlist the help of a few people every once in a while, but for the most part, its a one man job. As well, to the poster claiming that I didn't take into account the cost of storage, I know this thing is nice because of the power requirements and the fact that you don't need a dedicated server room to store it, I'm fairly sure that I roughly addressed that. BTW, we need clusters and none of our researchers has top secret clearance ;-P

    10. Re:Question by femtoguy · · Score: 3, Interesting

      This point is a biggy in the scientific computing world. It is easier to get capitol equipment money than it is to get salary money. (This is because equipment money is overhead free, while salary money incurrs a 50% overhead rate). We just had a donor give us many millions to guy a cluster, but he would commit to long term money for sys-admin support. We ended up including a lot of vendor support into the bid for the contract in order to turn support money into capitol equipment money. Considering that a sys-admin can easily cost $100K per year, this isn't such a bad deal.

  5. Wrong term by AndrewStephens · · Score: 5, Funny

    Its not duped, this article has been clustered.

    --
    sheep.horse - does not contain information on sheep or horses.
  6. Re:Just imagine... by Anonymous Coward · · Score: 2, Funny

    At 150lbs or 68KG you need a Beowulf cluster of people to move it!

  7. Still The Wrong term by Johan+Veenstra · · Score: 2, Funny

    It's called a follow-up:

    "In October, you'll be able to choose between"

    "is shipping the first 96-node desktop cluster. '"

  8. Re:Give it to me in terms I can understand. by spectrokid · · Score: 5, Funny

    You want to be impressed heh? Well, it is powerfull enough to play solitaire on Longhorn!!!!

    --

    10 ?"Hello World" life was simple then

  9. Re:Give it to me in terms I can understand. by Anonymous Coward · · Score: 2, Funny

    i doubt it, you microsoft astroturfer.

  10. Re:Fedora Core 2? by lachlan76 · · Score: 2, Informative

    Because they started development before FC3 came out maybe?

  11. Practical measures... by crottsma · · Score: 3, Funny

    With power requirements quintupling that of a standard desktop computer, I'd probably have to use it at my local coffee shop, or only turn it on briefly to scare away song birds.

  12. More Importantly... by mtrisk · · Score: 3, Informative

    Will this make or break Transmeta? It uses their processors (Transmeta Tinside as the Register calls it). Slashdot already pronounced the death of Transmeta though (it has no more niches!), maybe this could revive interest?

    --

    Without a proper flamewar, Anonymous was undecided on what shell to run.
  13. Re:Give it to me in terms I can understand. by William+Robinson · · Score: 2, Informative

    You might want go through this

  14. Re:$100,000 by raehl · · Score: 4, Insightful

    $100,000/96 = ~$1,000.

    Not a bad deal.

  15. Re:Standard comments by jim_v2000 · · Score: 2, Funny

    You forgot to mention:

    12. List of common Slashdot posts

    --
    Don't take life so seriously. No one makes it out alive.
  16. Interesting practical uses for Personal Cluster .. by torpor · · Score: 2, Insightful

    .. Computing?

    I mean, does Blender run on it at least? Can I do anything interesting from an 'immediate-personal' perspective with 96 nodes, and I don't just mean run Quake, or fire up "make -j 96" and such things..

    What sort of interesting modelling software is around? Could I use it to design stuff on a personal, non-hard-core science perspective? What are the practical uses for personal cluster computing?

    --
    ; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
  17. Needs silencing! by Slowleggs · · Score: 2, Interesting

    From TA "Sound power 55 bels"

    550 dBel noise? Perhaps the producers should look into Metal cooling ? :)

    ...and/or put the box in another room.

    1. Re:Needs silencing! by hcdejong · · Score: 2, Informative

      The loudest sounds possible (at 1 bar ambient air pressure) are about 180 dB. At that level, the pressure minimum of the sound wave is 0 bar (ie vacuum).

    2. Re:Needs silencing! by gnuman99 · · Score: 2, Informative
      From TFM:
      Sound pressure level 50dBA at operator position
      Sound power 55 bels

      There is a difference.

  18. FPS by strider44 · · Score: 2, Funny

    How many FPS can you get on Doom 3? I've got to plan my future purchasing decisions.

  19. Wanna know what it looks like? by io333 · · Score: 2, Informative
  20. What?! by Anonymous Coward · · Score: 5, Funny

    I can't hear you, the guy two cubicles over just fired up his new Opteron cluster. I'm just trying to hold on to my desk!

  21. Re:Inefficient ? by Slashcrap · · Score: 2, Insightful

    General CPU's just don't have the punch that special purpose or Fpga processors do.

    And FPGAs or special purpose CPUs don't have the generality that normal CPUs have. There's also the small point about the Merrimac system not actually exisitng.

    PS. Thanks for linking to Roland Piquepaille's fucking blog. He doesn't get nearly enough links on Slashdot in my opinion.

  22. Suspicious by cyberfunk2 · · Score: 2, Insightful

    I gotta say.. I'm a tad suspicious here.. there seems to be a lot of marketing flash (no pun intended) and scarce details.

    What kind of CPUs are we talking about ? I'm assuming we're talking non-shared memory here, and therefore nodes that "retain" their own identies. But then isnt each cpu running it's own kernal ? That is.. This ISNT SMP , right ?

    I think the details could be a lot clearer here. The lack of tech specs or simple explinations, and excessive use of buisness speak "Efficiency" "unprecendented power" etc. makes me a tad nervous.

  23. Wha...? by raehl · · Score: 2, Insightful

    General purpose processors have *WAY* more punch. Especially punch per dollar, as FPGAs are fairly expensive.

    They're just general purpose, whether they be scalar (CPU) or vector (GPU), so an FPGA that is specifically optimized for a specific problem will kick the general purpose processor's butt - in that specific problem.

    But try running Quake III on an FPGA - it will be killed by the CPU in processing and killed by the GPU in graphics. Assuming you can even cram everything you need to be a CPU or GPU into the limited real estate of the FPGA in the first place.

  24. Re:Suspicious [Transmeta] by Anonymous Coward · · Score: 2, Interesting

    All the current Orion systems, including this one, use Transmeta Efficeon CPUs. Not surprising since Orion was founded by a Transmeta co-founder.

    Actually, Efficeon performance is quite good on the type of repetitive loop-based code this system is intended for. It may not surpass an equivalent Athlon 64 or P4 based system, but in terms of bang per watt, it's not bad.

  25. Re:Seti by Vo0k · · Score: 2, Funny

    No worries. He can set up WINE on it and run the Windows version of SETI@home with the screensaver, emulated :)

    --
    Anagram("United States of America") == "Dine out, taste a Mac, fries"
  26. Only on /. (Re:$100,000) by po8 · · Score: 5, Funny

    You can tell you're on /. when dividing 1e5 by 1e2 to get 1e3 gets modded up to +5 insightful. :-)

  27. Too expensive by Buzh · · Score: 4, Interesting

    Although having 96 nodes in a single box makes it quite cute, from what I can interpret from the specs, you would get more bang for your $100K by getting what the beowulf crowd like to call MMCOTS (Mass-Market-Common-Off-The-Shelf, i.e. mass produced computers from Dell or the like), hooked toghether with a specialty high-bandwidth low-latency interconnect like Infiniband, Myrinet or SCI. Running a free beowulf cluster OS like for instance ROCKS would mean that a normal linux admin could maintain it quite effectively.

    I expect this thing to be marketed towards scientists in small or medium businesses that aren't employing many/any IT staff, who use commercial computer models to do things like theoretical chemistry (Gaussian, ADF etc), bioinformatics (Phase, BLAS, Paralign etc), fluid dynamics, statistics, crypto, you name it. I don't expect to see any of these types of systems used in normal supercomputing sites, where people write their own (parallel) code and skilled staff maintain the cluster.

    --
    -- Buzh
  28. Re:$100,000 by Donny+Smith · · Score: 2, Insightful

    >When building cluster of stock PCs it should not be more then $500/PC.

    What are you talking about?

    The now (in)famous Apple cluster cost them about 5 million for 1,100 nodes or $5K/node.
    http://radio.weblogs.com/0112083/G5cluster.html
    And that was supposed to be a good deal.

  29. Re:Apple? by EvilAndrew · · Score: 4, Informative

    Parent poster messed up on their calcs. Current XServes are 18.4 GFlops peak, not 35 eg Virginia Tech currently at #7/500 is 20240 GFlops peak for 1100 XServes. So 7 would only be ~129 GFlops peak, and 33 would be 607 GFlops peak. But not exactly fitting in a single tower case - though 8 would fit nicely one of those mini sound-padded racks which would be almost as good. And at least the last time I saw a price comparison made, the G5s were far cheaper than comparable rack P4s. (The G5 has 2x the FP hardware).

  30. Altitude by hey · · Score: 3, Interesting
    It has Altitude restrictions:

    Altitude -300 meters to +3000 meters
    -1000 feet to +10,000 feet

    I've never seen that before.
    So you cann't use it on a plane.
    1. Re:Altitude by omega9 · · Score: 2, Informative

      From the site itself:

      They're fully scalable so you can add performance as your needs expand. It can be used on site: in the office, the laboratory, on a boat, or even aloft in a plane.

      Ain't that sumpin.

      --
      I'm against picketing, but I don't know how to show it.
    2. Re:Altitude by Analogy+Man · · Score: 2, Informative
      Dumping heat at higher altitudes will be an issue (thin air).

      An airplane cabin is pressurized so you could as they say us this machine on a flight test or atmospheric research machine. Most data acquisition experiments would likely gather data with more modest equipment and do the analysis in a tera-ferma environment.

      --
      When the people fear their government, there is tyranny; when the government fears the people, there is liberty.
    3. Re:Altitude by Locke2005 · · Score: 2, Funny

      So you cann't use it on a plane.
      Heck, it probably wouldn't fit on the seat-back "table" anyway...

      --
      I've abandoned my search for truth; now I'm just looking for some useful delusions.
    4. Re:Altitude by flaming-opus · · Score: 2, Informative

      Hard drive heads are tiny air-foils. They depend on a certain barometric pressure to keep from crashing into the platter. I had a device that measured weather characteristics in the mountains of antarctica. We had to use magneto-optical drives, because the heads on a disk drive kept crashing. This was many years ago though. I'd think it has improved since then. Maybe not.

  31. And? by daVinci1980 · · Score: 3, Interesting

    Who cares? Modern graphics cards are capable of (sorry it's a PDF, it was all I could find) 40 GFLOPS. That's not even in SLI mode, which actually does push you to about a 98% over a single card (in terms of raw processing power).

    Why would you buy a 96-CPU setup when you could buy a 6-GPU setup and match the same theoretical performance? (All jokes aside about the costs being roughly equivalent, they're nowhere near the same.) 6 top of the line 6800s would run you about $3600. Even if you added top of the line parts for the rest of the system, you'd be looking at about $1600 per system. Add $0 for the linux distribution to drive the whole thing, and you're at a grand total of $10K.

    I'm not impressed.

    --
    I currently have no clever signature witicism to add here.
    1. Re:And? by sjwaste · · Score: 4, Interesting

      Serious question here. Does production software exist to drive arbitrary computation across a GPU? I've seen articles about software on its way, etc. Does it exist, either as an application or integrated into some OS? Man, if I could push some of my statistical computing off to the GPU...

    2. Re:And? by ravenwing_np · · Score: 2, Informative

      Look for GPGPU. They are trying to use the graphics processor for general purpose operations. It runs as any CG script would run. Just realize that it is focused more on parallel math operations then procedural. Please note that I have nothing to do with this project and haven't tried it yet.

    3. Re:And? by graphicsguy · · Score: 2, Informative

      you could try Brook

    4. Re:And? by timeOday · · Score: 2, Informative
      Serious question here. Does production software exist to drive arbitrary computation across a GPU?
      No, because graphics hardware cannot do arbitrary computation. At least not at anything like the FLOPS it achieves doing graphics.

      I've attended a workshop on using graphics hardware for accelerating other computation and it's mostly hype IMHO. It amounts to rendering images of your problem, then doing feature extraction on the image. So the *effective* FLOPs, i.e. the amount dedicated to *your* task rather than the overhead of reducing from a rendering task, aren't all that impressive.

      A more serious challenger to this transmeta-based system, IMHO, is the cell processor.

  32. The Islands of Patmos Super Computing by Danzigism · · Score: 3, Interesting

    Yea, I agree that Fedora was definitely an odd choice.. Well, I can trust that the kind of person whom can build a 96 node super computer, makes very educated decisions.. I'm glad to hear that company's are still involved in making these clusters. Its a great way to build something powerful for a cheap price, and not having to lean towards Crays etc.. I worked for a company called Patmos International for the longest time, and we never shipped a single cluster.. We had tons of investors that seemed interested of course, but after 2 years of contiuous development, and no sales, the investors simply stopped investing, therefore my job was done for.. We advertised the "$99,000" super computer that would supposedly be in "everyones" garage one day.. Of course that was just a saying because of how cheap we could offer a 32 node system with all of our custom applications and linux operating system. Pretty sweet setup.. it sucks to see the big guys go down sometimes.. to this day, it was the best job I've ever had.. You can still read about Patmos if you search for James Gatzka on google.. They tried their hardest to bring some damn technology and culture to the podunk town of the Eastern Shore of Maryland..

    --
    *plays the Apogee theme song music*
  33. Multichannel EEG processing in Canada vs Chile by mindpixel · · Score: 5, Interesting

    One of my best friends just bought a tiny little house in downtown Toronto for $377,000. I left Toronto last November and moved to Santiago, Chile and live downtwon where my rent is $260/month, for quite a nice, though small place, in an excellent area.

    So, if I spend $100K on the Orion DS-96, that leaves me more than enough for a 250 channel geodesic EEG system which would allow me to compute self-organizing maps of the human mind based on flashing the 1.6 million mindpixels I have collected over the past five years to various volunteers [english teachers], AND still have 56.73 years worth of rent left!

    Too bad no bank will loan me $377,000 for a computer and an EEG system and the time to play with it...

  34. Re:Just imagine... by tomhudson · · Score: 2, Interesting
    ...a Beowulf Cluster of these...
    Looks like they could use a cluster of clusters after all:
    Warning: mysql_pconnect(): Too many connections in /home/www/php/functions/executequery.php on line 21
  35. Slashdotted! by 3Suns · · Score: 3, Funny

    Warning: mysql_pconnect(): Too many connections in /home/www/php/functions/executequery.php on line 21

    How blissfully ironic!

    --

    -3Suns

    ~~~~
    The Revolution will be Slashdotted
  36. Re:$100,000 by Grayputer · · Score: 3, Insightful

    How about "plugs into a single 110 volt US outlet". This thing draws 1500 watts PEAK. That's about 14 amps for the math challenged or well under the recommended max for a 20-amp circuit from your household panel (think coffee maker on steroids). Let's try your barebones system approach and let's say you tweak it to use 50 watts per system (good luck). That's 50 watts per system multiplied by roughly 100 systems => 5Kwatts or 3+ times the power consumption. PLUS as an added bonus you get 96 cases, external cabling, at least 96 fans, a dozen power strips, and assorted other toys to trip over. Oh yeah and a few weeks of setup, integration, burn in, and testing.

  37. Re:Inefficient ? by timeOday · · Score: 2, Insightful

    That's not even a product, it's just a schematic. Talking about building a computer than only has $20K of parts, and running an actual business by selling those computers for $20K each are two very different things.