Slashdot Mirror


Cringely Wants A Supercomputer in Every Garage

Nate LaCourse writes: "Real good one from Cringely this month. It's on building his own supercomputer, but with some twists." You'll probably also want to check out the KLAT2 homepage to learn more about their Flat Neighborhood Network. And since KLAT2 has been around for nearly a year (check out the poster on this page!), perhaps a 3rd generation is in the works?

31 of 277 comments (clear)

  1. Great by evacuate_the_bull · · Score: 3, Funny

    Make sure it has a red dot and says things like "Dave, what are you doing Dave?" Can't wait for mine!

    --
    Satanists get good grades too...suspiciously good grades
  2. Now is the time for all good men.... by BlueJay465 · · Score: 5, Interesting

    This is a very interesting concept that he is putting forth, but at the same time, how many geeks out there are going to really make use of such a clustering farm? Not everyone I know does video compression projects, and it would seem kinda prohibitive for a black-hat to set one up to break encryption codes. Could someone please tell this naive soul what useful everyday application all these CPU cycles could be used for? (if you say SETI@Home, I am going to bitch-slap you)

    Secondly, UWB seems to be the holy grail of wireless networking, yes, however is this something that the agencies of the world are going to let out of the bag so easily as he says, I can think of the CIA and the NSA having a few choice words about such "undetectable signals" being used by commonfolk after September 11th...

    Just my two cents

    1. Re:Now is the time for all good men.... by An+Onerous+Coward · · Score: 4, Interesting

      I am now telling the computer exactly what it can do with a lifetime supply of chocolate.

      Okay, we need to burn some spare cycles. Lots of them, in fact. I have some ideas. There may even be a couple in here that can be taken semi-seriously.

      * SETI@H. . . Why are you looking at me like that? Admittedly, it's cliched, and I'm the impatient type who figured I'd find my first LGM within a week. Or by the end of the year at the very latest. But I still think that it's a pretty cool thing to be doing. Or load up one of the alternatives like Folding@Home.

      * Find a buddy with a similar supercomputer, and have them play chess. Or tic-tac-toe billions of times every second (sorry, War Games flashback).

      * There are lots of mathematical problems out there just begging to have a few supercomputers thrown at them. I'm not aware of what they are, so consult your local Mathematics department and offer your services.

      * If you're not interested in doing video compression or complex scene rendering, you might be able to find someone who was. Some indie film maker who wants to play with the big kids is going to become your new best friend. Be sure to ask for a walk-on.

      * Some sort of AI project could be interesting, providing you have some specialized training. Or you just give someone at MIT telnet access.

      --

      You want the truthiness? You can't handle the truthiness!

  3. HP Did This Too by MathJMendl · · Score: 5, Interesting

    ZDNet has an article of HP building a supercomputer like this as well, called the "I-Cluster." It has 225 networked computers running Linux Mandrake (so changes could be easily made) on 733 MHZ out of the box PCs. The only catch is that is is slightly more expensive- $210,000 (minus network cabling). On the other hand, they plan to release the open source tools they made as well, so that people can repeat this.

    --


    "I have not failed. I've simply found 10,000 ways that won't work." --Thomas Edison
  4. Re:genetic algorithms by An+Onerous+Coward · · Score: 5, Funny
    "Ironically, they needed a supercomputer to design a supercomputer."


    And it shall be called. . . Earth!
    --

    You want the truthiness? You can't handle the truthiness!

  5. easy cowboy by ProfKyne · · Score: 3, Funny

    Those are some interesting ideas.

    Now how about organizing them before publishing them? Call me pre-postmodern (and I'm still in my twenties), but I tend to learn more from a coherently-organized message than from a random jumble of statistics and facts. Cringely jumps from a detailed description of the KLAT2 and its innovative networking technology to a brief description of UWB. And then it's over.

    Maybe I'm missing something.

    --
    "First you gotta do the truffle shuffle."
  6. WELLL.... by Midnight+Ryder · · Score: 3, Funny

    Well, not to be one of those stick in the mud 'Read the %($#ING article' type people, KLAT2 is a reference to The Day The Earth Stood Still. Had you looked at the articles in question (particularly, the KLAT2 page) you would have discovered that indeed, they were intending the reference. Heck, go check it out - the poster they made up for it is worth the look! :-)

    --

    Davis Ray Sickmon, Jr - looking for something to read? Check out my three free novels at MidnightRyder.org

  7. /. needs a Cringley icon, any suggestions? by John+Harrison · · Score: 4, Funny
    I think a fake Stanford degree would do nicely.

    Maybe they could set things up so that ALL his articles hit the main page as soon as he posts them.

    If this were the case he could put a "discuss this article" link on his page and simply link to /.

  8. Re:What is 10base-100? by SuzanneA · · Score: 3, Informative
    As I understood it, 10base-100 was the original name for what most people call 100base-TX these days. Some people still seem to refer to it as 10base-100.

    Btw, a little nitpick, the TX would refer to 4 pair (all 8 conductors), 10base-T uses 2 pair, 10base-TX and 100base-TX use all 4 pairs.

  9. Re:Am I the only one? by dangermouse · · Score: 3, Informative
    "Klaatu barata nikto."

    I think it translates to "Klaatu says not", but I'm basing that off of a half-assed knowledge of German ("nikto" -> "nicht"), the similarity between "barata" and "berate", and context. Maybe there's some kinda Latin thing in there somewhere, idunno. It's the instruction Klaatu tells Love Interest to give to Gort the robot, so he won't destroy the entire planet.

    And since someone asked, yeah, that's the same line whatsisface has to use in "Army of Darkness", and it was the only part of that lame movie I can remember laughing at.

    Now, the question is: Will I get modded down because this is actually offtopic as hell, or because I insulted "Army of Darkness"? %-)

  10. Old news by SumDeusExMachina · · Score: 4, Offtopic
    Sorry to say it guys, but this is a repeat of an old Slashdot post that linked to an ArsTechnica article more than a year old.

    Still though, after having to wallow through Cringley's painful lack of comprehension of basic technical knowledge, reading the ArsTechnica piece again was quite refreshing.

    --

    Is your company running tools written by ma
  11. the ignorant are easily amused by markj02 · · Score: 5, Insightful

    Cringely is completely missing the point. KLAT2 uses multiple routes and switches, not channel bonding. And what the project contributes is not the basic idea of using multiple network interfaces (which is decades old), but a specific approach: using genetic algorithms to optimize the network topology. More traditionally, such clusters have used manually designed topologies with known performance bounds.

    1. Re:the ignorant are easily amused by funnyguy · · Score: 4, Insightful

      the FNN which was created for KLAT2, is not a speed increase of ethernet by using multiple network cards. It basically allows full speed (100mb full-duplex) without a 64+ port, full wire speed switch. If such a thing even existed. Cringley's network is just 4 channel bonded network layers. Channel bonding actually has slightly more overhead than FNN. With KLAT2's FNN, each machine is on 4 seperate networks. No matter what other machine a single machine needs to communicate with, they each share one common network. Each network is held together with one switch, so there is always a full speed route to every other computer in the cluster. The OS handles this directly by using /etc/ethers to hard code the hardware addresses of every computer. different networks are different subnets, and the network routes are layed out accordingly.... blah blah... I could go on and on, but aggregate.org has more info.

      As for the algorithm everyone is talking about. there are some versions which can return a pattern in a second or two on a slow celeron. then there are some version which are designed optimized for certain datasets which take time to run. but generally, you don't need a supercomputer to design a fnn. even with 64+ nodes.

    2. Re:the ignorant are easily amused by Zeinfeld · · Score: 4, Insightful
      Quite, the problem with measuring super-computer performance is that every single machine in the class is highly optimised to a particular niche. That is the main rason they are so expensive compared to the components - large machines sell in the tens rather than the tens of thousands.

      Anyone can build a machine with a really high processing performance. Just by a few thousand X boxes and plug them into the same ethernet cable. The real issue is how much communications bandwidth you have between the CPUs. Some problems require almost none - the 'trivial parallelism' problems like DEScrack and the mandelbrot set. In the 1980s we had a machine that had 1000 20MHz processors that could bang out mandelbrot sets like anything (using the goofy algorithm, not the modern optimizations). But is wasn't much use for anything else.

      The problem with competitions for supercomputers is that they rarely measure the communication bandwidth because (a) its hard to do and (b) the effect on performance is highly algorithm dependent.

      As for the KLAT's ingenious topology, I once did some research in the area myself when it was the fashion. I tried using minimum diameter graphs which should in theory have been better than a plain taurus. However as with Bill Dally at Cal Tech I concluded that the additional cost of exotic topology (more than double the price) was not really justified by the performance advantage (about 10-30% on a good day).

      Certainly the many companies that set up to build transputer based processing clusters with high performance switches inside did not seem to go anywhere much.

      Using a high performance router at the core of a processing cluster might be interesting. They are pretty cheap these days and are headed cheaper.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
  12. Supercomputing? Why bother. by Bowie+J.+Poag · · Score: 5, Insightful



    Speaking as someone who, yes, has actually worked with the big iron...

    Why bother. Remember, Moore's Law is still in effect. Recently, we've hit the point in the curve where supercomputers are no longer needed, nor cost-effective. That is, the time it takes for the industry to deliver a far superior product has eclipsed the average lifespan of your typical supercomputer.

    We're living in an age where a single graphing calculator you can buy at Walgreens has more horsepower under the hood than what got us to the moon 30 years ago. Your $2700 PC will be worth $150 within 3 years.

    Having a supercomputer in every garage makes about as much sense as taking a rocket fuel-powered dragster to the supermarket for a gallon of milk.

    Cheers,

    --
    Bowie J. Poag

    1. Re:Supercomputing? Why bother. by ASCIIMan · · Score: 3, Interesting

      Sadly, you are partially correct.

    2. Re:Supercomputing? Why bother. by Rasta+Prefect · · Score: 4, Interesting

      I don't know about every garage, but as someone who is currently working on a research project at a University, I can say we'd find something like this very interesting, as would a number of other departments on the campus. We've got a couple of Crays sitting around, but can't afford the cost of maintaining the things. Something like this would be way more affordable to buy and maintain for educational/research purposes where traditional supercomputers aren't even vaugely an option.

      --
      Why?
    3. Re:Supercomputing? Why bother. by Rocketboy · · Score: 3, Insightful
      Why bother. Remember, Moore's Law is still in effect

      What's Moore's Law got to do with this? This is more the area of Murphy's Law, I think. As for why bother, heck, I don't know: because I can. When I had a 286 PC, it did everything I wanted it to do at the time, why did I need a 386? My 386 was dandy, what was the benefit of having a 486? My trusty 486 was quite fast at the time: was the premium price of a Pentium worth it?

      Stuff happened! People thought up new applications for newer and faster machines, and then we couldn't do without them. Remember when your average machine could push out 5 frames per second of 160x120 video, tops? I remember when encrypting a 26k text file took almost a minute, each. Back in the day I didn't think I'd be watching DVD videos on my desktop or laptop PC: who'd want to, that's what TVs were for!

      Years and years ago I had a program that simulated stellar interaction in small globular clusters. A few hundred stars pushed a 086 as far as it would go and it was still an overnight crunch to simulate much interaction. I kinda gave up on it after a while: other interests, etc. I think about it occasionally, wondering when that sort of stuff will get commoditized to the point where I can take a look at it again without having to pull away from current projects for six months. Not quite there yet, I think, but gettin' close, gettin' mighty close... :)

    4. Re:Supercomputing? Why bother. by Zeinfeld · · Score: 5, Interesting
      Speaking as someone who, yes, has actually worked with the big iron...

      The machine I worked on in the early 90s is still in the top 100 of the supercomputer charts (or would be if the compilers knew about it).

      While a desktop Cray-1 can now be had at commodity prices the machine is now two decades old. The obsolescence rate is nowhere near as giddy as some would claim.

      The really big iron tends to have a lifespan of about five years and is typically retired because the power consumption and maintenance costs favor a move to newer hardware. True supercomputers rarely fall victim to Moore's law. Even the KLAC machine discussed only barely qualifies as a supercomputer, 64 processors is at the low end of the scale. People have Web servers with that number of CPUs. True big iron starts with a few hundred processors and goes up to the tens of thousand.

      If by working on the big iron you merely mean you used to use IBM 3090 class machines, then the joke is on you, those machines were often obsolete before they were manufactured. When I worked at one lab I had a desktop machine (first production run Alpha) that was considerably more powerful than the CPUs of the just-installed campus mainframe.

      Fact is that many of the people buying 'big iron' in the 1980s and 1990s were incompetent. They bought machines that ran the O/S they knew, which often meant they bought obsolete IBM mainframes for applications where a ntwork of IBM PCs would have served far better. I spent quite a bit of time in institutions where wrestling control of the computing budget from an incompetent IT dept was a major issue. In fact the World Wide Web began at CERN in part as a result of such a struggle. Tim, bless him wanted the physicists to switch from the IBM mainframe CERN VM to use NeXTStep machines. One of the schemes that the CERN CN division had cooked up to force people to use the mainframe was to only make information such as the address book available on the IBM mainframe. Attempts to make it more widely available were treated much the same way that Napster was treated by the RIAA. The Web took off at CERN initially because you could access the address book from a workstation or from the VAX.

      Very few mainframes were actually designed to provide fast processing. The IBM 3090 series was actually designed to perform transaction processing for banks. As a scientific CPU it offered tepid performance at a price arround 100 to 500 times the price of a high power workstation.

      There are certain applications in which CPU cycles are still the limiting factor. Admittedly they are much smaller as a proportion of the whole than they were 10 years ago.

      --
      Looking for an Information Security student project suggestion?
      Try http://dotcrimeManifesto.com/
  13. nah... by elmegil · · Score: 3, Offtopic

    I'd rather have a superMODEL in every garage.

    --
    7 November 2006: The day Americans realized corruption and incompetence weren't addressing 11 September 2001
  14. This is silly by Binary+Tree · · Score: 3, Funny
    Technically we almost all have a "supercomputer", depending on what era's standards you're referring to.

    Also, if everybody had a supercomputer in their garage, they would no longer be so "super."

  15. My Hank Dietz (creator of KLAT2) story by IvyMike · · Score: 4, Interesting

    Dr. Dietz used to teach at Purdue, and I had the good fortune to take a compiler course taught by him. On the first day, when introducing himself, he came to the part where he was describing how to get into contact with him. When giving out his phone number (at Purdue, on-campus numbers were 5 digits long) he mentioed that his phone number was "GEEKS". He added, "No, I didn't ask for GEEKS, but when I figured it out, I thought it was pretty cool."

    Needless to say, it was a pretty cool course.

  16. I've got one ..... by taniwha · · Score: 3, Interesting

    25 dual p650s in my home office ... when I crank it all the way up I come in at somewhere in the 50-100 range on the dnet rc64 dailies. Sadly the original reason I built it has evaporated and with the current cost of CA power I just have a fraction running

  17. A real supercomputer? Not exactly by fgodfrey · · Score: 5, Insightful
    The article would have people believe that all a supercomputer is is a collection of not-quite-modern processors, memory, and an interconnect of some sort. This is simply not the case. If it were, why do many (granted a smaller number than before) people still buy real big iron? The answer is that Cringely's (sp?) collection of processors is not a real supercomputer for the kinds of applications that are associated with traditional machines. Traditional vector supercomputers still have processors that are faster than Pentium 4 class systems. Traditional massively parallel supercomputers (which are the most similar to a cluster) have a number of features not found in your average garage built cluster like a truely low-latency interconnect, gang scheduling of entire jobs, single system image for users/administrators/processes.

    Clusters are great for embarassingly parallel applications (ie ones that have threads which don't communicate with each other much. This includes things like SETI@home and batch rendering of images. What they don't compare on is applications that communicate a lot like nuclear physics simulations. This is not to say that that will never change in the future, but for the time being it's still true.


    Last, and certainly not least, real supercomputers have memory bandwidth that can match the speed of the processor. A Cray or an SGI Origin has an absolutely massive amount of bandwith from the processor to local memory compared to a PC. That allwos a traditional supercomputer to actually *achieve* the fantastic peak performance numbers. On many applications, the working sets are huge and don't fit in cache so you end up relying on memory being fast. On a PC, it's not and I've heard from sources I consider reliable (though I have no actual numbers to back this up so it may be rumor only) that one large cluster site sees around 10% or less of peak on a cluster for a nuclear physics simulation, whereas, on a vector Cray, you can hit ~80% of peak. This means that the cluster has to be 8 times more powerful and when you start multiplying the costs by 8, they start looking like the same price as a real supercomputer.


    So my point is that building a real supercomputer does not mean grabbing a bunch of off-the-shelf components, slapping them together with a decent network and running Beowulf (or a similar product).

    --
    Go Badgers! -- #include "std/disclaimer.h"
  18. Imagine... by Tsar · · Score: 5, Funny

    ...a single-CPU version of this!

    1. Re:Imagine... by nuintari · · Score: 5, Funny

      and a cluster of those!..... oh wait, nuts....... never mind.

      --

      --Nuintari

      slashdot : where an opinion can be wrong.

  19. Two Reasons by nuintari · · Score: 3, Interesting

    One: Because we can.

    Two: Ever seen the stuff they run on supercomputers today? Simulating a supernova for 1 nano second can take a month of CPU time on some of the world's fastest supercomputers. Oh, its still very nessesary. If the past is any indication of the future, we will always need blazing fast machines to push the limits in the scientific world.

    I assume you mean big iron as in mainframe, which is NOT a supercomputer by any means. Mainframes do the work that runs this world, supercomputers help us discover what we'll do in tommorow's world. They are very different worlds.

    --

    --Nuintari

    slashdot : where an opinion can be wrong.

  20. QNX? Hey Cringely... by Chazmati · · Score: 3, Funny

    "(the operating system) will be QNX, a real time OS that supports massive parallelism and has very low overhead. QNX is fast! QNX is also Posix compliant, so there is lots of software that almost works under it."

    If you're looking for software that almost works, I know of an OS that might fit your needs. You're not going to hook this thing up to the Internet, though, are you?

  21. only fastest one percent is supercomputer by peter303 · · Score: 3, Informative

    By definition only the fastest devices are supercomputers. These days that is about a teraflop. Thta includes the US DOE ASCI series and the announced installation of the Blue Storm and Blue Gene IBM computers. Ten gigaflop computers a dime a dozen and a hundred gigaflops not so rare.

  22. A real supercomputer? Yes, exactly by Multics · · Score: 4, Interesting
    Your comments are true for a 486. They are not true for anything much newer. An IBM SP machine, which owns half of the top 10 on the top500 list, is basically a commodity parts built system.

    Yes, these systems are not sometimes the best for handling vectorizable jobs, but they are so inexpensive compared to the old specialized hardware that it is easier to waste cycles than build special hardware.

    As to memory bandwidth. Modern CPU caches make the question nearly moot.

    If all of this were not true, then people wouldn't be building clusters and the majority of the top500 list wouldn't be dominated by clusters. Instead there are 3 traditional architecture machines in the top 20. This is the reason that Cray (etal) no longer dominates the marketplace... commodity systems have overtaken nearly all of the specialized hardware world.

    -- Multics

  23. Even more interesting ideas. by 2nd+Post! · · Score: 3, Interesting

    I have a TiBook. Apple *could* do away with the screen, keyboard, and speakers, and replace the CD-ROM slot with a ram-bay.

    Not only could you hook them together using gigabit ethernet, you could take advantage of the firewire port as well, perhaps chaining them together with some sort of SAN, though you are still limited by the ~50MBps, though perhaps that's not useless, I don't know.

    Still, with the ram bay you could up the memory from 1GB to something crazy, like 16GB. The battery is useful as a backup-emergency device, allowing the slab to run for about 4 hours in case of emergency (woo!).

    You could even concievably netboot the thing, since OS X allows for that, right? Minimize the hard drive or get rid of it altogether... you could seriously make a slab about the size of 1/2" by 8" by 8" I suspect :)