Slashdot Mirror

You can't cheat Amdahl's law anymore than you can give birth in one month with nine women. The law is a rather simple idea similar to chemical kinetics, when you think about it. i.e. a rate limiting steps.

If you are interested in a non-mathematical description of Amdahl's law have a look at http://www.clustermonkey.net/Parallel-Programming/parallel-computing-101-the-lawnmower-law.html

For those who would like a gentle introduction to quantum computing take a look at: A Smidgen of Quantum Computing

For a portable case use a plastic footlocker, the kind with wheels and a hinged lid. This hinged lid was key for me as it allowed me to attach a keyboard, trackball mouse, and small monitor.

On the inside I have two ATX motherboards with dual core Athlon 64s, though I could have used anything had I felt like spending the money. The worker node has two graphics cards and an extra NIC for regular network traffic (the onboard gigabit NIC is used for message passing). The head node has an extra WiFi NIC as well for talking with the outside world. There are then two switches, one for each internal network, and two hard drives off the head node. The worker node boots off a USB stick. I found Ubuntu installed from a live CD provides a nice, small OS.

It's a little cramped (the top sides of the motherboards face each other), but there's enough room for the power supplies to divide the space down the middle, with the switches and hard drives mounted above that and opposite of one another. Everything is held in place with L brackets, plexiglass, screws and spacers. Between Newegg (computer hardware), Amazon (keyboard, mouse, and monitor), and the Home Depot (box and mounting hardware) the whole project only cost about $1,000.

What's really nice is that there's room enough in the box for four ATX systems with expansion cards, or probably eight-ten mini-ITX boards if you wanted to go that route.

If you haven't already, add these sites to your research:
http://www.clustermonkey.net/
http://debianclusters.org/index.php/Main_Page
http://www.calvin.edu/~adams/research/microwulf/
They were extreemly valuable to me.

It won't be particularly easy, but it will be fun and rewarding like no other, and it makes a great mobile monster to show off to your friends!

Been there done that, maybe want to consult Linux Cluster Urban Legends before you continue down this path

Forgot this link! Why Linux On Clusters?

If you look at the disruptive growth of Linux HPC clusters, you will find that there are no IP agreements. I actually wrote about this: Why Linux On Clusters?. The absence of IP agreements allowed the HPC community to work together and grow faster than anyone imagined. On the famed Beowulf mailing list (started by Don Becker BTW) their is a free exchange of ideas and no one claims ownership of any IP. I call it a "Lawyer Free Zone" similar to what was proposed in Scotland back in the late 90's.

I entered "clustermonkey", which is a site about HPC Linux Clusters, and it came back with references just fine. However, it had totally out of context graphics like flowers and catalogs or something I cannot really make out. I don't really think this "cute" feature helps at all, as a matter of fact I think it reduces the quality of the search. Clustermonkey.net is a cluster geek site, if you search and scan the thumbnails (which is what attracts your eye) you will be seriously mis-led as to the results. Obviously, it is try to grab "contextual" graphics that does not work quite right.

First, the Top500 list has plenty of value. What most people do not realize (or should realize) is it is one data point on the HPC spectrum. If your HPC program does not perform the same or similar matrix operations as HPL then the ranking is meaningless to you. To some the list has become a public relations contest.

Second, performance is virtually independent of the OS (unless you are using TCP). Most big clusters use InfiniBand and run applications in "user space" by-passing the kernel. The rest of the code is crunching numbers.

Third, for the right cost, anyone can get a system on the Top500 list. It is a rather simple price/performance calculation, by the way. Breaking into the top 10 might be a little more difficult.

Finally, HPC and Linux are synergistic. Take a look at Why Linux on Clusters? to get the full story. The Windows model does not work very well in this space.

The HPC Cluster people have thought about this stuff for a while. One approach that I have thought about is described in the article:Cluster Programming: You Can't Always Get What You Want There are two follow-on articles as well Cluster Programming: The Ignorance is Bliss Approach" and Cluster Programming: Explicit Implications of Cluster Computing.

Of course if you really want to know how I feel about this: How The GPL Can Save Your Ass

enjoy

The HPC Cluster people have thought about this stuff for a while. One approach that I have thought about is described in the article:Cluster Programming: You Can't Always Get What You Want There are two follow-on articles as well Cluster Programming: The Ignorance is Bliss Approach" and Cluster Programming: Explicit Implications of Cluster Computing.

Of course if you really want to know how I feel about this: How The GPL Can Save Your Ass

enjoy

The HPC Cluster people have thought about this stuff for a while. One approach that I have thought about is described in the article:Cluster Programming: You Can't Always Get What You Want There are two follow-on articles as well Cluster Programming: The Ignorance is Bliss Approach" and Cluster Programming: Explicit Implications of Cluster Computing.

Of course if you really want to know how I feel about this: How The GPL Can Save Your Ass

enjoy

Every time these "connect desktops to become the fastest computer in the world" articles come up, I have to dust off my Cluster Urban Legends article to clear up the mis-conceptions that abound. I also did a piece on the Linux Magazine site as well that debunks much of the spam-bot supercomputer legend (need to register for that one)

This link would be more appropriate for slashdotters:

http://www.clustermonkey.net//content/view/211/1/

Well I have not solved the problem just yet either, but I thought about it quite a bit. I looked at it mostly from a HPC cluster standpoint, but problem is still the same -- software!

Imagine taking your CPU and breaking it into sub-units and then having the user reconnect them with other hardware. Sounds like a programming nightmare. That is exactly the state of parallel computing with clusters and with multi-core (Intel, AMD, and IBM, Sun multi-core are different enough to make a single programming approach "hard")

The problem "depends" on what you are trying to do. Parallel rendering -- simple. Proteins in water - hard (but much of this work has been done fortunately).

Parallel programming is faced with the problem that there is no guarantee that a parallel program will be BOTH portable and efficient for a given architecture. (largely due to what I mention above)

Over my 15+ years of working with parallel programming I have had some ideas. I think the expectation levels need to be lowered i.e. 3 weeks with a functional language equals 2.7 speedup, 2 months with threads or OpenMP or MPI equals 3.5 speedup on 4 cores. Of course the functional language is not quite ready yet!

Imagine taking your CPU and breaking it into sub-units and then having the user reconnect them with other hardware. Sounds like a programming nightmare. That is exactly the state of parallel computing with clusters and with multi-core (Intel, AMD, and IBM, Sun multi-core are different enough to make a single programming approach "hard")

The problem "depends" on what you are trying to do. Parallel rendering -- simple. Proteins in water - hard (but much of this work has been done fortunately).

Parallel programming is faced with the problem that there is no guarantee that a parallel program will be BOTH portable and efficient for a given architecture. (largely due to what I mention above)

Over my 15+ years of working with parallel programming I have had some ideas. I think the expectation levels need to be lowered i.e. 3 weeks with a functional language equals 2.7 speedup, 2 months with threads or OpenMP or MPI equals 3.5 speedup on 4 cores. Of course the functional language is not quite ready yet!

Those of us that use HPC clusters (i.e. Beowulf) have been thinking about these issues as well. For those interested, I wrote a series of articles on how one might program 10,000 cores (based on my frustrations as programmer and user of parallel computers). Things will change, there is no doubt.

The first in the series is called Cluster Programming: You Can't Always Get What You Want The next two are Cluster Programming: The Ignorance is Bliss Approach, and Cluster Programming: Explicit Implications of Cluster Computing.

Comments welcome.

Those of us that use HPC clusters (i.e. Beowulf) have been thinking about these issues as well. For those interested, I wrote a series of articles on how one might program 10,000 cores (based on my frustrations as programmer and user of parallel computers). Things will change, there is no doubt.

The first in the series is called Cluster Programming: You Can't Always Get What You Want The next two are Cluster Programming: The Ignorance is Bliss Approach, and Cluster Programming: Explicit Implications of Cluster Computing.

Comments welcome.

Those of us that use HPC clusters (i.e. Beowulf) have been thinking about these issues as well. For those interested, I wrote a series of articles on how one might program 10,000 cores (based on my frustrations as programmer and user of parallel computers). Things will change, there is no doubt.

The first in the series is called Cluster Programming: You Can't Always Get What You Want The next two are Cluster Programming: The Ignorance is Bliss Approach, and Cluster Programming: Explicit Implications of Cluster Computing.

Comments welcome.

This announcement is great news because the parallel programming problem is quite difficult and is becoming more important as multi-core systems emerge. One important distinction that is often not made, is the difference between concurrency and parallel execution. (although the article does touch on it)

Basically, determining whether a program or algorithm is concurrent (parts can computed independently) is possible but can be difficult in some cases. Many people think that is the essence of parallel computing. It is not.

Once you have the concurrent parts, the questions becomes "whether they should be executed in parallel". The answer depends upon the ratio of computation to communication (parallel overhead). All parallel computers (and clusters) have different ratios and therefore, something that runs well in parallel on one machine, may run poorly on another.

Having a language where concurrency can be expressed and controlled, allows researchers to investigate the second issue (parallel scheduling).

If you want to read more about this kind of thing (and some other parallel programing ideas) take a look at some of the Cluster Programming articles on ClusterMonkey.

And in some cases the Pentium D performs quite well

We have been using dual processor systems for a while. No one really complained that they were not "dual core" and got quite a lot of work out of them. Gluing two cores together was quicker and easier than a true dual core, and if it is the same price as a single core, I'll take it.

The military people will probably laugh at this (should they even encounter it). First, there are very few High Performance Computing applications that can use this type of computer (For those that need some background please see Linux Cluster Urban Legends pay attention to the latency part -- there is a shower scene)

Second, if the military had some use for this type of computing, they would either build their own software, hire someone to write it for them, or just buy a cluster. The administration and security headaches of a "open network p2p computer" certainly outweigh any advantages they would gain from this software.

But, your software - your license.

Cray finally figured it out. I have been saying for years: HPC/Beowulf clusters are about building machines around problems

That is why Clusters are such a powerful paradigm. If your problem needs more processors/memory/bandwidth/data access, you can design a cluster to fit your problem and only buy what your need. In the past you had to buy a large supercomputer with lots of engineering you did not need. Designing clusters is an art, but the payoff is very good price-to-performance. A good article on this topic is the Cluster Urban Legends, which explains many of these issues.

I haven't looked into that "Killer-NIC" (not interested anyway) but i know a bit about latency; people building cluster are investing a great deal of time reducing the latency the operating system's TCP/IP stack introduces. This can make up a difference of ~100-microseconds. So chosing the right network technology to deliver your data from one host to another one takes a great deal of time and lot's of effort is put into the research of new networking-technologies.

See Infiniband, or the various TCP-offloading features, take Level 5's Etherfabric or projects like MPI/GAMMA who circumvent the whole IP-shebang directly and let the programm build up it's own Ethernet-frames.

So i wouldn't necessarily denouce the whole "Killer-NOC" thing as bullshit; but on the other i must admit: The network in between (speak: Internet) is the problem for gaming, NOT necessarily the IP-stack or NIC.

I'm not a gamer so i can't tell if a difference in latency of, let's say, 100 us really brings an advantage (he, folks, that's 0.01 milliseconds!) - but well, let's see what kinda specs the company is going to release.
Alex.

P.S.: The Cluster Monkey had quite an extensive comparison of different low-latency high-bandwdith network-technologies: http://www.clustermonkey.net/content/view/124/34/

I have re-read this bit like twenty times and still have no idea what it means. The terms used clashes badly, which leads me to believe that the guy has no idea what he was talking about

It's called Clustering. You might say that clustering is the Quantum Mechanics of the networking world, in that the usual rules are... well... weird.

There are some good tutorials on this at ClusterMonkey's web site.

Oh, and BTW, TFA has been edited to fix the "millisecond" goofiness. Well, some of it anyway.

It is the cluster equivalent of how many people you can stuff into phone booth. If you have nothing better to do, well why not I guess.

HPC/Beowulf clusters are about building machines around problems

That is why Clusters are such a powerful paradigm. If your problem needs more processors/memory/bandwidth/data access, you can design a cluster to fit your problem and only buy what your need. In the past you had to buy a large supercomputer with lots of engineering you did not need. Designing clusters is an art, but the payoff is very good price-to-performance. I even wrote an article on Cluster Urban Legends the explains many of these issues.

Not that every program needs to use extra CPUs, but developers who have experienced continued speed "free lunch" improvements are going to hit a wall unless they start thinking in terms of threads, OpenMP, and MPI. You can check out Cluster Monkey for infromation on cluster computing which has been dealing with these issues for the last ten years.

This is kind of odd. Burton Smith is not really a cluster guy, although he probably knows his way around HPC (High Performance Computing). Cray is not really a cluster company (except for the system they bought from Octiga Bay deal). If you want to read a review of what Bill Gates said at the recent Supercomputing conference, check out Where is the Cluster? at Cluster Monkey.

Doug - a genuine Cluster Monkey

The lines have blurred due to clusters. My definition is "a collection of hardware that provides a non-trivial level of performance on a single problem" Of course, "non trivial" has various interpretations. And, working toward solving a single problem is important. Rendering is a trivial parallel application as it is really a bunch of small independent problems. Most supercomputer applications would probably run "sub-optimal" on this system (I assume it has GigE as an interconnect) because they require much more processor to processor communication. BTW, I run the ClusterMonkey site that talks about clusters and HPC if you want to learn more about clusters.

For those interested, there is a new website on clusters called ClusterMonkey. It just got started and has plenty of good free content (and more is coming).