It is not very often a CPU fails. I'd imagine the nominal operating life of those cpu's is at least 7 years, and I bet the failure rate is probably something like one failure in a thousand years. Since the machine runs rather cool, I'd bet the whole thing will last a lot longer due to the reduced heat rise in the cabinet.
WEll, GFLOPS is not the whole picture. With 12 cpu's you have a lot of parallelism. Plus I think the higher instruction clock rate (in aggregate) will add a lot more horsepower than one might think. Plus, if you happen to need to develop code for many nodes and don't want to spend a bunch of money on the machine, power, and air conditioning, this kind of cluster might be the ticket. I think the correct answere to the question "Is a low power cluster appropriate?" is "It depends on what you are doing and how you are doing it".
you might try a 1U power supply, it will lstill be big, but at least will only be 1 inch high
Re:Waaaiiit a Second! Credit where Credit is Due..
on
Mini-ITX Clustering
·
· Score: 1
Don't be such an asshat.
Re:Waaaiiit a Second! Credit where Credit is Due..
on
Mini-ITX Clustering
·
· Score: 1
That is really cool. There is alslo a fellow who built a 16 node cluster and was getting quite good performance from it. I do not have the url for his page, maybe he will come forward with a link.
Re:A beowulf cluster of FreeBSD machines?
on
Mini-ITX Clustering
·
· Score: 1
Oh yah. FreeBSD does a fine job for clustering.
Linux is a little more popular for clustering due to the fact that places like NASA tooled up for parallel on Linux a long time ago. Now you have an assortment of commercially available tools for parallel computing that are available for Linux. Less so, for FBSD, when it comes to commercial applications, but I've yet to find a properly written tar that I could not extract and build on FBSD. All of the essential clustering tools work on FBSD with no problems.
First of all I used EPIA V8000A mainbords, not the EPIA 800. The performance is more than adequate. Secondsly, there will likely be another article giving the final , offical stats on performance power and cost per mflp since these seem to be the figures people want to see. From my experience using the mini cluster so far, it seems to be fast and effective, and quite usable for serious work. I don't thikn I'd want to put 200 users on it and run the human genome project code on it, but I'd sure not hesitate to use it for parallel type applications that would normally bog down may single cpu boxes.
MPI puts the parallel tasking in the hands of the programmer. In essence, you pass a variable to the nodes and all the nodes chew on it and the results are then collected and hopefully assembled into something usable. It is recursive, and a bit indeterminant because you don't know exactly when a particular node is going to finish it's share of the computing tasks.
Alternatively uou can bypass MPI and just run individual programs on all the nodes. Some people even go so far as to send source code from the controlling node to the computational nodes where it is then compiled and executed. Another common thing to do is to just use shell scripts to talk to a bunch of computers on the network and send different parts of the job to variaous machines. There are lots of ways to do things in parallel. The definition of "cluster" is pretty fuzzy.
I'd like to know what benchmark you used to yield 6 GFLP on a single 2.4 GHz p4 ? I'd believe 6 GFLP peak, but not sustained. I might accept close to a GFLP sustained. If you are basing that on a theoretical maximum of 6 GFLP, then one could argue that the mini cluster is good for 9.6 GFLP minimum at 12 nodes.
Try this test: use your favorite compiler and write a program that does 6 floating point calculations inside a loop that iterates 12 billion times, that comes out to 12 billion calculations. I'm betting your 2.4 GHz p4 is gonna take several minutes to do it. the mini cluster does it in about 30 seconds.
Gigabit would be faster, but by the time you add gigabit cards you end up with a lot more cost and a LOT more power use (the machine has 12 nodes).
For just running the parallel programs fast ethernet seems fine. MPI is not inordinately chatty on the network. If you do a lot of file I/O you can run into bottlenecks there unless you buffer to memory, or have your file I/O servers on a separate network.
Oh I had a problem to solve. The problem was how to put a cluster in an efficiency apartment without turning it into a sauna and/or going broke. The mini itx boards turned out to be the only solution that was workable. In particular, the EPIA V8000A board looked like the lowest cost and power use, with best usable performance ratio, so that is what I went with. I was originally going to use the M series boards and use firewire at 400 Mbit for the network, but working support for firewire at full bandwidth was questinable at the time I started the project, and the cost of the M series boards went up a lot, so the EPIA V8000A proved to be the best deal at the time.
I concur. I learned a lot from building this beastie. The project was fun, I am getting great results , and I learned a new trick or two, and yes it was all done for really cheap (for a cluster project). Total cost was under $3500 and it could be done a lot cheaper if you wanted to use pxeboot.
1) bar stock was not as stiff as angle stock. 2) bar stock would be heavier. 3) It is more about low cost than looks, so it does not matter if the diagonal stiffeners look funny. All they have to do is keep the chassis from falling apart.
The original 6 microdrives have been up for a few months with no problems. They are little bitty drives, so don't work them to death and they will probably last a long time.
Let's clarify this. The EPIA has the C3 cpu which runs the FPU at system clock.
The EPIA 800 has the eden processor and runs the fpu at 1/2 clock. The EPIA V 8000 A has the C-3 and runs the fpu at full clock. This info comes from the spec sheets on the VIA web site btw. For floating point, the C3 is fine. The Mini Cluster is using the EPIA V 8000 A motherboard, so FPU performance is fine and more than adequate.
Did you use EDEN processors or C3 ?
the EDEN processor in the EPIA 800 mainboard is EDEN and the fpu clocks at 1/2 of system clock (400 MHz).
The C3 processor in the EPIA V8000A is a C3 and clocks at system clock (800 Mz)
It is understandable that you only got 1 GFLP.
Of course, you get much lower power use with the EPIA 800 than with the EPIA V 8000 A board...
Go to the VIA web site and read the specs on C3 and EDEN processors. they are different cpus , designed for different tasks. For a computational cluster, the eden core is not a great choice. It would be okay for a database cluster or a web server, etc.
If you want to do some numbers and get okay performance, use the C3 or the M series processors.
low cost, reasonable usable computing power, and low power dissipation.
they are so inexpensive you just use more of them to get the computing power you need, and still end up with less power dissipation.
correct, the original design was 10, and I decided that 12 would be needed.
Interestingly enough, I accidentally kiiled one node, and another flaked out during stress testing.
I've since replaced the two and it is back at capacity with 12 nodes.
It took about 30 minutes to replace them and bring it all back online.
As I said in another post, I started wit hthree and kept doubling.... anyone ever read A.C> Clarke "Rondevouz with Rama" ? They did everything in threes...
Yes, it has more power than several p4's and is quiet, cooler, and I hope reliable...
But it is also fun to build a computer that is faster than anything I could afford to buy ready-made.
Re:Image a Beowulf Cluster of these babies!
on
Mini-ITX Clustering
·
· Score: 1
Yes, linux is a popular os for clusters. I found FreeBSD to work a little better for me.
Slashdot Mirror
It is not very often a CPU fails. I'd imagine the nominal operating life of those cpu's is at least 7 years, and I bet the failure rate is probably something like one failure in a thousand years. Since the machine runs rather cool, I'd bet the whole thing will last a lot longer due to the reduced heat rise in the cabinet.
Imagine that!
WEll, GFLOPS is not the whole picture. With 12 cpu's you have a lot of parallelism. Plus I think the higher instruction clock rate (in aggregate) will add a lot more horsepower than one might think. Plus, if you happen to need to develop code for many nodes and don't want to spend a bunch of money on the machine, power, and air conditioning, this kind of cluster might be the ticket. I think the correct answere to the question "Is a low power cluster appropriate?" is "It depends on what you are doing and how you are doing it".
This machine went up for sale on ebay today. Go to ebay and search for "mini itx cluster" and give it a look.
you might try a 1U power supply, it will lstill be big, but at least will only be 1 inch high
Don't be such an asshat.
That is really cool. There is alslo a fellow who built a 16 node cluster and was getting quite good performance from it. I do not have the url for his page, maybe he will come forward with a link.
Oh yah. FreeBSD does a fine job for clustering. Linux is a little more popular for clustering due to the fact that places like NASA tooled up for parallel on Linux a long time ago. Now you have an assortment of commercially available tools for parallel computing that are available for Linux. Less so, for FBSD, when it comes to commercial applications, but I've yet to find a properly written tar that I could not extract and build on FBSD. All of the essential clustering tools work on FBSD with no problems.
First of all I used EPIA V8000A mainbords, not the EPIA 800. The performance is more than adequate. Secondsly, there will likely be another article giving the final , offical stats on performance power and cost per mflp since these seem to be the figures people want to see. From my experience using the mini cluster so far, it seems to be fast and effective, and quite usable for serious work. I don't thikn I'd want to put 200 users on it and run the human genome project code on it, but I'd sure not hesitate to use it for parallel type applications that would normally bog down may single cpu boxes.
cool. I saw the aurora in Alaska last year. It was really awesome.
MPI puts the parallel tasking in the hands of the programmer. In essence, you pass a variable to the nodes and all the nodes chew on it and the results are then collected and hopefully assembled into something usable. It is recursive, and a bit indeterminant because you don't know exactly when a particular node is going to finish it's share of the computing tasks. Alternatively uou can bypass MPI and just run individual programs on all the nodes. Some people even go so far as to send source code from the controlling node to the computational nodes where it is then compiled and executed. Another common thing to do is to just use shell scripts to talk to a bunch of computers on the network and send different parts of the job to variaous machines. There are lots of ways to do things in parallel. The definition of "cluster" is pretty fuzzy.
I'd like to know what benchmark you used to yield 6 GFLP on a single 2.4 GHz p4 ? I'd believe 6 GFLP peak, but not sustained. I might accept close to a GFLP sustained. If you are basing that on a theoretical maximum of 6 GFLP, then one could argue that the mini cluster is good for 9.6 GFLP minimum at 12 nodes. Try this test: use your favorite compiler and write a program that does 6 floating point calculations inside a loop that iterates 12 billion times, that comes out to 12 billion calculations. I'm betting your 2.4 GHz p4 is gonna take several minutes to do it. the mini cluster does it in about 30 seconds.
Gigabit would be faster, but by the time you add gigabit cards you end up with a lot more cost and a LOT more power use (the machine has 12 nodes). For just running the parallel programs fast ethernet seems fine. MPI is not inordinately chatty on the network. If you do a lot of file I/O you can run into bottlenecks there unless you buffer to memory, or have your file I/O servers on a separate network.
Oh I had a problem to solve. The problem was how to put a cluster in an efficiency apartment without turning it into a sauna and/or going broke. The mini itx boards turned out to be the only solution that was workable. In particular, the EPIA V8000A board looked like the lowest cost and power use, with best usable performance ratio, so that is what I went with. I was originally going to use the M series boards and use firewire at 400 Mbit for the network, but working support for firewire at full bandwidth was questinable at the time I started the project, and the cost of the M series boards went up a lot, so the EPIA V8000A proved to be the best deal at the time.
I concur. I learned a lot from building this beastie. The project was fun, I am getting great results , and I learned a new trick or two, and yes it was all done for really cheap (for a cluster project). Total cost was under $3500 and it could be done a lot cheaper if you wanted to use pxeboot.
1) bar stock was not as stiff as angle stock. 2) bar stock would be heavier. 3) It is more about low cost than looks, so it does not matter if the diagonal stiffeners look funny. All they have to do is keep the chassis from falling apart.
The original 6 microdrives have been up for a few months with no problems. They are little bitty drives, so don't work them to death and they will probably last a long time.
Let's clarify this. The EPIA has the C3 cpu which runs the FPU at system clock. The EPIA 800 has the eden processor and runs the fpu at 1/2 clock. The EPIA V 8000 A has the C-3 and runs the fpu at full clock. This info comes from the spec sheets on the VIA web site btw. For floating point, the C3 is fine. The Mini Cluster is using the EPIA V 8000 A motherboard, so FPU performance is fine and more than adequate.
Did you use EDEN processors or C3 ? the EDEN processor in the EPIA 800 mainboard is EDEN and the fpu clocks at 1/2 of system clock (400 MHz). The C3 processor in the EPIA V8000A is a C3 and clocks at system clock (800 Mz) It is understandable that you only got 1 GFLP. Of course, you get much lower power use with the EPIA 800 than with the EPIA V 8000 A board... Go to the VIA web site and read the specs on C3 and EDEN processors. they are different cpus , designed for different tasks. For a computational cluster, the eden core is not a great choice. It would be okay for a database cluster or a web server, etc. If you want to do some numbers and get okay performance, use the C3 or the M series processors.
low cost, reasonable usable computing power, and low power dissipation. they are so inexpensive you just use more of them to get the computing power you need, and still end up with less power dissipation.
Clustering is a fine use for these. They are cheap to buy and operate. You just use more nodes to get the same power.
correct, the original design was 10, and I decided that 12 would be needed. Interestingly enough, I accidentally kiiled one node, and another flaked out during stress testing. I've since replaced the two and it is back at capacity with 12 nodes. It took about 30 minutes to replace them and bring it all back online. As I said in another post, I started wit hthree and kept doubling.... anyone ever read A.C> Clarke "Rondevouz with Rama" ? They did everything in threes...
Yes, it has more power than several p4's and is quiet, cooler, and I hope reliable... But it is also fun to build a computer that is faster than anything I could afford to buy ready-made.
Yes, linux is a popular os for clusters. I found FreeBSD to work a little better for me.
The home made rack costs about $120 to fab with all new stuff. and that houses all 12 nodes.