Slashdot Mirror
Mini-ITX Clustering
NormalVisual writes "Add this cluster to the list of fun stuff you can do with those tiny little Mini-ITX motherboards. I especially like the bit about the peak 200W power dissipation. Look Ma, no fans!! You may now begin with the obligatory Beowulf comments...."
... a beowulf cluster of obligatory beowulf cluster comments.
I decided against a mini-ITX cluster because the floating point performance (why else would you build a cluster?) of VIA CPUs is just abyssmal.
Is there any reason why there are no P4 or AMD mini-ITX mobos around?
The owls are not what they seem
All things considered, what's the cost-per-tflop of that sort of system. These guys don't require as much cooling, space, or whatever else you care to think about.
Has anyone tried stuffing several into a single 1U chassis? For a sort of cluster of clusters?
tasks(723) drafts(105) languages(484) examples(29106)
...a new, original joke. Now imagine another one, because that last one wasn't that funny.
In fact, maybe you just aren't that funny. Except in Soviet Russia.
Shit, now I'm doing it.
Do not taunt Happy Fun Ball(TM)
We studied 3 mini beowulf systems a while back, here at University of Central Florida, one of which was a mini-ITX beowulf. Here's some info and preliminary results: http://helios.engr.ucf.edu/beowulf/miniature.phtml
Here's a picture of our first 4 boxes. The USB stick seen sticking out from one of the boxes is bootable and an excellent replacement for floppy disks...
"I love my job, but I hate talking to people like you" (Freddie Mercury)
There was no cutting or bending involved. All metal bits were simply cut, drilled, and bolted together using 4-40 hardware.
So what was it? No cutting, or cutting?
They who would give up an essential liberty for temporary security, deserve neither liberty nor security
Whilst not clustering, a good use for these low power systems would be for web hosts or budget dedicated servers. I'm sure a server room full of these would require much less airconditioning (and power) than the typical servers. Many people require dedicated servers for security (they are the only one on the box) and don't require fast FPU performance.
Oolite: Elite-like game. For Mac, Linux and Windows
Just what do you do with such a thing? I don't mean obvious commercial uses, but as a home-bound geek, what reason can I use to justify this to my wife?
I built a Mini-ITX based massively parallel cluster named PROTEUS. I have 12 nodes using VIA EPIA V8000, 800 MHz motherboards. The little machine is running FreeBSD 4.8, and MPICH 1.2.5.2. Troubles installing and configuring Free BSD and MPICH were few. In fact, there were no major issues with either FreeBSD or MPICH.
/usr on the controlling node via NFS, for storage and to allow for a very simple configuration. No official benchmarks have been run, but for simple computational tasks the mini cluster appears to be faster than four 2.4 GHz pentium 4 mcahines used in parallel, at a fraction of the cost and power use.
The construction is simple and inexpensive. The motherboards were stacked using threaded aluminum standoffs and then mounted on aluminum plates. Two stacks of three motherboards were assembled into each rack. Diagonal stiffeners were fabricated from aluminum angle stock to reduce flexing of the rack assembly.
The controlling node has a 160 GB ATA-133 HDD, and the computational nodes use 340 MB IBM microdrives in compact flash to IDE adapters. For file I/O, the computational nodes mount a partition on the controlling node's hard drive by means of a network file system mount point.
Each motherboard is powered by a Morex DC-DC converter, and the entire cluster is powered by a rather large 12V DC switching power supply.
With the exception of the metalwork, power wiring, and power/reset switching, everything is off the shelf.
At present, the idle power consumption is about 140 Watts (for 12 nodes) with peaks estimated at around 200 Watts. The machine runs cool and quiet. The controlling node has 256 MB RAM , and an 160 GB ATA 133 IDE hard disk drive. The computational nodes have 256 MB RAM, each and boot from 340 MB IBM microdrives by means of compact flash to IDE adapters. The computational nodes mount
Power and Cooling
Mini-ITX boards have very low power dissipation as compared to most motherboard/cpu combination in popular use today. This means that a Mini-ITX cluster with as many as 16 nodes won't need special air conditioning. Low power dissipation also means low power use, so you can use a single inexpensive UPS to provide clean AC power for the nodes.
In contrast, a 12-16 node cluster built with Intel or AMD processors will generate enough heat that you will likely need heavy duty air conditioning. Additionally, you will need adequate electrical power to deliver the 2-3 kilowatts peak load that your 12 node PC cluster will require. Plan on having higher than average utility bills if you use PC's...
Hardware Construction
The cluster is built in two nearly identical racks. Each rack has two stacks of three motherboards and dc-dc converters mounted on aluminum standoffs.
The compact flash adapters used to mount the microdrives are also in stacks of three. Each stack of boards is mounted on a 7 inch by 10 inch 0.0625 thick 6061-T6 aluminum plate as are the microdrive stacks. There are seven metal plates in all, in each rack.
The top cover plate has the mounting bracket for the 6 on/off/reset switches.
The plate below it is home to the power distribution terminal block. The power delivery cable for each rack is heavy duty 14 gauge stranded wire with pvc insulation. The power cabling from the terminal strip to each of the dc-dc converters is 18 gauge stranded pvc insulated hookup wire. The wiring for the power/reset switches is 24 gauge stranded, pvc insulated wire.
The top rack houses nodes one through six (node one is the controlling node). The bottom plate of the top rack also houses the 160 GB ATA-133 hard disk drive used by the controlling node. All other nodes make use of the IBM microdrives. Node number three has a spare compact flash adapter which can be used to duplicate microdrives for easy node maintenance.
The disk drive and power cabling to the motherboards was dressed as was sanely possible on the back panel. The liberal use of nylon cable ties helps reduce the ten
sheesh that didn't take long.
l m l m l
I managed to get it mirrored here:
page 1:
http://www.phule.net/mirrors/mini-itx-cluster.htm
page 2:
http://www.phule.net/mirrors/mini-itx-cluster2.ht
page 3:
http://www.phule.net/mirrors/mini-itx-cluster3.ht
In Soviet Russia...michael would be rotting in Siberia!
Actually, he's not. IBM Micro Drives are not CF, they just have a CF form factor/interface to be compatible with hand held devices. They are hard drives.
.technomancer
Too Many Users
Evidently they didn't cluster enough...
What?
I agree, but that's actually a very interesting use. It also lets you play around with network topologies, and interconnects, and such. And of course, these boards do have one PCI slot, as well as the standard assortment of serial and parallel, so the hardware people can have fun too. For real number crunching? Not a chance. For doing a $2000 prototype, in 15 nodes, of a $50000 50-node cluster? I can't really think of a more flexible, more convenient, or more affordable option. For doing a $1000, 6-node flexible network simulator, purely for education? Also more than worth it, with few other options around.
I've had this sig for three days.
I have been thinking about this lately. I get disgusted by the fanns everywhere (especially since the one in my laptop makes an awful amount of noise sometimes and still doesn't prevent the beast from overheating and shutting down). Aside from being noisy, computers have way more CPU power than I need, and cost more than I am willing to spend. And they suck up a lot of power. (Some might add that they take a lot of space.)
I think all of these could be solved at once. What if someone built low-power, low-noise, and low-cost computer, good enough for running light office applications? I don't mean OpenOffice, but rather lightweight programs that implement the functionality people use _without_ the bloat. My 486 handles email just fine and the WYSIWYG word processors were once satisfied with a first-generation Pentium (and even these were already bloated).
Current PDAs have more than enough processing power to handle those tasks, and I've noticed that company's like gumstix build and sell devices almost like what I have in mind (the gumstix don't seem to have display connectors, though). Hey, these machines could actually be portable and have a really decent battery life (more than a full working day); that would be a killer!
Am I just daydreaming here or are others with me? Maybe you know of devices that do this job? Someone recommended Sharp's Zaurus, which is excellent, but still rather more expensive than what I have in mind.
Please correct me if I got my facts wrong.
I'd just like to point out that 12 nodes is not "massively parallel."
A beowolf of mini-itx boards is probably the cheapest way to get bragging rights. As a practical way of fast and cheap parallel computation they are not.
However, I have purchased three (V10000 boards) thus far and intend to add more to my network as low power (as in Watts) servers.
I worked out that given the power of 10.78W (source: mini-itx.com's power comparison tool) for the V series (probably the one with the slowest CPU in the series, board only), I could save a fortune on electricity compared to a more regular computer.
The electricity company sells electricity at the rate of 0.63 ($1.18) per watt per year. Compared with a standard PC of 100W, I can regain the purchase costs (in savings) of the board and memory within two to three years.
Also, I found rack mount chassis available cheaper than one for a regular sized case. This influenced my decision a little - who doesn't want a network of rack mounted computers?
Overall, because of the low price and low power the mini-itx boards are a no brainer if and only if the CPU power of each computer isn't important.
Jonathan
You are someone who actually did imagine a beowulf cluster of those!
Around here, that must make you a god!
Tuus crepidae innexilis sunt.
The original links went to NASA/GSFC, but the Beowulf project central site has moved.
I mean, those IBM 340 MB microdrives aren't really that cheap... you can get full size hard drives for the same price...
I've always wondered; why not PXE boot something like this? Set your node controller to also do DHCP and you're set.
While you're at it, use the CL version for the controller which has two network cards and build a NATTING firewall into the node controller too. Then you have a plug-in appliance that doesn't interfere with your network topology at all. PXE boot it and the motherboards will only need RAM.
The board he used is available for $99 with proc. A stick of 256 is probably around $20.
The best price froogle would give me on the drives he's using is $60, and they're prone to wear and tear.
Add in the $10 CF-IDE adapter and the drive is %60 of the cost of the motherboard itself...
Hell if you don't want the network bogged down with a bunch of PXE booting nodes all the time, just get cheap CD drives and put dyne:bolic on it, which does automagic clustering...
Personally, if I were to do it, I'd set dynebolic to PXE boot, get a huge stack of motherboards and RAM, and do it that way. Then adding/changing nodes is relatively simple... IIRC, they're even factory set to try PXE booting if no IDE devices are found...
The only other change I would make would be to ditch the 16-port switch... move to 4-ports, connect those to a 4-port with gigabit uplink, and connect that to a gigabit switch. Of course at this point I'm talking about really scaling the cluster up, to a few hundred nodes or so. At that point I'd stop using a mini-ITX board for my node controller and go with a motherboard with a bit more juice behind it, dual procs, RAID 0/1, the whole shebang...
Now if only I had a couple grand burning a hole in my pocket... speaking of which:
motherboard: $100
RAM: $20
DC-DC converter: $30
CF adapter: $10
Microdrive: $60
Total: $220
Total PXE booter: $150
Savings: 30%
So, not counting the costs of cabinets, power rectifier/UPS, wiring, network gear, and labor, you can increase the size of your cluster by %30 for the same cost, just for setting up PXE boot...
I am disrespectful to dirt! Can you see that I am serious?!
LO, praise of the prowess of people-kings
:)
of spear-armed Danes, in days long sped,
we have heard, and what honor the athelings won!
Oft Scyld the Scefing from squadroned foes,
from many a tribe, the mead-bench tore,
awing the earls. Since erst he lay
friendless, a foundling, fate repaid him:
for he waxed under welkin, in wealth he throve,
till before him the folk, both far and near,
who house by the whale-path, heard his mandate,
gave him gifts: a good king he!
To him an heir was afterward born,
a son in his halls, whom heaven sent
to favor the folk, feeling their woe
that erst they had lacked an earl for leader
so long a while; the Lord endowed him,
the Wielder of Wonder, with world's renown.
Famed was this Beowulf
Sample from the Project Gutenberg Text of Beowulf.
Why not do yourself a favour and download it. Classic stuff.
siggy played guitar