Mini-ITX Clustering
NormalVisual writes "Add this cluster to the list of fun stuff you can do with those tiny little Mini-ITX motherboards. I especially like the bit about the peak 200W power dissipation. Look Ma, no fans!! You may now begin with the obligatory Beowulf comments...."
A beowulf cluster of these? There, done... and it felt good!
... a beowulf cluster of obligatory beowulf cluster comments.
I decided against a mini-ITX cluster because the floating point performance (why else would you build a cluster?) of VIA CPUs is just abyssmal.
Is there any reason why there are no P4 or AMD mini-ITX mobos around?
The owls are not what they seem
Just imagine Dilbert's boss asking him for a Beowulf cluster.
Kind of like that strip where he (the boss) wanted to have a SQL database in lime.
... but that's about all it'll be useful for. A Nehemiah CPU is really weedy by todays standards, even the 1GHz one is about the same as a 600MH P3. So, he's got 12 of them, which is probably less CPU power than an average dual P4 motherboard...
Still, you can get some stats on how the clustering works, what's the best algorithm for dispersing problems, and these boards are cheap, but that's about the only advantage I can see...
Simon
Physicists get Hadrons!
All things considered, what's the cost-per-tflop of that sort of system. These guys don't require as much cooling, space, or whatever else you care to think about.
Has anyone tried stuffing several into a single 1U chassis? For a sort of cluster of clusters?
tasks(723) drafts(105) languages(484) examples(29106)
...a new, original joke. Now imagine another one, because that last one wasn't that funny.
In fact, maybe you just aren't that funny. Except in Soviet Russia.
Shit, now I'm doing it.
Do not taunt Happy Fun Ball(TM)
You know I seriously wonder if this would be a viable option for Computer chess programs (http://www.chessbase.com/newsdetail.asp?newsid=25 ). It certainly is getting cheap to get massive hardware processing power.
We studied 3 mini beowulf systems a while back, here at University of Central Florida, one of which was a mini-ITX beowulf. Here's some info and preliminary results: http://helios.engr.ucf.edu/beowulf/miniature.phtml
Here's a picture of our first 4 boxes. The USB stick seen sticking out from one of the boxes is bootable and an excellent replacement for floppy disks...
"I love my job, but I hate talking to people like you" (Freddie Mercury)
There was no cutting or bending involved. All metal bits were simply cut, drilled, and bolted together using 4-40 hardware.
So what was it? No cutting, or cutting?
They who would give up an essential liberty for temporary security, deserve neither liberty nor security
Whilst not clustering, a good use for these low power systems would be for web hosts or budget dedicated servers. I'm sure a server room full of these would require much less airconditioning (and power) than the typical servers. Many people require dedicated servers for security (they are the only one on the box) and don't require fast FPU performance.
Oolite: Elite-like game. For Mac, Linux and Windows
Just what do you do with such a thing? I don't mean obvious commercial uses, but as a home-bound geek, what reason can I use to justify this to my wife?
I built a Mini-ITX based massively parallel cluster named PROTEUS. I have 12 nodes using VIA EPIA V8000, 800 MHz motherboards. The little machine is running FreeBSD 4.8, and MPICH 1.2.5.2. Troubles installing and configuring Free BSD and MPICH were few. In fact, there were no major issues with either FreeBSD or MPICH.
/usr on the controlling node via NFS, for storage and to allow for a very simple configuration. No official benchmarks have been run, but for simple computational tasks the mini cluster appears to be faster than four 2.4 GHz pentium 4 mcahines used in parallel, at a fraction of the cost and power use.
The construction is simple and inexpensive. The motherboards were stacked using threaded aluminum standoffs and then mounted on aluminum plates. Two stacks of three motherboards were assembled into each rack. Diagonal stiffeners were fabricated from aluminum angle stock to reduce flexing of the rack assembly.
The controlling node has a 160 GB ATA-133 HDD, and the computational nodes use 340 MB IBM microdrives in compact flash to IDE adapters. For file I/O, the computational nodes mount a partition on the controlling node's hard drive by means of a network file system mount point.
Each motherboard is powered by a Morex DC-DC converter, and the entire cluster is powered by a rather large 12V DC switching power supply.
With the exception of the metalwork, power wiring, and power/reset switching, everything is off the shelf.
At present, the idle power consumption is about 140 Watts (for 12 nodes) with peaks estimated at around 200 Watts. The machine runs cool and quiet. The controlling node has 256 MB RAM , and an 160 GB ATA 133 IDE hard disk drive. The computational nodes have 256 MB RAM, each and boot from 340 MB IBM microdrives by means of compact flash to IDE adapters. The computational nodes mount
Power and Cooling
Mini-ITX boards have very low power dissipation as compared to most motherboard/cpu combination in popular use today. This means that a Mini-ITX cluster with as many as 16 nodes won't need special air conditioning. Low power dissipation also means low power use, so you can use a single inexpensive UPS to provide clean AC power for the nodes.
In contrast, a 12-16 node cluster built with Intel or AMD processors will generate enough heat that you will likely need heavy duty air conditioning. Additionally, you will need adequate electrical power to deliver the 2-3 kilowatts peak load that your 12 node PC cluster will require. Plan on having higher than average utility bills if you use PC's...
Hardware Construction
The cluster is built in two nearly identical racks. Each rack has two stacks of three motherboards and dc-dc converters mounted on aluminum standoffs.
The compact flash adapters used to mount the microdrives are also in stacks of three. Each stack of boards is mounted on a 7 inch by 10 inch 0.0625 thick 6061-T6 aluminum plate as are the microdrive stacks. There are seven metal plates in all, in each rack.
The top cover plate has the mounting bracket for the 6 on/off/reset switches.
The plate below it is home to the power distribution terminal block. The power delivery cable for each rack is heavy duty 14 gauge stranded wire with pvc insulation. The power cabling from the terminal strip to each of the dc-dc converters is 18 gauge stranded pvc insulated hookup wire. The wiring for the power/reset switches is 24 gauge stranded, pvc insulated wire.
The top rack houses nodes one through six (node one is the controlling node). The bottom plate of the top rack also houses the 160 GB ATA-133 hard disk drive used by the controlling node. All other nodes make use of the IBM microdrives. Node number three has a spare compact flash adapter which can be used to duplicate microdrives for easy node maintenance.
The disk drive and power cabling to the motherboards was dressed as was sanely possible on the back panel. The liberal use of nylon cable ties helps reduce the ten
Yet another example of why you shouldn't do everything that you can do! These puppies aren't exactly famous for their flop-per-dollar ratio. In truthfully, it would be more efficient ( and cost effective) to make the cluster out of PIIIs. Anyhow, I'm off to go cluster a few toaster ovens, I hear that they offer a great delicious to efficiency ratio. Chris
sheesh that didn't take long.
l m l m l
I managed to get it mirrored here:
page 1:
http://www.phule.net/mirrors/mini-itx-cluster.htm
page 2:
http://www.phule.net/mirrors/mini-itx-cluster2.ht
page 3:
http://www.phule.net/mirrors/mini-itx-cluster3.ht
In Soviet Russia...michael would be rotting in Siberia!
Oh, I forgot: each of these boxes contains a 2,8GHz P4 Northwood CPU (200/800MHz FSB), 1GB RAM. The Shuttle barebone used is the S75G2 and one of the reasons we chose it was that it has an on-board gigabit ethernet adapter. The CPU cooler that came with it is also very interesting - it uses a rather unique design with a heatpipe ...
"I love my job, but I hate talking to people like you" (Freddie Mercury)
Actually, he's not. IBM Micro Drives are not CF, they just have a CF form factor/interface to be compatible with hand held devices. They are hard drives.
.technomancer
I can't wait for the new, smaller nano-itx boards to come out. 4.5" on a side, 1GHZ CPU and draws 7 watts. I got an email from VIA claiming they will be released in April.
MB, slim DVD and laptop HD in a case the size of a large paperback book!
It will make my "K-Mart Toolbox Mini-ITX PVR" look like a full tower in comparison!
Six times as much as what? My entire mini-itx system was under $500, and most of the cost of that was a solid-state drive large enough for a decent linux distribution... and most of the rest was a touch-screen monitor.
I've had this sig for three days.
I have been thinking about this lately. I get disgusted by the fanns everywhere (especially since the one in my laptop makes an awful amount of noise sometimes and still doesn't prevent the beast from overheating and shutting down). Aside from being noisy, computers have way more CPU power than I need, and cost more than I am willing to spend. And they suck up a lot of power. (Some might add that they take a lot of space.)
I think all of these could be solved at once. What if someone built low-power, low-noise, and low-cost computer, good enough for running light office applications? I don't mean OpenOffice, but rather lightweight programs that implement the functionality people use _without_ the bloat. My 486 handles email just fine and the WYSIWYG word processors were once satisfied with a first-generation Pentium (and even these were already bloated).
Current PDAs have more than enough processing power to handle those tasks, and I've noticed that company's like gumstix build and sell devices almost like what I have in mind (the gumstix don't seem to have display connectors, though). Hey, these machines could actually be portable and have a really decent battery life (more than a full working day); that would be a killer!
Am I just daydreaming here or are others with me? Maybe you know of devices that do this job? Someone recommended Sharp's Zaurus, which is excellent, but still rather more expensive than what I have in mind.
Please correct me if I got my facts wrong.
I'd just like to point out that 12 nodes is not "massively parallel."
You are someone who actually did imagine a beowulf cluster of those!
Around here, that must make you a god!
Tuus crepidae innexilis sunt.
What characteristics do you want? There are two major types of solid state drives -- battery-backed ram, and flash ram. Both are expensive and small. Only one is fast.
My requirements were essentially (1) no moving parts, (2) affordable if not cheap, and (3) small. I settles on one of these. Debian is fine on 128MB, with 512MB of ram and no swap. Performance, it should be said, sucks. The next step up, for slightly more performance, much more capacity, and a whole lot more cost, is here; but I wanted to avoid using a case that needed drive bays, plus I haven't pockets that deep.
Neither of those is likely to be what you want for a database system, though. You're probably more in the market for a bunch of ram and a battery, unless your primary concern is reliability. If speed is the goal, you want this, or, for more capacity and more money, this. Note that I haven't used either extensively, and in playing around with the rocket a little, I was surprised just how much of a bottleneck PCI became. Also, the rocket doesn't have a battery... so really, unless you have a board with 8GB of memory, and you just need another 8GB of low latency space, it's not such a great deal today.
If you fit into any of the niches above, solid state is wonderful. It's always more expensive than you think, though. And for any database systems I've dealt with, a disk is without question the way to go, perhaps with more memory on board. But if you want any further tips, I'm glad to help.
I've had this sig for three days.
I mean, those IBM 340 MB microdrives aren't really that cheap... you can get full size hard drives for the same price...
I've always wondered; why not PXE boot something like this? Set your node controller to also do DHCP and you're set.
While you're at it, use the CL version for the controller which has two network cards and build a NATTING firewall into the node controller too. Then you have a plug-in appliance that doesn't interfere with your network topology at all. PXE boot it and the motherboards will only need RAM.
The board he used is available for $99 with proc. A stick of 256 is probably around $20.
The best price froogle would give me on the drives he's using is $60, and they're prone to wear and tear.
Add in the $10 CF-IDE adapter and the drive is %60 of the cost of the motherboard itself...
Hell if you don't want the network bogged down with a bunch of PXE booting nodes all the time, just get cheap CD drives and put dyne:bolic on it, which does automagic clustering...
Personally, if I were to do it, I'd set dynebolic to PXE boot, get a huge stack of motherboards and RAM, and do it that way. Then adding/changing nodes is relatively simple... IIRC, they're even factory set to try PXE booting if no IDE devices are found...
The only other change I would make would be to ditch the 16-port switch... move to 4-ports, connect those to a 4-port with gigabit uplink, and connect that to a gigabit switch. Of course at this point I'm talking about really scaling the cluster up, to a few hundred nodes or so. At that point I'd stop using a mini-ITX board for my node controller and go with a motherboard with a bit more juice behind it, dual procs, RAID 0/1, the whole shebang...
Now if only I had a couple grand burning a hole in my pocket... speaking of which:
motherboard: $100
RAM: $20
DC-DC converter: $30
CF adapter: $10
Microdrive: $60
Total: $220
Total PXE booter: $150
Savings: 30%
So, not counting the costs of cabinets, power rectifier/UPS, wiring, network gear, and labor, you can increase the size of your cluster by %30 for the same cost, just for setting up PXE boot...
I am disrespectful to dirt! Can you see that I am serious?!
Think "Office Space" people...
Peter Gibbons: Um, the 7-Eleven, right? You take a penny from the tray.
Joanna: From the crippled children?
Peter Gibbons: No, that's the jar. I'm talking about the tray, the pennies for everybody.
Video encoding? (Now, where'd I put that parallel-processing version of AVISynth?)
dyne:bolic is a Live CD distribution, very small, that can be PXE boot, with full audio/video capture/editing/processing/streaming capabilities plus the usual suite of tools, a few games and whatnot... and is auto-clustering on a private network.
I am disrespectful to dirt! Can you see that I am serious?!
LO, praise of the prowess of people-kings
:)
of spear-armed Danes, in days long sped,
we have heard, and what honor the athelings won!
Oft Scyld the Scefing from squadroned foes,
from many a tribe, the mead-bench tore,
awing the earls. Since erst he lay
friendless, a foundling, fate repaid him:
for he waxed under welkin, in wealth he throve,
till before him the folk, both far and near,
who house by the whale-path, heard his mandate,
gave him gifts: a good king he!
To him an heir was afterward born,
a son in his halls, whom heaven sent
to favor the folk, feeling their woe
that erst they had lacked an earl for leader
so long a while; the Lord endowed him,
the Wielder of Wonder, with world's renown.
Famed was this Beowulf
Sample from the Project Gutenberg Text of Beowulf.
Why not do yourself a favour and download it. Classic stuff.
siggy played guitar
As a green geek I can't resist pointing out this merit: with only a 200W power dissipation this would be "home friendly" even in a non air conditioned house during the hot Mississippi summers. And with only a 200W PEAK draw, the entire system could be powered by a single PV panel and one or two storage batteries. Trade the "high quality UPS" for a couple of batteries and a PV panel (or cheaper still if you're in the midwest or near a coastline, a windmill) and you have a cluster that could run without any "store bought" AC at all.