Mini-ITX Clustering

NormalVisual writes "Add this cluster to the list of fun stuff you can do with those tiny little Mini-ITX motherboards. I especially like the bit about the peak 200W power dissipation. Look Ma, no fans!! You may now begin with the obligatory Beowulf comments...."

Comments

Floating point performance

[October_30th]

I thought about this some time ago.

I decided against a mini-ITX cluster because the floating point performance (why else would you build a cluster?) of VIA CPUs is just abyssmal.

Is there any reason why there are no P4 or AMD mini-ITX mobos around?

Re:Floating point performance

[steveha]

the floating point performance [...] of VIA CPUs is just abyssmal.

Older C3 cores run the FPU at half the clock rate. If you get the fanless 600 MHz EPIA motherboard, the FPU will be running at 300 MHz.

The newer, Nehemiah core C3 chips run the FPU at full clock speed. Any C3 newer than Nehemiah should run the FPU at full speed.

He used the VIA EPIA V8000A motherboard with an Eden core CPU. From what I found on google (here), the Eden core does run the FPU at full clock speed.

In any event, he said the cluster has more processing power than a four-P4 SMP system, while taking less electricity to run. And it will be quieter and more reliable. I'd like to see actual benchmarks, but it seems like it makes enough sense.

I read about a cluster of PocketPCs, and that didn't make practical sense. It was just a fun project.

steveha

Re:Floating point performance

[mangu]

The floating point is just a convenience. Almost any algorithm can be modified to work with fixed point precision -- and without loss of performance.

But at a significantly higher development and debugging cost. Why go for integer adaptation, if a P4 can do four FP operations in one clock, using SSE2? I have tested my 2.4GHGz P4 at 6 gigaflops, in a practical application doing matrix inversion. The theoretical maximum for my machine would be 9.6 Gflops. If you RTFA, you'll see they mention 3.6 Gflops performance for their cluster, about 60% of my single-processor system. I see no point at all in building that cluster.

Seriously, though...

[Short Circuit]

All things considered, what's the cost-per-tflop of that sort of system. These guys don't require as much cooling, space, or whatever else you care to think about.

Has anyone tried stuffing several into a single 1U chassis? For a sort of cluster of clusters?

Re:Seriously, though...

[drinkypoo]

You could get (maybe) 2-4 boards into a deep 1U box. It would be better to use a ~6U box and put lots of them on their sides. You could make a 12" deep 6U with probably 18 or so of these things in it, without having to have cables coming out the front AND back of each box.

Cool stuff ...

[Lazy Jones]

This rocks - we were considering something similar for our clustering-R&D needs (for trying out new network file systems, failover solutions etc.), but we decided to go with plain P4 barebones instead. They can be stacked nicely, are relatively quiet and the fast CPUs with HT come in handy when you want good latencies at CPU-intensive tasks (dynamic websites etc.).

Here's a picture of our first 4 boxes. The USB stick seen sticking out from one of the boxes is bootable and an excellent replacement for floppy disks...

Whilst not clustering...

[Alioth]

Whilst not clustering, a good use for these low power systems would be for web hosts or budget dedicated servers. I'm sure a server room full of these would require much less airconditioning (and power) than the typical servers. Many people require dedicated servers for security (they are the only one on the box) and don't require fast FPU performance.

Re:FLASH...

[technomancerX]

"He's using flash as the HD for the computing nodes"

Actually, he's not. IBM Micro Drives are not CF, they just have a CF form factor/interface to be compatible with hand held devices. They are hard drives.

Re:Inexpensive for testing purposes,

[addaon]

I agree, but that's actually a very interesting use. It also lets you play around with network topologies, and interconnects, and such. And of course, these boards do have one PCI slot, as well as the standard assortment of serial and parallel, so the hardware people can have fun too. For real number crunching? Not a chance. For doing a $2000 prototype, in 15 nodes, of a $50000 50-node cluster? I can't really think of a more flexible, more convenient, or more affordable option. For doing a $1000, 6-node flexible network simulator, purely for education? Also more than worth it, with few other options around.

Sounds Fun

[RAMMS EIN]

I have been thinking about this lately. I get disgusted by the fanns everywhere (especially since the one in my laptop makes an awful amount of noise sometimes and still doesn't prevent the beast from overheating and shutting down). Aside from being noisy, computers have way more CPU power than I need, and cost more than I am willing to spend. And they suck up a lot of power. (Some might add that they take a lot of space.)

I think all of these could be solved at once. What if someone built low-power, low-noise, and low-cost computer, good enough for running light office applications? I don't mean OpenOffice, but rather lightweight programs that implement the functionality people use _without_ the bloat. My 486 handles email just fine and the WYSIWYG word processors were once satisfied with a first-generation Pentium (and even these were already bloated).

Current PDAs have more than enough processing power to handle those tasks, and I've noticed that company's like gumstix build and sell devices almost like what I have in mind (the gumstix don't seem to have display connectors, though). Hey, these machines could actually be portable and have a really decent battery life (more than a full working day); that would be a killer!

Am I just daydreaming here or are others with me? Maybe you know of devices that do this job? Someone recommended Sharp's Zaurus, which is excellent, but still rather more expensive than what I have in mind.

Why this particular set of software / booting?

[merlin_jim]

I mean, those IBM 340 MB microdrives aren't really that cheap... you can get full size hard drives for the same price...

I've always wondered; why not PXE boot something like this? Set your node controller to also do DHCP and you're set.

While you're at it, use the CL version for the controller which has two network cards and build a NATTING firewall into the node controller too. Then you have a plug-in appliance that doesn't interfere with your network topology at all. PXE boot it and the motherboards will only need RAM.

The board he used is available for $99 with proc. A stick of 256 is probably around $20.

The best price froogle would give me on the drives he's using is $60, and they're prone to wear and tear.

Add in the $10 CF-IDE adapter and the drive is %60 of the cost of the motherboard itself...

Hell if you don't want the network bogged down with a bunch of PXE booting nodes all the time, just get cheap CD drives and put dyne:bolic on it, which does automagic clustering...

Personally, if I were to do it, I'd set dynebolic to PXE boot, get a huge stack of motherboards and RAM, and do it that way. Then adding/changing nodes is relatively simple... IIRC, they're even factory set to try PXE booting if no IDE devices are found...

The only other change I would make would be to ditch the 16-port switch... move to 4-ports, connect those to a 4-port with gigabit uplink, and connect that to a gigabit switch. Of course at this point I'm talking about really scaling the cluster up, to a few hundred nodes or so. At that point I'd stop using a mini-ITX board for my node controller and go with a motherboard with a bit more juice behind it, dual procs, RAID 0/1, the whole shebang...

Now if only I had a couple grand burning a hole in my pocket... speaking of which:

motherboard: $100
RAM: $20
DC-DC converter: $30
CF adapter: $10
Microdrive: $60

Total: $220
Total PXE booter: $150
Savings: 30%

So, not counting the costs of cabinets, power rectifier/UPS, wiring, network gear, and labor, you can increase the size of your cluster by %30 for the same cost, just for setting up PXE boot...