FreeBSD Cluster At Purdue

luddite writes: "Two guys at Purdue University have assmbled a FreeBSD based cluster built cheap - very cheap. With under $2500 spent on the cluster, it's one sweet set-up. Just shows that if you take the time and put some effort into something, money doesn't have to limit your resources! The site also goes into some detail about what the cluster is made of, where they found the parts, how it's been configured, and what they plan to use it for."

Comments

Re:Motherboards not shown in budget

[Anonymous Coward]

Every single mother board is an old Asus P55T2P4 modified to accept a K6-2 400. Most boards came out of machines being upgraded and therefore had zero cost. Some were also purchased at swap meets for little cost. The memory also what was left over from upgrades and available at no cost.

Why not BSD?

[emil]

Or why not Linux? A couple of reasons each way:

• There is still a lot of hardware that BSD doesn't support. In December, I had to migrate my BSD email server to a Compaq system with the Compaq SMART RAID controller. BSD didn't support the controller at the time, so the decision was a no-brainer. The Linux "hype" does get real vendor participation, which translates into a larger suite of higher-quality device drivers.
• There is the issue of how you feel on the whole GPL versus BSD license thing.
• OpenBSD is incredibly slow in implementing the latest standards. They are still on Bind 4 (and yes, I know there was a root vulnerability, but everybody else is using Bind 8).

The BSD people are great, and Linux owes a lot to them. BSD continues to make great contributions to the world of Linux. It would be the best of all possible worlds if each had the same capabilities. But, because of the hype factor and the real development it brings, BSD has no hope of being as flexible as Linux in the near future.

I guess it is a question of what you grew into, the level of risk you are willing to tolerate, and the hardware that you need to support. The decision of BSD or Linux starts there.

Re:Why not BSD?

[Duke of URL]

OpenBSD is slow including new features/software sets on purpose. Some OpenBSD developers have made remarks that if you want the latest feature "X", then OpenBSD may not be for you. They like to audit and examine things and make sure everything works and is secure first before including it. Some people may not like it, but that's their basic design philosophy.

But its really not a big deal (in my opinion). If you want Bind 8 on your OpenBSD box you can just install it.

My Slashdot Observation...
What really funny though, is how the same general questions show up on every BSD related article on slashdot. "Whats the difference between Linux and BSD, which is better...." Someone should just make a Slashdot FAQ for this and be done with it.

CPU Upgrades?

[emil]

These guys are buying AMD K6-2 3D 450 processors, which they say work in a variety of motherboards. Do these work in non-MMX (single voltage) motherboards?

I'm using an old Gateway P75 as a masquerade box for my cable modem. It would sure be nice to upgrade it to 450 on the cheap.

I am looking for the best way to squeeze a little more life out of this box.

Re:YABC (Yet Another Beowulf Cluster)

[BigD42]

Actually beowulf clusters commonly use eithe MPI or PVM. Neither is required to classify a cluster as a Beowulf. The first demonstrated Beowulfs were by NASA and ran on the PVM libraries. Refer to the Beowulf into http://www.beowulf.org/intro.html

will the clustering extensions work on !x86?

[option8]

let me see. free/netbsd runs on 68k machines.

macbsd (netbsd/mac68k) runs on my LCII.

LCIIs can be had at the local surplus auction for $5 apiece. Most of these are formerly lab machines, and have ethernet already.

i think a cluster of 25 of these low profile 16mhz monsters could fit on a desk, _maybe_ put out the MIPS of a PII (and only about twice the heat :) and all for about the price (including cabling & hubs) of a cheap celeron machine.

hmm.. for $10 i can get powermac 6100s by the truckload, and freebsd/ppc...

Re:will the clustering extensions work on !x86?

[JatTDB]

Just to clarify, FreeBSD does not currently support 68k. NetBSD and OpenBSD do, though.

Re:I believe...

[theCoder]

So THAT'S where all the money we pay for Ethernet in the dorms goes...

Re:FreeBSD v. Linux

[toolj23]

There is a date in there from 1997. Sounds like a lot of that stuff is old and outdated. Linux has come a long way since 1997.

Re:Linux 8 Node Cluster for $250 @ Purdue

[Multics]

And what did you do with your $250 cluster? You could have joined SETI or gone after a few prime numbers or any of a zillion other cool things.

The point of ACME is to solve a few very hard (yes, NP-hard) problems. We don't particularly care the form of ACME in the end as long as we can solve the problems at hand.

One of my gripes about the typical /.er is that they are in love with the technology and not with *doing* things with the technology. ACME is about doing, not screwing around. Our intention is to produce papers that used the results from ACME, and not to do papers about ACME. Being /.ed was a pleasant surprise.

Re:Purpose?

[be-fan]

44X CDROM drives in a cluster system? What are you smoking? Plus, why do cluster systems need high end graphics cards? I don't know of any GL implementations that can be clustered. (Get a WildCat 4200 for the main machine and be done with it.)

FreeBSD v. Linux

[linuxonceleron]

This is cool that they were able to make a cluster for so cheap, but what advantages does this have over a beowulf cluster of Linux boxes? Also, in what areas is FreeBSD superiour to linux outside of clustering?

Re:FreeBSD v. Linux

[QBasic_Dude]

Some excerpts from the FreeBSD vs. Linux page at http://www.futuresouth.com/~fullermd/freebsd/bsdvl in.html page:

Subject: Re: Why FreeBSD?

Any response to a question like this is bound to upset someone. I'll
answer with the caveat that this is my opinion that developed over the
past three years following them both as well as other commercial OSs.
Those of you offended in any way by this, please cat flames > /dev/null.

That said -- the differences between FreeBSD and Linux can best be
understood in the context of American politics. There are essentially two
philosophies: Republican (FreeBSD) and Democrat (Linux).

The FreeBSD organization is a republican structure -- we have our say as
users, but the final decisions devolve to the core team who take the final
responsibility for their decisions. FreeBSD takes a conservative approach.
In other words, better things should work correctly at the expense of a
minorities desires, than to please all of the people all of the time and
have unexpected components of the OS breaking on a regular basis. We are
free to vote our approval or disapproval by changing our OS.

Linux is a democratic group. There is no single authority to accept final
responsibility except for Linus as it relates to the kernel. Linux adopted
early on a consensus approach (POSIX, etc.). In a sense, Linux is much
like current Democratic politics -- the mob pretty much rules. The end
result is that there is really no such thing as Linux -- there are
distributions that use the Linux kernel and from then on you have
essentially different operating systems. Slackware, for example, doesn't
look at all like Red Hat. Describing Linux is much like describing Mach.
(There isn't much - both are just micro kernels. _Anything_ can be
implemented over them.)

So as I see it, it comes down to this: vote for the philosophy that
appeals to you. I use FreeBSD because I rely on my machine for many other
uses besides tinkering with operating systems. FreeBSD doesn't change the
world on me every 6 months. Linux is in constant change. New things are
showing up all the time. If you like tinkering with operating systems and
having things that used to work break, Linux may be your answer. If you
don't know Unix -- pick one and get started. You'll learn how to pick the
best choice. No matter which one you pick, it will be infinitely better
that Micros**t anything.

Enjoy.

-- Jay

----------
Subject: Re: Why FreeBSD?

And the clouds parted on 21 Mar 97, and Jeff Roberts
said:

>On Fri, 21 Mar 1997, Bob Dole wrote:
>
>> Hi, I plan on changing to UNIX and I wonder wether I should take Linux or
>> FreeBSD...
>> Both seem to be an excellent choice, so you can't say one is better than
>> the other. But in what are they different, in what is each specialized?
>

Then try them both: they're both "free", but you'll have to pay something
for you Internet connection or CDROM distribution, depending on your
circumstances. The following is not impartial, as I don't play with Linux
much, but when I did I wasn't as happy as I am now 8).

[opinions on]

Linux is SysV-flavored (barely); FreeBSD is BSD-flavored (definitely).

Linux's kernel is authored by one person (Linus Torvalds); FreeBSD is
authored by (essentially) the core team.

Linux addons come from pretty much everywhere; FreeBSD's get submitted from
a lot of places also, but have to pass review to be included as part of the
release.

Linux has multiple releases (based on who's packaging), all somewhat
different from each other, and somewhat inoperative as well. There's only
one release to FreeBSD (per major version)

Linux tends to be more cutting-edge and trendy, and tends to work with more
hardware (to some degree), partly because of the "arrangements" made with
vendors. FreeBSD requires that source code be freely obtainable for
(nearly?) all it's parts, which scares some vendors into not cooperating,
or at least not as well. The hardware that _is_ supported tends to be done
pretty robustly.

Linux is snappier for low-user-count systems, depending on what you're
trying to do. FreeBSD tends to shine under real load (like WWW/FTP
servers), and I don't really know if any major sites base such Internet
services on Linux; quite a few seem to be using FreeBSD, particularly
Walnut Creek CDROM, which carries quite a load on a consistent basis.

There are far more books on Linux than FreeBSD per se, something I draw no
conclusions on.

The support on the Linux list(s) is something I haven't any personal
experience with; the support on the FreeBSD lists is exemplary.

[opinions off]

Please correct any sins of commission and ommission you find above; I don't
do this often enough to be any good at it.

your mileage may vary, and best wishes,
larry

--------
Subject: linux vs freebsd testimonial

A few weeks ago, my single linux box fried. I replaced both the hard
drive (with an identicle one) and linux with freebsd 2.1.5.

The machine runs majordomo, ftp, apache, and an irc server.

The performance is way up there! Under linux it would frequently slug
down to a crawl. under FreeBSD it just keeps zipping along.

There is a very definite noticable difference in response and load
handling.
-------
ubject: Re: linux vs freebsd testimonial

Since we've gotten along fairly well in our migration from Linux, I
thought I'd share my experiences as well...

We currently run on FBSD:

1 Shell/user www (was NetBSD 1.0)
1 DNS/mail/dialup auth/syslog (these two are sharing mail spool over NFS)
1 utility/backup/freebies
1 DNS/mail for seperate, wacky project
2 virt www servers (3 are still Linux)
2 co-locate www servers (www.firstview.com 3.6G/day, www.villagevoice.com)

These have been the most trouble-free machines we've worked with. Some of
the recent security problems were a bit tough (lots of cvsup-ing), but
nothing compared to the nasty Slackware Linux Bug-o-the-month. The only
reboots *any* of these machines have seen were intentional, which is
something I just can't say about Linux. Performance is much better, and
the "out of box" configuration is a lot more sensible than Slackware. A
few of these machines really get beat on hard, and they just ask for more.

We have to keep one Linux web server for compatibility with some odd
sourceless C cgi's, but the other two will be history soon. Our news
server is running Linux, but it's being replaced with a machine to be
named "fridge" which will have 3 SCSI busses and 15 drives, and of course
be running FBSD.

I must say, this has made my job much easier. Linux is just too
unpredictable when you don't have the time to play the
"kernel-of-the-week" game. One of the Linux boxes still does the routine
of freezing with no log entries or other hints; which is extremely
frustrating. FBSD just seems like it was meant to be in a production
environment...

Thanks to all involved,
-------

Re:I believe...

[b_pretender]

Hang on there, dude.

Read more of the article first...
From the news page:

6/2/00 - PUCC and the University announce the purchase of a $10,000,000 IBM SP/2 (272 processors) for general purpose scientific computing. It will complement (nicely) the smaller (?32? node) SP/2 they've already got. This comes after a long period where PUCC had little to offer in BIG computing. "Hey PUCC! Wanna race? Our MIPS/$ versus yours?" We look forward to running some jobs over there. It should be in production before the start of the school year in August 2000.

I think that the tide still says that the power is in the almighty dollar$$$$

--

Re:Clustered BSD - with added Mach and Aqua?

[b_pretender]

There's a problem with that...
Apple uses proprietary licensed stuff in many of the features of Aqua, so it couldn't be open-sourced. I'm sure a watered down version of Aqua could be created, but lacking the PDF windows and openGL programming, it'd be no more then the Aqua theme which I use with gnome.

--

Missing components?

[Some Strange Guy]

Page says this is what they've paid so far. From the looks of that list, they've got a ways to go.

Interesting start, but they've still got memory, motherboards, and some other stuff to go. That's going to crank the price tag at least a couple thousand dollars...

Motherboards not shown in budget

[bak6926]

Look through the budget, and there's no entry for motherboards. That is unless you can get a AMD K62 450 + mobo for 64 dollars.

Cost is misleading

[rotten_]

From what I can tell the price (~$2500) doesn't include the motherboards or memory which they obtained from various sources. This is probably one of the most significant outlays they had to make, next to the processors.

It is still really a great price, and I can't believe what they paid for the racks.

-k

University Advantage.

[Dungeon Dweller]

Ok, look at the budget they laid out. Yes, a university with a significant excess of computers can do this cheaply. That's the argument that many people are using trying to get MY university to build one. LOOK AT WHAT THEY ARE PAYING FOR EQUIPMENT. If I got CASES at $1 a piece, and NIC's for the same price, and RACKS for $10. Then I could build just about whatever you want for under $3Grand myself...

Re:University Advantage.

[bko]

First, realize that the machines they are talking about are Pentium (not PII/III) class machines. Thus, they require SIMMs, not DIMM memory. This is why they're using K6/2-500s and the like--they run in Socket 7 mainboards (albeit only at 66MHz memory bus). Memory like this is available quite cheaply.

Second, good quality motherboards are basically there for the salvage -- on the news page they mention a an ASUS P55T2P4, which is, I believe, a 430HX board. But there's a big integer compute difference when it's outfitted w/ a 500MHz K6/2 vs the (likely) P133 that used to be sitting there.

Thirdly, they mention that the machines are outfitted with at least 32MB of memory. This is not 128MB. You don't need 128MB to do a lot of tasks on either *BSD or Linux -- as long as you're doing things that have a <32MB resident set, you're going to be fine on either. FreeBSD is particularly good in low memory situations (its swap performance is better than Linux in my experience), but i'm pretty sure that this isn't important, because they are looking for big integer performance first and foremost. Otherwise, they're probably better served with fewer, faster nodes w/ K7s or PII/IIIs.

Note that the machines have local HDs, so they can do local swap -- they don't need to keep shells, etc. swapped in over a network drive, either.

So this sounds good, for the right task. there are obviously a bunch of tasks that would be better served by other styles of clusters, or other resource allocations, but for doing fast integer calculation on the super-cheap, this is a great way to go.

Re:University Advantage

[connorbd]

Nononononono. The Ethernet is used only on the fighters. Anyone can clearly see that the fighter umbilical is FDDI. (And of course the mother ship runs Windows -- don't you know that the reason Jeff Goldblum was using a Mac (apart from the fact that they own him anyway) was that the aliens couldn't backcrack him? Go ahead, moderate me down; my karma is a bit swollen anyway, and this bit wasn't half as funny as I'd hoped... /Brian

1, 2, 3...15, 22.

[dmccarty]

ACME currently consists of sixteen Pentium-class computers, [...]

Is it just me?--or does anyone else see 22 computers in that rack.

--

Re:1, 2, 3...15, 22.

[luddite_geek]

You see 22 cases. 16 in the cluster, the two controlling machines, and EMPTY cases. Misleading, I know. They'll be filling them in later. The top 5 machines in the nearest 3 columns, and the top one in the 4th column, are all currently in the cluster. The bottom machines in the two nearest columns are the controlling machines. The bottom four in the 4th column are not connected to the cluster yet. They also plan on adding another rack with 5 more machines for a total of 25 machines. (The cases can be seen sitting in a stack at the left side of the racks)

Hot Setup

[PopeAlien]

Boy.. I bet that thing gets hot..

ACME currently consists of sixteen Pentium class computers, each with 450 MHz AMD K6/2 processors and at least 32 MB of memory.

I have two 450 Mhz Amd boxes in a small room, and they sure pump the heat up there..
-

YABC (Yet Another Beowulf Cluster)

[BigD42]

Despite the synicism of the subject, its nice to see a beowulf cluster on Slashdot which doesn't use Linux. It helps to remind people that Beowulf != Cluster of linux boxes. After all the definition of a beowulf is
It's a kind of high-performance massively parallel computer built primarily out of commodity hardware components, running a free-software operating system like Linux or FreeBSD, interconnected by a private high-speed network.

http://www.dnaco.net/~kragen/beowulf-faq.txt