Slashdot Mirror
Beowulf For Dummies?
Pheno writes: "This looks like a fun LUG project. A simple setup for a Linux cluster called OSCAR from the Open Cluster Group. The people behind it are Oak Ridge National Labs and the National Center for Supercomputing Applications and some private companies. According to this Newsforge (part of the Keiretsu) story their 'Supercomputer on a CD' software is supposed to make it so easy to put a Beowulf cluster together a high school student or MCSE can do it in a few hours."
"high school student or MCSE" So.. EVERY MCSE = a high school student. Wow.
Ok so it should say:
"Any retard could do it"
There. Happy now?
Since the average Linux user is not capable of setting up a Windows 2000 network in a few hours, this proves that Windows users are smarter than Linux users.
since the average windows 2000 user is not capable of setting up a windows 2000 network in a few hours either, this proves that Windows users are not smarter than Linux users.
Zetetic
Seeking; proceeding by inquiry.
Elench
A specious but fallacious argument; a sophism.
If you are familiar with some distro already, go ahead and install that. Or you can just install RedHat (6.2 would be better). Minimally, you will have to select the base packages, no need for the compiler or most devel libs (or sources).
/usr/bin/make to this directory tar and bzip2 the directory (or just gzip). Move this directory over to the webserver, tar -Ixvf (or tar -zxvf). cd $srcdir;cp make /usr/bin;su -c make install;rm -f /usr/bin/make.
I do recommend installing ncftp though.
Basically, unselect all and install in GUI mode on one machine (better help in the GUI). Note what packages you are installing, and install them on your webserver. (except the X packages, of course).
Then go to a Redhat mirror and download your updates: bash, csh (ksh also if you installed it), libc, kernel, apache (bind if you are installing a nameserver also, even if just a caching nameserver). Also download ssh and openssl rpms (and the rpm packages from there -- openssl depends on the rpm-devel rpm.).
If you need to install php, download those updates as well).
rpm -U *.rpm in the downloads directory.
Then just init 1, init 3 and you are done [Just to be safe].
This should take about one hour, for a reasonably fast download line. (I have done it in 35 minutes, but I have some experience).
Its not hard to setup a webserver, but making it a secure machine is a lot tougher.
Also, if you need to setup a ftp server, do *not* use wu-ftpd. I personally use proftpd from CVS, and if youwant to try it out, the OpenBSD ftp port also has been recommended.
Set up another machine with compilers and libraries on it, so you can compile stuff.
Here is how you can install stuff that you have to compile:
download the source on the development machine.
./configure;make
This gives you the necessary binaries in the directory (or sub directories) where the source was compiled. Copy
Remove the installation binary directory, and you are done.
I can throw myself at the ground, and miss.
Usually the people who got MCSE certification and actually know what they're doing (i.e. can administer a multiuser NT or UNIX machine with ease) don't go around bragging their credentials, because they've realized how little the MCSE certification has come to mean, and how much more important the other stuff is.
--TheOrangeSquid Is it any wonder things seem so awry? We swim in a sea of confusion and don't have to think to survive
So much for doing calculus homework all day. Reimann, eat your heart out.
I am !amused.
I never wanted to spend the time learning how to set up a [whatever] manually...
Now I wouldn't consider myself a master programmer or sysadmin. I can do very basic C/C++ programming and I can get a basic idea of what some code is doing if I study it for a while. I've also set up things like Samba/NFS/ipmasq and some other things. However, I do NOT consider myself capable of installing them. Why? Because I've only installed packages and commented out some lines in config files. Most of the network services in my distro of choice, Slackware, are setup by default. I didn't do anything to get them to work. The creators of the packages did most of the hard work. Even the source I have to compile is infinitely easier with autoconfig (./configure, make, make install occasionally some more work is involved). At most, I follow some of their examples in the config files.
Now, I don't want to spend the time to learn how to setup a machine from scratch, either. However, I think that, in the end, the struggle will pay off. That reminds me of my physics professor from the last physics class I took. One of the things he said on his syllabus was that the exams would be more focused around the struggle to solve the problem than the algebra. In this case, the struggle around how to set up a cluster and what to think about would be the major learning experience. You can learn how to install Linux and build a computer without a cluster.
So, this may make the task easier, but I would not recommend it to someone who is interested in learning. Well, it could be nice just for a short project. But if you want to say that you know how to set up a Beowulf cluster, I don't think this sounds like a good idea. Most of the knowledge you'd learn from building a cluster would arise from the things you'd have to struggle with (MPI/PVM coding, for instance).
It's probably been said before by some famous person, but I'll recall the saying that comes to mind: He who makes no mistakes learns nothing. (If this project makes cluster-building as foolproof as it sounds, it will only be beneficial to those who've already set up a cluster and know what's giong on.)
Withdrawal before climax is very ineffective and those who try this are usually called "parents."
'Supercomputer on a CD' software is supposed to make it so easy to put a Beowulf cluster together
I've wanted to build a cluster for a long time. I was given that chance at school & work. School(U of C) allowed me to do it as a term project that is still ongoing. Work allowed me to use a pile of spare machines that were waiting for new users.
So, early one Saturday morning, I sit down with the O'Reilley book "Building Linux Clusters" and the CD that came with it. I followed the instructions in the book, and was frustrated beyond belief. The CD contained all the Beowulf software that was required. The downside was that the software had all been thrown over top of Red Hat 6.2. Being the second time I had ever installed Red Hat, I wasn't sure what the magic sequence was to get everything to work.
I first had problems with unsupported video cards... I tried 3 different cards. Each time, I needed to re-install Red Hat. Why wouldn't it let me install all the drivers for all the cards?
DHCP? Why? You only need to set the IPs once. Don't force me to do this... oh wait, I don't know Red Hat's weird config script structure... sigh.
The book mentioned nothing about re-compiling the kernel. But, in order to add support for the network cards, this is what I needed to do. Oh wait, where are the kernel sources?
This is when I got sick of this "wonderful" Beowulf CD.
I went to Slackware 7.1.
Installed it on the master no problem. Enabled frame buffer support for the video card so that it would work on ANY video card. Enabled native support for the network cards I was using.
Next step, I went to THE beowulf site, did a search for PVM and PVMPOV. I downloaded all the source code I needed.
Now, without the help of the book, I was at a bit of a loss. Luckily, there was this site that explained EVERY STEP in about a page and a half. The how-to was written by Christopher Johnson and I must say, he did an excellent job. I found only one thing that was lacking, you may also need to set PVM_DPATH=/home/pvm3/lib/pvmd in your profile.
Now the purpose behind all of this was to get PVMPOV running, well, with a little searching, I was able to find everything I needed here.
Conclusions:
Use a Linux distro you are used to.
Get a book if you want to know the theory.
Always remember that some PVM Books are free.
I hope this will help someone out there.
Beware TPB
don't kill yourself.
go see the movie "chocolat" instead. it's pretty good.
then take a music or acting class or something. many people with strong emotions/feelings do well at things like that.
it might be fun!!
Treatment, not tyranny. End the drug war and free our American POWs.
See my user info for links.
That's not a fair comment, that a "high school student or MCSE could do it". If someone is trained primarily, or exclusively, on one OS, what makes you think it would be easy for them to set up a cluster on a foreign OS? That's like asking Linus Torvalds to set up a Win2K cluster; it's just not fair to the respective persons talents.
Clustering is complex, and the design decisions behind implementing one are complex, regardless of architecture or OS issues. When you consider how few people in the world even ever have the proper hardware to implement clustering (and don't preach about cheap PC's and Linux as an easy entrance..shared storage in a clustered environment is hard & expensive to do at home), it becomes totally unnecessary to take potshots at where/how they obtained their training.
Apparrently after reading the comments this article is really about how to bash MCSE's for Dummies or Trolling for Dummies... ie. all the dummies that have made statements like:
/. readers would be a little more openminded. Instead of assuming every tech with a cert is an idiot. Sure many are, but many aren't. I've also met plenty of college graduates that are dumber than rocks, yet I don't associate that with all.
/. would also have enough journalistic integrity to refrain from making such cheap shots. Wonder how many MCSE's are avid linux users also that read slashdot. I know some- MCSE's in the local lug. Surely they aren't idiots if they use linux... then again how would you know if you have never spoken to them?
"I know it's probably a bit too much to expect journalistic integrity out of a site like Slashdot, but can you at least consider the possibility of perhaps posting at least one story that isn't chock-full of anti Microsoft/MCSE bigotry? In case you've forgotten, Slashdot has a huge readership, and you're likely to alienate a large percentage of that readership (and by extension, a large percentage of your banner ad viewership) with statements like the one you make about MCSEs. Has anybody ever explained the meaning of the word "objectivity" to you?
Sure, it's true that Microsoft includes a handkerchief with MCSE certificates so that graduates can wipe the drool off the corners of their mouths. Sure, MCSEs sometimes put their shoes on the wrong feet. Sure, they sometimes have difficulty remembering their middle names. But does any of that justify the sort of blind prejudice that you display in this article? The fact the the certificate is a useless piece of paper that will be obsolete in six months is no excuse for you to lash out at MCSEs the way that you have. "
Wow, what a good person it takes to stand up for generalizing people that way. Such people are surely deserving of the Noble Peace Award. It apparently also gives moderators the chance to mod trolls up +3 funny for MCSE dirt.
Saddly you would think that
It seems like
So now any dummy can slap together a cluster - what's he gonna do with it? Beowulfs aren't designed to run standard services - i.e. they won't make a good web server - they run specialized software, custom coded for each application. What _dummy_ is gonna write such an app?
Nevertheless, this is an incredibly interesting, and cool, idea....
Think outside the... Hey, where'd the friggin' box go?
I am in your debt.
Carousel is a lie!
My mantra.
- I don't care if they globalize against free speech. All my best free thoughts are done in my head.
You mean "a beowulf cluster of AC's".
- I don't care if they globalize against free speech. All my best free thoughts are done in my head.
Those MSCE's can be pretty thick. I'd recommend Beowulf for Dummies but they'd probably just end up at Beowulf for Dummies
Steven
-- I have marked myself unwilling to moderate-- I don't have other accounts to artificially inflate the karma of
Heck man,
I'm in the 10th grade, and our MCSE quit because he was so stupid.
Now we have some senior that thinks he know what he's doing, but just thinks its cool the fool around on the servers and mess stuff up. I don't know much about winNT, but I know it shouldn't be down more than once a week. I asked him if he had ever thought of using Linux or *BSD on the boxes, and he just stared at me like "Cletus the Slack-Jawed Yokel" and asked what that was?
We had some program for typing in the Typing Lab, called NewDeal, which sucked, I sugested Abiword or StarOffice, or switching to Linux and using KOffice. He just said he was happy with NewDeal, even though the Typing teacher was begging the guy to get something else. So basicaly he was afraid to get out of his range of knowledge.
Anywho, I have access to a pentiom 120, and some 486's, how hard and how long do you think this would take to setup as a webserver?
I only wish an open mind would accompany an open source.
Dan
As if we needed proof.
--
Steely Glint
No, you don't beowulf cluster web servers. That may involve load-balanced clusters or failover clusters but not beowulf clusters. Beowulf clustering is for distributable computationally intensive jobs like physical modelling.
"moo" - cow 3, 1906
a beowulf cluster of dummies?
...a high school student or MCSE can do it in a few hours.
You expect too much of MCSEs.
I know it's probably a bit too much to expect journalistic integrity out of a site like Slashdot, but can you at least consider the possibility of perhaps posting at least one story that isn't chock-full of anti Microsoft/MCSE bigotry? In case you've forgotten, Slashdot has a huge readership, and you're likely to alienate a large percentage of that readership (and by extension, a large percentage of your banner ad viewership) with statements like the one you make about MCSEs. Has anybody ever explained the meaning of the word "objectivity" to you?
Sure, it's true that Microsoft includes a handkerchief with MCSE certificates so that graduates can wipe the drool off the corners of their mouths. Sure, MCSEs sometimes put their shoes on the wrong feet. Sure, they sometimes have difficulty remembering their middle names. But does any of that justify the sort of blind prejudice that you display in this article? The fact the the certificate is a useless piece of paper that will be obsolete in six months is no excuse for you to lash out at MCSEs the way that you have.
Shame on you.
We're going down, in a spiral to the ground
Suffield Academy students, myself included (once upon a time, that is), attempted this about two years ago with PROJECT:EFFECT (sadly, the website is now-defunct). The goal of the project was to demonstrate that clustered supercomputing was an extremely viable, which we tried to prove by assembling a cluster of our own - from donated parts!
Anyhow, enough of me rolling in the past; just wanted to be an old man for a sec and yell "It's been done before, ya punks!!".
root_dev_X
bear with me, the sig is old....
===== Warble://VX
I, as a high school student, this comparison leaves me deeply insulted by this jab at our collective intelligence. This is an unfair and cheap shot at us, and as a 10th grader, I demand an apology from the Slashdot crew! Hell, the MCSE at my school was panicking after tech support for the firewall-in-a-box got him to cut off the web cam from outside access. Five minutes and a few remote logins to the firewall and webcam later, I had it all fixed up. Gimme a break.
Friends don't let friends use multiple inheritance.
The smartest one I ever knew was a "Network Engineer" (his official designation from the technical school he graduated from.) Here's a story about him:
I was a System Administrator for a small (but growing) company - we had approx. 15 computers, most of them running Windows 3.11 (this was a few years ago..)
He came to me one day, said he was upgrading one of the stations to Windows95, and asked what the IP address for the station in shipping/receiving was.. so I told him. He came back a few minutes later, and said that the IP I gave him was wrong, because Windows said that the IP address was in use.. I was in the middle of something, so I told him I'd come see him in 10 minutes.
So I'm on my way to see him, I walk through the shipping area, and the computer is there, and someone is using it, so I figure he's figured out the problem he had with the address..
An hour later, he comes up to me and asks if I'd forgotten about him.. I told him that I saw the station running..
He says "Yes, but I'm gonna replace it, I have the new machine all set up on the network, but when I put in it's IP address, it doesn't work because it says another machine is using that address."
To which I reply "Of course it doesn't work, the old station is still running!"
His reply: "Oh, is that important?"
MCSE's are generally people who aren't smart enough to get a job with other skills.
...Beowulf for Dummies.
- I don't care if they globalize against free speech. All my best free thoughts are done in my head.
. . . . . . Confessions of an MCSE . . . . . . .
Once upon a midnight quite particularly dreary
Since I chanced upon a net node that illuminated clearly
How to scrounge and hack together former CPU's that yearly
lose all function and so merely
waste the space upon your floor,
Since I chanced upon this web site that had promised so much more,
for the price of a For Dummy's book and a drive to the all-night store,
(for we all know that King Amazon ships slowly from off-shore),
then I swallowed hard, and knowing that what sleep I'd had before
would have to do for a day or two, as I readied to explore
this strange book, why then, astonished, on the cover this I saw:
How too bild a Beowulf clust-or
I dropped my jaw. What's the meaning of a spelling that so liberally spews
anti-publishing convention, anti-literally views
as early as the title-page?! With what shall I excuse
such an idiotic purchase?....or is it but a rouse
through the value of its shocking to get you to peruse
its contents? Who can know? I turned the page.
A CD-ROM! How helpful! I checked my former rage.
Reading on, I noticed these directions for the disk:
"Put it in, and execute drive-letter-setup-exec.
But do so at your personal and solitary risk."
That is all.
Alright, I said, that's clear enough, and further then I read.
but instead of more instructions, there was the same, in Spanish instead.
On I flipped, through German, French, and Chinese simplified
on through Russian, Japanese and Netherlandy Dutch beside,
on I flipped through all the Slavics, nay, all Indo-European
languages, then other groups, like Ugro-finnish Hungarean,
On through japanese, swahili, even African Tschadide
as escape I vainly tried,
but there was no end, until, at last, I reached: About the Authors.
So was this some get-rich scam meant but to fill the writers' coffers?
No returns on midnight offers.
Well, I sighed and looked again at that mysterious cd
I guess it could not hurt just to browse it for to see
whether any use in it at all there actually can be.
So I popped it in the drive-door,
browsed on over, and, well, gee:
One file was all there was: setup-dot-e-x-e.
Six hundred megs. I switched from root
to users more restricted
I went and clicked the execute
clutter-clutter, on it shifted!
my cd-rom, up,up it sped
my hard-drive whirled, and flashed its red
my system cluttered, clunked, and crappled,
as with this "huge" file it grappled
(what, you really don't suppose,
that the system slowness these expose,
isn't natural with the computer speed woes
that only clustering overthrows?)
So it thrashed and scuttled, a quarter hour,
When with an effort it found the power
to treat this MCSE dummy to an explanation
of why he could not yet achieve his clustering elation:
"OS not supported"
It dutifully reported.
Now it doesn't take my well-earned MCSE
To realize that Linux is bullshitting me
Windows 2000, you see,
Is our hallmark, it supports damn near every
worthwhile application:
And this then was my salvation.
At 4 AM I concluded
The merits of clustering had been refuted.
Our distribution is considerably more sophisticated than what OSCAR is attempting to do. The advances have lead to a simpler system to install and run.
Our distribution CD may be used as an install disk for the cluster master, or after 20 seconds it automatically boots the machine as a cluster slave and tries to contact a master. Once the master has been installed (about 20 minutes), each slave takes only a few seconds to join the cluster.
Look for our demos at LinuxWorld Expo next week, and our latest distribution is now available for $2 from LinuxCentral.com. You don't have to wait weeks or months for a less capable OSCAR system.