ClusterKnoppix

chronicon writes "Knoppix is the ultimate live CD. No geek-kit should be without it. Now Wim Vandersmissen has taken it a step futher by adding openMosix functionality. Drop the clusterKnoppix CD in your "server", boot up... boot up some networked clients... Knoppix built in LTSP magic kicks in and ta-da--instant cluster!"

Comments

Imagine a

[Anonymous Coward]

... forget it.

Re:Imagine a

[jjeffries]

I thought you were taking the John Lennon approach there for a minute...

Instant cluster's gonna get you
Gonna knock you right on the head...

hehe

Re:Imagine a

[Jeremiah Cornelius]

Why do I get the feeling that Knoppix is going to save the world someday?

Every day I find another use for Knoppix. I've re-built my own - with non re-distributable software, like X-Over plugins.

Imagine a Beowulf cluster!!

[stanmann]

Ok, Kidding. I'm actually quite impressed with the wide support Knoppix provides for hardware and functionality. 5 years ago, the network computer theory was being trundled out, AGAIN. Now we have the capability for a truly functional dumb terminal/server configuration and it will run on any commodity hardware/software higher than a 486DX(allegedly). It ran well on my oddball Celeron 300 with a 640x480 monitor, although right now that is my only complaint with the various implementations of X...

Re:Imagine a Beowulf cluster!!

[Gill Bates]

You can tell knoppix your screen res at boot-up. F2 will give a list of boot options one of which is 'screen=[whatever your screen res is]'

e.g., knoppix screen=1600x1200

works like a charm

As if I needed another reason...

[mercan01]

I found out about OpenMosix recently, and I'd been looking for an excuse to test it out. This just makes it even easier.

I'm wondering how difficult it is to setup. Is it as easy as the poster made it sound?

Re:As if I needed another reason...

[1qa2ws3ed]

i didn't touch the keyboard. ten clicks with the mouse (i'm sure they where not more than ten...) and i was running a single system image cluster based on 3 diskless nodes... it is probably easyer to set it up than to explain what it can do. and it can do A LOT...

Re:Hehe Neat

[rkz]

This is going to be great for quickly encoding DIVX movies in computer labs ;)

Say this three times fast

[Tuxinatorium]

Can cretans create clusterfuck of Clusterknoppixes?

Heh

[arvindn]

The author's got some really funny images on his site.

mirrors needed!

[Anonymous Coward]

From here: mirrors are needed! please contact me

All I have to say is.. <voice actor="nelson">Ha ha!</voice>

Re:mirrors needed!

[Cond0r]

http://condor.madoka.be/clusterKNOPPIX_V3.2-2003-0 5-20-EN-cl1.iso.torrent

Re:mirrors needed!

[NTworks]

http://condor.madoka.be/clusterKNOPPIX_V3.2-2003-0 5-20-EN-cl1.iso

Interesting...

[Flabby Boohoo]

"Knoppix is the ultimate live CD"

That's what they said about "Peter Framptom Comes Live" too.

There can only be one ultimate!

clusterKnoppix Mirrors

[Anonymous Coward]

clusterKnoppix is in desperate need of mirrors. here's one (but i urge you all to make a .torrent or something):
http://www.openmosixview.com/clusterk noppix/

for a crappy yet less bloaty altenative, check out PlumpOS: http://plumpos.sourceforge.net/

Re:clusterKnoppix Mirrors

[Mr. McGibby]

How about a non-copyright-infringing make-your-own-torrent web site? I'm sure it wouldn't get the bandwidth load that the other torrent sites. Then random people on slashdot could actually just do this themselves and post a comment with a torrent link.

Re:clusterKnoppix Mirrors

[caino59]

that is a wonderfull idea.

bestoy this man with mod points...

(and go patent the idea while he checks his karma) ;o)

Re:clusterKnoppix Mirrors

[gladbach]

this site used to host bittorrents for slashdot stuff, but haven't seen anything there in a while... used to have 6 or so downloads from recent /. stuff.

http://207.44.142.96/

Re:clusterKnoppix Mirrors

[digidave]

What we really need is an HTTP torrent-like technology, so when you hit a site that's being slashdotted, you transparently download it from everybody else who's there.

That sure puts IBM's e-business on demand to shame.

Re:Awesome...

[stanmann]

Well, think of an environment where you have boxen sitting around unused part of the time, and want to be able to plug and unplug cluster components dynamically and not have any persistent data stored on the part time cluster members, Possibly even using them for windows and word processing during the day and cracking the xbox key at night.

Re:Hehe Neat

[rkz]

I wasnt trying to be funny I'm actually going to do this! It takes about 14 hours on a single windows machine right now (1Ghz AMDs). If I get 30 machines it should take little over half an hour to do a single movie. woo hoo

The name needs adjustment

[fudgefactor7]

"OpenMosixKnoppix didn't quite sound good, so I called it ClusterKnoppix ;)"

I would have chosen Kloppix...the "l" for cluster, the rest is self-explanitory.

Re:The name needs adjustment

[Repugnant_Shit]

And if your cluster gets a virus, you could call it Klappix

Minimum hardware?

[wlj]

What is the minimum hardware needed to support this? Obviously a NIC, but can it run diskless (no HDD or CD)?

That suddenly makes for a VERY cheap grid node. (Didn't want to use the "B" word :-)

Re:Minimum hardware?

[stanmann]

From Knoppix.org What are the minimum system requirements?

Intel-compatible CPU (i486 or later),
20 MB of RAM for text mode, at least 96 MB for graphics mode with KDE (at least 128 MB of RAM is recommended to use the various office products),
bootable CD-ROM drive,
or a boot floppy and standard CD-ROM (IDE/ATAPI or SCSI),
standard SVGA-compatible graphics card,
serial or PS/2 standard mouse or IMPS/2-compatible USB-mouse.
And for graphics mode, you need a monitor that will support 800x600 for reasonable performance. it will run at 640x480, but feels cramped.

bittorrent

[parkanoid]

Bittorrents going up in ~20 mintes, stand by.

Re:bittorrent

[Cond0r]

Right here: http://condor.madoka.be/clusterKNOPPIX_V3.2-2003-0 5-20-EN-cl1.iso.torrent

Re:bittorrent

[Latent IT]

Give up yet? =p

Applications I could run?

[gpinzone]

What applications can I run right away if I burn a bunch of these and boot up a few of the machines on my network? Do I have to configure IP addresses? Does it assume I have DHCP installed? Which Linux programs will automatically benefit from the cluster?

Re:Applications I could run?

[Anonymous Coward]

Not too many 'common' apps, but things like PVM PovRay, make, various CD ripping/video processing utilities. There is a list on OpenMOSIX.org

With the experimental DSM patches being developed, Apache even runs, but most things like databases and web servers generally don't because they depend on shared memory to work, and shared memory on a cluster is a difficult thing to provide if oyu want any kind of performance.

Re:Applications I could run?

[Bobzibub]

Open Mosix would allow the migration of regular processes, not specially written parallel programs which are required for Beowulf varieties. With Open Mosix you can run 50 instances of what-ever program.

I think it won't be long before we have an insta-Apache cluster, insta-mySQL cluster, insta redHat ecomm application cluster... I don't think Oracle's unbreakable claims hold a candle to this:

It is very exciting. (well I dig it!) But not simply because you can use your Mamma's Windoze network without her knowing: Say a bunch of machines in an e-commerce configuration insta-cluster, each with a couple gigs of ram and a cdrom (or even network boot). This makes it extremely unlikely that an unstable node wouldn't be able to get up on its feet again if it was rebooted or simply crashed. It is much more likely that your traditional installed OS will go AWOL and pooch your hard drive, making it unusable w/o a re-install. This time, every boot is a fresh install. Your data is the only thing that is written to storage. Maybe to a SAN via an mySQL cluster?

Also if a node is mis-behav'n then you can tell the processes to get lost (move onto other nodes) reboot the machine w/o worrying about the stability of the installation. Have your script reboot each node weekly in staggered times during low volume times just for the hell of it!

You could always use diskless nodes 'n stuff but this way it will be brought to the starving masses. Much less administration. Much more turnkey. Much more scary for Redmond based software companies (Lycoris excepted) because this sort of thing beats traditional point and click.
-b

OK, really dumb question here...

[gosand]

From the page:
"openMosix terminal server" - uses PXE, DHCP and tftp to boot linux clients via the network. No CDrom drive/harddisk/floppy needed for the clients

How do the clients work if no CDrom/HD/Floppy is needed? I am trying to wrap my brain around this one. I get the cluster server idea, but then does the server determine which clients on the network will boot into the cluster? Is it via DHCP? Doesn't there need to be *something* on the client side like a HD/floppy/CDrom so it can boot?

Re:OK, really dumb question here...

[SnowDeath]

Good reverend, you should search for PXE info - it is basically a chip on a network card that can basically "boot" to the network...it seaches for DHCP, BOOTP, etc and blah blah blah :)

Re:OK, really dumb question here...

[SharpFang]

The network card has just enough of simple BIOS to grab some necessary info from the net - first a simple bootup code, then the kernel, then mount the network drives... Everything kept in RAM. Few modern network cards support this, but if you see an empty chip socket on a network card, it's most probable it's place for the boot code EEPROM.

Re:OK, really dumb question here...

[jhealy1024]

PXE describes a method where the NIC in the computer bootstraps the information it needs to boot off of the network. Many modern computers have NICs that support this. Newworld Macintoshes can netboot, as can most recent 3com cards (even my 3 year old Dell supports it).

Basically, the NIC makes a DHCP (or BOOTP) request for an IP address. The DHCP protocol allows the server to return the address of a TFTP (Trivial FTP) server along with the IP address for the client. The client contacts the TFTP server to get a kernel (vmlinuz), and then boots directly into that. From there, the kernel should be configured to mount its filesystems over NFS, and finish the boot process. I'm sure Google can point you to a more complete explanation.

What makes ClusterKnoppix so cool is that it's usually a huge pain to set up a TFTP/DHCP/NFS server correctly for multiple clients. ClusterKnoppix does it all for you, so all you need are some (really) "dumb" clients and all the heavy lifting is done for you.

Re:Hehe Neat

[kcurrie]

Do yo have an encoding program that will actaully break up the encoding to work on multiple CPUs? If not, you'll gain nothing.

Mosix works like a big SMP box, no special code is required, so you just fork and forget.

It should speed up my SETI@home contributions

[silviuc]

I'm expecting big speedups for my SETI@home work. Now lemme get my hands on those ISOs...

Re:Hehe Neat

[rkz]

well I do actually.... GNU/Vidomi

really needs to be said

[b17bmbr]

this is truly remarkable and only could be done in an open source envronment. it is projects liek this that clearly show that it is only a matter of time before we look back and go "micro who?". forget the billions in the bank, the fud, the monopoly, etc., could they really do something like this? and when the cat is finally, really out of the bag, about the quality of F/OSS, it will be amazing.

IT/Clustering made cheap

[mmol_6453]

Linux Magazine just did a three-article bonanza on how Beowulf clusters came about, and the costs and issues involved with running one.

This could blow all of that away; Just insert the CD in all of the machines in your office, and let fly. Air conditioning? Already accounted for. Power consumption? Not much more than usual. Floorspace? Just a little under everyone's desk.

What I'd like to see would be companies switching over to all-Linux or mostly-Linux shops, running all their machines as an OpenMosix cluster. They could sell off their spare CPU cycles, quite easily. Ironically, IT's never been so cheap.

Re:hmmm. game servers?

[1qa2ws3ed]

one example: http://openmosix.sourceforge.net/#Documentation the last one.

Hmmm....

[SharpFang]

Start a cluster of these with some profitable computations at work in the evening, using every worker's own PC, then come in first in the morning to remove all evidences quickly and painlessly... (or even watch over that all during your graveyard shift as a sysadmin)... Instant cluster - that's clever.

A much better name

[Mr_Perl]

From the site

OpenMosixKnoppix didn't quite sound good, so I called it ClusterKnoppix ;)

I would propose the moniker "Cloppix"

Brings to mind images of a certain powerful one eyed giant...

Re:Instant Beowulf Cluster

[lacrymology.com]

I did this... ... ..... .. . What's wrong with my computer?

Knoppix

[ajs318]

has replaced tomsrtbt as my rescue tool of choice.

It probably would have done so even if any of my latest machines had a floppy drive ..... what these people have managed to pull off is fantastic.

I really want to understand...

[JesterOne]

Everyone is talking about how cool this is and how well it runs but what would I use this for in the real world (even the 'fake' world). So, I have a network by day, cluster by night. What am I going to run on it? I mean folding proteins in realtime may be cool to some but, come on... I really would like to understand (I'm an MCSE running all MSFT except one server - finance server is running RedHat). Someone explain this to me (linux avocates - this is your chance)...

Re:I really want to understand...

[b17bmbr]

I'm an MCSE running all MSFT...

sorry to hear that. but really, it isn't "what can i do with this", it is simply, "i can do this". it's like an education, what you do with it is your business, but you have an education and you are able to do so much more. and in fact, try to do this with your msft boxes. it can't be done. because they tell you what you can and can't do. period.

Re:I really want to understand...

[krb]

pay no attention to the "insightful" comments that serve to dress up a "Fuck you, MS dude."

I'll try to give you an actual response. People have been quick to mention Knoppix CD's for rescue operations -- this doesn't apply to the Clustering feature, just knoppix in general. I used one of these last night to fix my roommate's system which had gotten totally owned and was halting at LILO. Could i have done it with a floppy based linux distro? Probly, but it would've been a bigger pain, because the floppy is small and may not have the tools i need, whereas a CD is big enough to have damn near everything.

That being out of the way - some uses for the cluster disks.
1. say your server (using ClusterKnoppix), which has a hard disk and lots of ram, etc, runs a really dynamic web site which needs lots of CPU. If you see that you're getting shitload of connections you take some other systems that aren't critical, pop in a CD and reboot and add their processors to the pool to help out the web server

2. as has been mentioned, in academic institutions, you could use this to harness the computers down the hall in the public lab for experiments overnight...

3. i don't today, but someday i may need a cluster, and why make it difficult if i can pop a CD in 6 LAN systems and get it going rather than spending a week on configuration. Shit, i've had occasions where my computer was compiling for 3 days straight... would've been nice to fire up a couple of secondary systems to help out...

i suppose you could call these contrived examples, but they're not wholly unrealistic. i think what you're getting at is, "why should normal people care?" which is a good question. is this useful for 90% of computer users? fuck no. 1%? Maybe. it solves the problem of running a cluster which can be simply and arbitrarily resized (keyword simply). If you have no need for a cluster, then you certainly don't care about a resizable one.

keep in mind though, that lots of things can be cool without being useful to yourself. i have no need for a supercomputer, but i still think they're pretty interesting and cool. i think this is a cool technology too, useful for a certain class of problem, and a limited set of users.

that's my 57 yen... for what it's worth.

bittorrent up

[Anonymous Coward]

clusterKNOPPIX_V3.2-2003-05-20-EN-cl1.iso.torrent

(also added to the main clusterknoppix website)

BITTORRENT UP!

[parkanoid]

here! Be gentle, the torrent itself is hosted on my school account, and I'll get ownzored by the administration if we get /.ed.

Let me see if I understand this

[LWATCDR]

If you set this up correctly all the computers that you boot up with this become a mosix cluster? Then all the users are terminals off of this cluster?
So all of the users have some of all of the power of the Mosix cluster?
This could be very very cool. Imagine a whole campus of users running this. Each user would have access to a super computer.
I just wonder how well mosix handles nodes dropping off and back on again. Plus how well will can is scale? Could you have five hundred or a thousand systems off in the cluster. Where is Mr. Barr when you need him?

Re:Let me see if I understand this

[1qa2ws3ed]

> If you set this up correctly all the computers that you boot
> up with this become a mosix cluster?

an openMosix cluster, not a mosix cluster.

>Then all the users are terminals off of this cluster?
if you want, yes.

> So all of the users have some of all of the power of the
> Mosix cluster?

yes

> I just wonder how well mosix handles nodes dropping
> off and back on again.

if a node goes down for a small time, and then comes back, no problem. if a node goes down for a time long enough to finish his work, processes won't come back where they came from, so you (or your apps or scripts) have to take care of this situation. tipically in a cluster you don't want nodes to go down, never. this can be a situation tipical in a pc laboratory or the like, for an entire campus this probably is not adequate, you need something more "grid computing aware"

>Plus how well will can is scale?

it depends a lot on the speed of the connection between nodes, on the type and amount of traffic generated and so on the type of computation being made, on the number of nodes, on the speed of the clients, etc...

>Could you have five hundred or a thousand systems off in the cluster.

tecnically up to 65535 nodes (last 2 bytes of ipv4 address) if i'm not wrong. i was told biggest cluster of this types count 1-2k nodes, but i'm not sure.

How'd you boot the clients?

[brave1]

Did you boot the client workstations over the network (Using PXE or something similar?) or did you boot both the clients and the server from CD? (The story mentioned the LTSP

I'm curious about how difficult it is to boot clients over the network.

Re:How'd you boot the clients?

[1qa2ws3ed]

you can do it in both ways. i had 2 nics that support pxe boot, i just had to follow the wizard on the server, and turn the clients on. job done. ah, i precautionally turned off my already running dhcp server after booting the first clusterknoppix machine, don't know if it was necessary, i was afraid of conflicting dhcp servers because clusterknoppix starts it's own with the wizard.

Re:How'd you boot the clients?

[#undefined]

i haven't messed with clusterknoppix (just knoppix itself), but have looked into net booting a few machines at home to serve as x-terminals (since ltsp packages exist for debian and is included with knoppix).

most newer machines with an on-board nic (my ecs k7s5a) support pxe boot as an option within the bios. but for older machines, net booting requires either a nic that supports net booting (should have an eeprom on the nic), which are not the cheap nics that i've always bought, or with a floppy (which can probably be burned to a cdr to make a bootable cd).

for net boot floppy images see the section "Making Boot Disks for Legacy PCs" in the k12ltsp client set-up instructions. those instructions reference the rom-o-matic website, which supplies dynamically-generated downloadable boot floppies using etherboot.

so for a small (1.44 MB) simple net boot solution for computers that don't support pxe booting, but with a floppy drive (or bootable from a cd), look at the k12ltsp instructions and visit the rom-o-matic website for a net boot floppy.

ps the images might even be on the knoppix cd as net boot floppy images are (sometimes?) distributed with ltsp, but probably not due to space constraints on the knoppix cd.

Computer Lab?

[GrouchoMarx]

From the web site:

* "openMosix terminal server" - uses PXE, DHCP and tftp to boot linux clients via the network.
No CDrom drive/harddisk/floppy needed for the clients
* openMosix autodiscovery - new nodes automatically join the cluster (no configuration needed)
* Clustermanagement tools - openMosix userland/openMosixview
* Every node has rootaccess to every other node via ssh/RSAkeys
* MFS/dfsa support
* Every node can run full blown X (PC-room/demo setup) or console only (more memory available)

Aside from the "every node has root access" bit, am I way out in left field thinking that this would make a good computer lab system? Just start up the clients and they pull from the Knoppix central server and you're done. No need to have floppies, or even to bother locking down a system. The student does something screwy to the PC, hit reset and you're back to fresh configuration.

Or am I missing something completely here?

Re:Computer Lab?

[Flammon]

Well, yes, you're missing an equally important point that it is a cluster. X diskless workstations are part of this system but making those X workstations part of a Mosix cluster is what makes this setup interesting.

Re:Hehe Neat

[Suidae]

Mosix works like a big SMP box, no special code is required, so you just fork and forget.

I can see that that would be extremely cool for processing, but (and excuse my ignorance here), how does this apply to things like network servers? Can I install apache on it and get the benefits of multiple servers without having to do any special configuration? What about database servers?

Knoppix is impressive

[FreeUser]

Knoppix is very impressive. As a former Debian (now Gentoo) user and administrator, I can appreciate the quality of the "back-end" engineering in distributions like Debian, which is IMHO hands down the best binary distribution out there (Gentoo is a source based distro, as is Linux from Scratch and Source Mage. It is my preference for source based distros, and portage in particular with Gentoo, that led me to switch, not any argument with the quality of Debian or apt-get, which is excellent). To see such a slick, astonishingly easy live-cd environment put on top of such a quality distribution is delightful, and while I yearn for a Gentoo knoppix (and will likely get my wish with their ever-improving but as yet no-where-near-as-good-as knoppix live-CDs), I have on more than one occasion used a knoppix CD to rescue a non-debian (Gentoo, Red Hat, Mandrake, Suse, you-name-it) distribution.

Having such easy clustering, with such an idiot-proof interface ("put the CD in the drive, boot, and you're ready to go"), built upon such a solid foundation where shortcuts that afflict other distributions haven't been taken, is truly an achievement worthy of praise and respect.

In short, knoppix already rocked, and now they have surpassed themselves again! Very, very cool!

Re:Hehe Neat

[gladbach]

since when does vidomi work on linux? I know it was talked about, but havent seen anything come to fruition yet...

As Louis Armstrong once said

[gosand]

I really would like to understand (I'm an MCSE running all MSFT except one server - finance server is running RedHat). Someone explain this to me (linux avocates - this is your chance)..

You ain't from around these parts, are you? :-)

In all seriousness though, I do think that your MCSE and your Windows environment is limiting you here. I actually think the MCSE should be changed to CMSE, because you are a Certified Microsoft System Engineer. You are taught how to admin Microsoft systems only. It's OK, those are necessary things. But the problem is that you have been taught how to think in a "Microsoft world". There is a lot outside that world. Clustered computing is one of them. A bootable distro (ala Knoppix and others) is another.

I am sure when the bootable floppy distro came out, the MCSE's cried "what would I do with THAT?". Then CDRWs came about, and the bootable floppy turned into the bootable CD distro. The MS crowd said "Neat. Big deal." That has now turned into a bootable cluster server. Who knows where it might go from here. At some point, someone at Microsoft will say(or has already said) "Hey, that is cool. Can we do that?". They will try to buy the technology, and will find it can't be done. And they will try to build it from scratch, and there will be much wailing and gnashing of teeth.

I think it was Louis Armstrong, who when asked what Jazz is, said "Man, if you gotta ask, you'll never know." I am afraid that applies here.

Re:ObVMSPost

[TrekkieGod]

Linux must stop trying to be like Windows.

Yeah, exactly. If I want an instant cluster, I can just boot my server off my WindowsXP cd, thank you very much.

Re:BitTorrent = SHIT and still not legal anyway

[itzdandy]

i must second the thought of the AC. i am using bittorrent to grab this file in a %100 legal manner and im getting well over 200K.

bzzzzt... sorry.

[Ayanami Rei]

So, when an incoming connection comes into the head-end, and then the incoming socket data is routed over the network to the remote machine running the accepting thread instance, and the reply is shuttled back over to the head-end, and then forwarded to the internet... how is that any better than having Apache running on each server individually behind a load-balancer?

It would be nice to run a database or app. server in the mosix cluster with a web front-end. Apache itself will not scale over the cluster.

Useful

[AvantLegion]

Ahh, instant clusers.

For those times when you have to prove that you are absolutely, positively the biggest nerd at the party.

Re:Not Flamebait but a sincere question/comment

[raju1kabir]

I tried out the previous release of the bootable knoppix and found it incredibly cool. But really, WHAT are the uses for this? or cluster knoppix that we should try or be interested? The only use i have for it is as a bootable cd in case my main os installation is trashed and I need to retrieve some files right away.

I think the utility of Knoppix increases dramatically if you tend to be on the move a lot.

In the office, I don't do much with it. But when I'm traveling, it becomes essential. I can stick it in an idle machine and get to the bottom of network problems while other people - even the people who set the network up - are still pointy-clicketing around to find some diagnostic tools on their Windows machines.

I can stick it in a machine at an internet cafe (or anywhere else on the planet) and get reliable, secure remote access to my office desktop.

At hotels and conventions, when nobody can figure out how to get on the show net, I can boot up and sniff for a few seconds to get the lay of the land.

Knoppix has made a hero of me more times than I can count.