Linux Network Install Options?

verch writes "I work at a Large Corporation that is considering Linux as a competing platform to our large Solaris and Wintel install bases. One of the biggest concerns is how easily a large number of linux machines can be installed mostly unattended. For Solaris we mostly use jumpstart which allows us after minor preparation to simply remotely attach to the console of a machine and boot and install it over the network. x86 hardware tends not to be as nice, I don't know of any linux hardware that has remote console capabalities as nice as Sun's, and network boot options are limited (PXE?). Installation procedures that involve having to lood a CD or boot floppy into every machine could potentially be dealbreakers. So, what options exist for completely remote unattended installs of linux machines?" Had a similar question a while ago; have things improved since then?

Comments

If you use RH, check kickstart

[Papineau]

Just finished installing RH 7.2, and it left a nice trail (not only a log) of what I did. So next time, I could just copy that file (kickstart) to an install floppy and it would bring it back to what I just installed. Be sure to check all the options (interactive or not, partionning, etc.).

I know RH has always been installable n network (ftp, nfs). Probably some other ones too (Debian). It's before you start that your problem lies (booting the computer to start the install). Sorry, I don't see how you could just hook a new PC to the network and start installing without being at the console.

Also, what do you use on the Wintel side? It works with your current hardware, right? Is it (or could it be made) enough OS agnostic to be used with Linux?

Re:If you use RH, check kickstart

[SuiteSisterMary]

Windows 2000 had a PXE install; only needed a boot disk if the computer wasn't PXE-boot capable. Of course, you needed to stand there and hit F11 or some such. If you had the appropriate GUIDs from the machines, you could even pre-load the active directory and it would go.

Re:If you use RH, check kickstart

[rtaylor]

Booting the computer off a network share if it fails booting from the harddrive (bootp, tftp) will get it to step 1 with a known good ssh key and account.

Step 2: is to dynamically (dhcp) assign it a static (internal) IP. Have ssh come up on a non-standard port.

Login using the key to the IP and do your installation probably via a local ftp or nfs server.

Step 3: is tricky. Mount the drives overtop of currently existing partitions (ie. Drop existing /usr, mount new /usr. Mount new / overtop of old /). Run daemon startup scripts for ssh (standard port, freshly installed copy).

Step 4: Login via ssh on standard port to ensure it works and you have access. Now shutdown your old ssh connection, and stop the daemon on the non-standard port.

Step 5: Configure away. Live IP (alias on card with DHCPd internal IP, or a seperate nic entirely), passwords, other basics.

Step 6: Reboot.

It'll boot from disk successfully, and go from there. You've already verified ssh works on the internal IP, so if there are problems you should be able to get in to fix them.

If you prefer, you could setup the kernel of the network booted copy to load serial console support, as well as the freshly installed copy. I'd to ssh anyway, just in case.

Did this with 5 boxes quite a while ago with early FreeBSD 3.x. May have been even earlier, so modify to suit current requirements.

I never did figure out what to do if the box couldn't get to the point of console or network connection. The bios won't start a netboot then as it's found something on the harddrive, it just fails.

Re:If you use RH, check kickstart

[t0qer]

Actually, it's called Remote Installation Services. It's pretty nifty for win2k, but the guy want's linux/solaris systems, so I'll save the howto if the question is ever asked for those M$ systems.

Oh and you wouldn't have to sit there and press F12 either, thats an option that can be turned on or off.

Re:If you use RH, check kickstart

[SuiteSisterMary]

Well, you can set the bios for the machine to include 'network' in the boot order, but that has it's own whack of problems.

System Installer Suite

[grundy]

System Installer Suite I can't say enough about how cool this project is. Plus the developers all hang out on irc.openprojects.net in #sisuite.

See the Diskless-HOWTO

[tshoppa]

Most of the questions you need to be asking at this point (especially "how do I boot a PC over the network?") are answered in this section of the Diskless-HOWTO. It discusses TFTP, BOOTP, and network boot ROM's.

Mandrake 8.2 (and probably 8.1)

[dodald]

I just installed Mandrake 8.1 and it has an option to create a auto-install boot disk for unattended installs. I believe you could make a CD with only the packages you need, and use it to create the auto-install boot disk, it would be fairly unattended (you would need to put the disk in and probably confirm the install)

Re:Mandrake 8.2 (and probably 8.1)

[sharkey]

Here you go. Mandrake's auto-install page.

It's not supposed to be that easy

[Skapare]

If you can initiate a network install from the network, then you have a security problem. I'm glad it doesn't work like that. But PXE comes close, allowing a machine to boot from the network. Redhat supports this and the support code is available from them and Intel (who created PXE, which is found in their server grade network cards). There are also some machines that have serial consoles (like Sun). The Intel ISP-1100 rack mount servers have this, and I can control the BIOS and select the boot source this way.

Tell me why installing from a CD is a deal breaker. Is it because you see it as an interactive thing, requiring you to sit at the keyboard for each machine? I'm working on the design of a CD-based headless non-interactive install system. It can either have the files to install on the CD, or on the network (uses DHCP or generates a 10.X.X.X IP address from the low 24 bits of the ethernet address). You make sure the machine is configure to boot from a CD, stick in the CD, press reset to reboot, and the CD boots up and does it's thing without any more interaction from you. You have to have pre-built your system tree (I've done this with Slackware as well as Linux From Scratch). Unfortunately, my work is still a work in progress.

Re:It's not supposed to be that easy

[battjt]

Don't think of the CD as a boot device, think of it as a boot rom. It should have the same stuff that the sun rom has (ie, not the kernel or any of the real OS).

It is just a more easily replaced boot rom.

(A floppy would work, but there may be reliability isses.)

Joe

kickstart

[josepha48]

I think redhat has something they call kickstart. You basically have a file that tells it how to install RH and just boot the machine with the disk in then go away.

Also there are a number of places that do stuff like this with Linux as they have networks of computers. Try going to linuxdoc.org and readding the howtoos... http://linuxdoc.org/HOWTO/HOWTO-INDEX/howtos.html

Also read this howto http://linuxdoc.org/HOWTO/KickStart-HOWTO.html as it is about the RH kickstart. It talks about network installs and nfs install as well.

Re:kickstart

[edyu]

My company also need to network boot a large number of linux boxes. We use Redhat kickstart. It's one of the easiest way to install a large number of workstations.

Real Weasel plus RH

[gmhowell]

Not a customer of real weasel, but try them with RH install scripts plus a boot PROM. Should do the trick. Just remember to check for a pre-existing install of RH before installation, or you'll wipe out your install at every boot.

(Sorry if this is a repeat. Just had some browser weirdness)

Solaris Jumpstart Design Notes

[bihoy]

The Solaris Jumpstart Design Notes provide a fairly good outline of what would be needed to accomplish remote installation of linux boxes.

Here's the overview from that document:

1. Client sends a RARP for its IP address

2. The Boot Server responds via RARPD (in.rarpd) with the IP address in /etc/ethers or the ethers NIS/NIS+ map depending on the ethers setting in /etc/nsswitch.conf

3. The client sends a tftp request for a bootimage

4. The server starts in.tftp from inetd and sends the small net kernel image

5. The client then sends out a bootp request

6. The server responds with the clients entry from /etc/bootparams

7. The client NFS mount it's root partition from the install server

8. The client then mounts the configuration server (/jumpstart) and runs "sysidtool".

9. It then mounts the install image and runs Suninstall to begin the install process.


It seems that Kickstart provides one, but not all, of the key components that are needed.

Checking out the Kickstart Documentation it seems that no provisions are made for remote installations. You must insert a boot floppy and be at the console to handle special cases that will most assuredly arise.

VNC may meet the need for the remote console component. Of course you would need to get a basic kernel and IP running on the box before you could even start up VNC. That mght present a chicken and egg scenario.

There certainly would also be a fair amount of code required to glue it all together.

PXE: anyone actually used this in anger?

[robin]

I remain to be convinced that anyone has actually had this working.

I spent ages working with various combinations of dhcpd, tftpd, Solaris and Linux in a vain attempt to boot my laptop off the network.

In the end I cracked and bought a floppy drive for it.

Could anyone that has a working config publish it, together with the version numbers and config files of all the servers installed? All the vendor-specific messing in the DNS, ick.

Correction.

[bihoy]

Here are the correct links. (Note to self: Drink more coffee, less vodka.)
Jumpstart Design Notes
Redhat Kickstart
Kickstart How-to

Remote console

[PinkFreud]

http://www.dell.com/us/en/esg/topics/esg_pedge_rac kmain_servers_1_pedge_1650.htm

I seem to recall these having decent serial consoles, supported by the BIOS (one can even configure the BIOS via serial console).

Kickstart

[OrcSlicer]

RedHat and Mandrake's answer to jumpstart (wonder how they came up with the name :-) ). You have 1 server dedicated to being the jumpstart server, and then boot each machine off a specially made floppy. You can do multiple machines at once. The machines can be set to boot off dhcp after that, and you ssh in and tweak things out. You can set them with a static, but then the floppy has to be customized for each static address, which is a pain, and relatively or you can set them with a static, but then the floppy has to be customized for each static address, which is a pain, and relatively pointless for a mass installation

I'm not sure if the machines can get an IP address through dhcp while loading packages, which would mean the floppies couldn't be clones of each other, but clones minorly modified. I think they can get from dhcp while loading, but I'm not certain.

I've loaded 20 machines in one day using this.

We never had them reboot automatically, because we needed to eject the floppies, but if you make CD images, you can probably figure a way to eject them and then they will load off the HD correctly on the reboot.

You can also use the same jumpstart server with floppies customized for different machines if you have clusters of machines with different baseline package sets.

Drawbacks: It works over nfs (not sure if there are other delivery methods), which you would want to firewal off at the router.

Random nice things: you can put the passwd files on the floppy, which is more secure than having `them fly across the netowrk, methinks. It does mean you need to take care with the floppies, however.

Rob

Network booting Linux on Sun hardware... fails.

[Nonesuch]

One of the many negative experiences that soured me on ever using Linux for 'real work' was trying to network boot a SparcStation LX a couple of years ago.

I had the process working perfectly with SunOS and NetBSD, but Debian was rumored to have the best support for the dBRI audio chipset...

I tried Debian, I tried RedHat, I read the HOWTOs and FAQ files, but Linux just would not work with network booting a Sparc and mounting a NFS root. There's been orders of magnitude more effort put into NetBSD's netboot support, to the point that I have better luck booting Linux from the 64K NetBSD tftp bootloader than the Linux method of tftp'ing a 1.6M kernel.

Eventually, I give up, and go back to NetBSD for these LX boxes. They boot, use NFS for root, and even swap to a file mounted on NFS. Works like a charm, boots fast and reliably, they just can't play music.

Kickstart + RPM

[Linux_ho]

In '99 I started using the procedure below. This works really well if you need to support hardware that might vary from box to box. If you know you will be using the exact same hardware for every box, you might as well use dd or some drive imaging package.

1) Figure out what you want your final image to look like, set up a test system, and configure it fully. Make note of all config files that you changed and all packages you have installed.
2) Create an RPM containing all the config files you modified.
3) copy RedHat CD 1 to the hard drive of an FTP server
4) copy all CD 2 RPMS to the RPMS directory that was created when you copied the first CD
5) replace all outdated RPMS with the update versions, and put the RPM you created in step 2 into this directory.
6) install anaconda on the FTP server & run the genhdlist script.
7) create a kickstart config file listing all the packages you want to install, including the RPM you created in step 2, and specifying the address of the FTP server.
8) make as many boot disks with the above kickstart config file as machines you want to install at once.
9) plug in as many machines at once as your network can handle, put a copy of the floppy in each, turn them on.
10) go get some coffee while the install completes.
11) yank the floppies and reboot the boxes. Your drives are partitioned, all packages installed, and all software configured. Proceed to the next batch of machines. I'm pretty sure there's a way to do this from a network-boot setup and just put the kickstart file on the FTP server, but it seemed to me like more hassle than it was worth given how easy it is to use floppies.

Sun and Solaris and painless mass install/upgrades

[Nonesuch]

Sparc hardware and Solaris makes it easy, but secure. Once you put the effort in to build a Solaris Jumpstart server, installing any number of machines is trivial.

My servers have Sun LOM/RSC serial consoles connected to a 'secure' (OpenBSD) serial console server, so forcing a re-install of the base OS is as simple as shuttting down, entering 'boot net - install' at the 'ok' prompt, and waiting 45 minutes or so. I can do this as easily from across the office or across the country.

I routinely build, hack, and rebuild test servers several times a day. Others report doing 500 machine rollouts with little or no human intervention.

Some of the features of the Sparc PROM and Jumpstart can be duplicated with a boot floppy, but many of the coolest features are not as easily imitated. The PC Weasel is a pale imitation of the boot PROM. APC remote power cycling doesn't come close to the functionality of Sun's Remote System Control. Linux 'netboot' is years behind Sun's Jumpstart software.