Preventing Shutdown on Active NFS Servers?

Ed Almos asks: "Like many Slashdot Readers, I run a small network at home with a server and a number of desktops. The server holds all our files as NFS shares and doubles as a desktop machine should the need arise. Problems however occur if the server is shut down whilst there are NFS shares in use, the minimum disruption is a crashed desktop and a couple of times I have had to deal with corrupted files. Does anyone know of a way to prevent shutdown of a machine if someone else has drives mounted to its NFS shares ? I have already explored use of the /etc/shutdown.allow file but all this does is determine who can kill the machine. The minimal solution would be something similar to a Microsoft Windows system, where a request to shutdown brings up a warning window that there are users connected to the system, but I am not sure how to achieve this on a Linux system. Ideally I would like to prevent shutdown of a system with active NFS shares altogether, or at least until the user has unmounted and logged off the network."

Comments

But...

[Trbmxfz]

Not quite an answer to the article's question, but...

Theoretically, once the NFS server has crashed, shouldn't all clients simply freeze until the server is back? On all systems I used, this was the observed behavior, and it is quite useful actually: it seems to avoid all data loss problems (under conditions). When the NFS gets reachable, all running program go on executing as if nothing had happened.

A solution to the original problem, though, would be: tell all user that the NFS machine is to be powered on constantly.

Re:But...

[danbeck]

This is only useful if the clients are never doing anything important. Try this on a huge cluster of webserver nodes with nfs mounts and you have one hell of a dead website. Remember kids, if you are going to go to bed with your nfs, bring along a few intrs for a safe, enjoyable time.

You've got something wrong...

[Whip]

If your NFS server rebooting, shutting down, or crashing causes any problem but temporarilly 'hung' clients, you have something wrong.

NFS is explicitly designed to be stateless, precisely to allow it to function across server reboots, crashes, and other fun. If your clients are crashing, or getting back corrupted data, something is screwed up somewhere.

And, by the way, if you're getting corrupted data on a server crash, and the server is linux, you just had an object lesson on why it's bad that linux NFS defaults to async writes. :)

Try rwall or similar

[ctr2sprt]

There's a network-able version of wall that uses RPC (I think). It's not a foolproof solution, since it won't work if your users are logged in without an open terminal window, but it's a help. I'm sure it's terrifically insecure, but since you're running NFS you're already insecure (and so hopefully have a firewall).

If that isn't good enough for you, there are a couple other possibilities. You could probably cobble together an utterly trivial Python (or Perl or whatever) script on your client machines, then have the server invoke it via ssh when a shutdown starts. If you aren't a programmer at all, you could try firing off an email to the client machines. As long as you have a periodic mail-checker going, it would alert you to the arrival of a new message. (Since you'd be able to use the local spool, you could have it check every 15 seconds.)

Mod parent up

[Emnar]

I work for a major NAS storage company. Using the mount option "hard" is the right advice. It sounds like the submission author is using soft NFS mounts, which is a big no-no with rw mounts where you want any kind of data integrity.

Yeah, but the client thinks it's stateful

[bill_mcgonigle]

put a program on each client machine, call it nfsmounts and it would go a little something like this:

mount | grep $1 | wc -l

then write a wrapper on the server that does

foreach client (client list)
mounted = ssh client nfsounts `hostname`
ok = false if (mounted)
end

you can hook that into your shutdown script, and abort if there are any clients who think they have a mounted drive.

of course, read the other suggestions about mount options. Noone's mentioned sync yet, but don't mount your shares async even though the performance is so much better or you'll loose data.

Couple suggestions....

[!3ren]

First off, I would recommend that you never run applications on your file server. That just seems like tempting fate to me. Get a cheap old system from reseller/friend/garbage and use that.

Second, use FTP instead of NFS. Allow it to support resume, only let it talk locally. I believe there are utils which will allow you to mount an FTP as a drive in windows as well...

couple of ideas anyways.

Solved a similar problem ...

[Tux2000]

... a long time ago. Basically, every user gets a simple switch, all switches are connected in parallel, so you get a "wired-or". All users are told: If you need the server, turn on that switch. If you don't need the server any more, switch it off again. A little piece of electronic connected to a RS232 port and a tiny C programm control the power supply for the server. Details (german only, sorry) are here.

The trick is: The server now knows best when to start and to shutdown, there is no more need for manual boot or shutdown. If you replace the switches with a relay per workstation that closes the contact whenever the workstation is powered on, you don't even have to tell the users about the switch. You don't even need extra cables if you use 10/100 MBit/s ethernet, there are two unused wire pairs in every ethernet connection.

OK, this is a hardware solution, not a nice popup window. But don't we all hate those nasty popups? ;-)

Tux2000

Home network server

[1eyedhive]

I use samba to share my files so i can't comment on NFS too much.

Similar setup:
one Linux file server, 3 desktops (mixed nix and win2k), when the server shuts down, any cients active go nuts (winamp freezes, explorer complains about the sudden disconnect)
best theing to do, since you're on a home network is to make sure no one's using the damn thing before you shut it down.

This is easy, even with 6 systems, (assuming samba), just make sure no programs are activly using the shares and reboot the box, the workstations will yelp in agony, until the server returns, then all's well.

In windows, when a mounted (read: drive mapped) smb share goes off line, windows complains, programs too, then windows marks the drive as disconnected, when the server's back up, reconnectying by just clicking the drive in My Computer restores everything to normal. I haven't messed with client side SMB under nix enough to know what happens when a samba server reboots (gracefully or otherwise) while files are in use client side.

--

Re:Can't do it

[Anonymous Coward]

NFS is not stateless. There are files in

/var/lib/nfs

/var/state/nfs
which specificly tell IP addresses and mountpoints of clients. If those files are deleted/modified while nfs is off, when nfs returns it won't know anything about the clients already connected, and their accesses will fail.

showmount anyone?

[loony]

showmount -a is your friend ... that will show you all clients that currently have a filesystem mounted... just move your shutdown, reboot or whatever out of the way, replace them by a wrapper script that checks if showmount -a returns any clients, and only executes the real shutdown when noone has the filesystems mounted... if they do you can always print out a list of workstations that have mounted filesystems...