patrick42 asks:
"I am working on a project where I'll be replacing a DOS/Windows-based system with that of Linux or FreeBSD. The company for whom I'm working uses cheap PC's running some proprietary software on DOS/Windows to perform a certain task. The machines are deployed in environments where there are no keyboards or displays, and minimum-wage clerks are the people watching these machines. The company has decided to go with a free Unix system because they no longer wish to pay the licensing fees for Windows. The machines get unplugged all the time when they are moved or whatnot. They do not get a proper shutdown procedure ever, and it's not possible to change this due to the environments in which they are deployed. I've been told that they've never had a problem running DOS in terms of filesystem corruption. So I guess I'm looking for the safest filesystem possible that I can use with either FreeBSD or Linux. My head would be served on a platter if I picked something that sometimes requires user-intervention." Note that
Ask Slashdot covered a similar question back in 1999, the situations differed, but the need remains the same: can Linux work in environments where proper shutdowns are rare-to-non-existant?
"I have run many Linux machines, and I've experienced firsthand (only on occassion) where a machine did not get properly shutdown, and then on the next boot user-interaction was required to run fsck manually.
I really want to use either FreeBSD or Linux, but if there is any chance of this happening (hardware failures excluded) where someone needs to manually run fsck, I will not be able to use them.
I've been reading about the ext3 filesystem, and how corruption is quite rare, but it still seems possible. UFS claims to be quite stable as well, but fsck-less booting will not be available until FreeBSD 5.0 (from what I've read).
These machines aren't doing too much writing to the disk -- they are mostly just reading data, but that isn't to say that there will be no disk writes at all.
Can anyone offer some advice?"
Slashdot Mirror
The simplest solution is to go read-only for all system data, such as binaries and static configuration. Even better: use something of a commit-system to commit configuration changes to disk and have the disk be read-write only when committing changes.
Even though you'll still run with fsck this will not be a problem, as stuff can't have changed for reasons other than hardware failure... and you're not going to work around that in software anyway.
Disable (or strictly limit) the write cache (-> Relevant documentation). Use a journaling filesystem. The result will be at least as good as with using the FAT-filesystem and DOS. The journaling filesystem means that the filesystem *structure* will always be consistent. Disabling the diskcache will reduce the chance of inconsistent *data* to DOS levels. But in the end, the application has to have precautions against inconsistent data, much like a journaling filesystem protects against inconsistent filesystem structure.
I've had to deal with a situation like this before - a hard drive in a laser printer, where there was no shutdown procedure, only an on/off switch. I used the "sync" option - e.g.
/dev/hda2 /usr
mount -t ext2 -o sync
This causes the filesystem to be mounted synchronous, so that there are no deferred writes and all disk writes are committed to the disk before the I/O call returns.
This is not 100% fool proof either, as it is still possible to power down the machine in the middle of a write, but it makes it much more difficult to screw up.
You didn't really say much about what the "certain task" these machines do. Do they need to save a lot of data? You could boot off a CD-ROM and use RAM disks for /var and other writeable partitions. Each time the machine is unplugged, it returns entirely to its initial state.
If you want to save a small amount of data, you could put a VFAT formatted floppy and write persistent data there.
-- Don't Tase me, bro!
Secondly, once the box is configured, edit your fstab file and change any partitions which don't need to be written to to be mounted read-only. If there are no writes to a volume, then there is no need to check the volume (this is how I used to speed up post hard-down boots before journalling filesystems). It's a good security practice as well - in combination with chattr it can be a very effective "escalation of priviledges" block.
UNIX? They're not even circumcised! Savages!
Ext3 in ordered mode was my first thought. I'm comfortable with the stock kernel - after crashes (and on regular intervals), I don't think I've had to intervene with the fsck at bootup. If you'd prefer more QA, then you might examine the patches that, say, RedHat and Debian provide in their kernels, and stick with one.
If so, then it shouldn't matter what filesystem you use, so long as you mount it read-only. Then, keep all writable data on a RAM disk (for /temp and friends) and on the network (for real data).
If the systems are new enough, I'd even consider booting from CD or network and doing away with the hard drives completely.
Says the RIAA: When you EQ, you're stealing bass!
Darwin and Mac OS X runs fsck itself at boot and does not ask questions. It seems to handle it well. Darwin will run on x86. Mac OS X is proprietary, but does not have the steep licensing fees that windows does.
I do this for my CF-based firewalls.
The CF has two partitions on it: A 3M boot/config partition (Minix), and the rest is reserved for the cramfs (right now the cramfs takes about 10M).
The boot/config partition holds the kernel, GRUB, an initrd image and the system configuration. At boot time /conf is booted and the initrd is loaded and executed. It loads a few modules, mounts the cramfs and then pivot_roots so that cramfs is /. The cramfs boot process mounts /dev/pts, proc and /var (as a 4MB tmpfs), untars the basic /var system and runs init. The rest is a standard boot.
The advantage to this is that I don't waste RAM by decompressing the entire root filesystem; cramfs decompresses the program to RAM at execution time. /var is the only point that is always rw but it's RAM anyway. Any time I need to do a config change I remount /conf as rw, make the change, and remount ro. If power is lost there is no fsck.
At present I have a complete firewall (including ssh, ipsec, the iptables 1.2.8 connection helpers (h323, ah/esp, etc.) and a full Perl install (I use it for SNMP, XMLRPC, integrity checking, etc.)) to fit in a 16M CF card. Cramfs is *awesome* :-)
In your situation I would probably do away with having the config in /conf, instead mounting it from the network (perhaps using BOOTP).
Use linux, and go ext3 (the journalling version of ext2). Mount the ext3 filesystem with options "sync,data=journal", and you should never have any issues.
11*43+456^2
[A hard drive that has lost power] probably performs a controlled end-of-write as soon as possible and moves the head into the parking position.
Why can't such a drive be fitted with a capacitor or something to store just enough power to be able to commit the controller's cached data to disk? Does the motor use too much current for that to be practical?
Will I retire or break 10K?
Definitely enable SoftUpdates if you're using one of the BSDs. It will ensure metadata stays consistent as much as possible. Much better behaviour in case of unexpected shutdown. Actually, with SoftUpdates a boot-time fsck isn't required, it can happen after boot in the background - but that won't be in a released version of FreeBSD until 5.0. Much faster startup on a system with large filesystems.
Soft Updates ensure that the filesystem is always in a consistent state. Updates are effectively not marked as complete until they have actually all gotten to disk. This ensures that after a re-boot, the system is consistent, maybe with the disk state as that of a some seconds earlier. The Soft Updates technique is also much faster than journalling, which is your other option (reiserfs, ext3fs etc in Linux).
I said above that fscking is practically eliminated - in fact a fsck task still needs to run to recover sectors that are 'dirty' but the system is stable without it - critically the system boots up without it, and in the background at some point when the system finds time to do so it recovers the sectors marked 'dirty'; the soft update people call this a "background fsck".
Note that this won't stop loss of data - but then nothing will stop loss of data. fsck certainly won't even if it is run properly, because that's not what it does. What it does do is ensure the filesystem metadata is always consistent (i.e. whether a file has been created/deleted, contents of directories etc).
More details on soft updates can be found in the OpenBSD FAQ and also in the FreeBSD handbookFreeBSD handbook.
If you want to get the same kind of disk flushing that you get with DOS, then you can only really do that with a single-tasking operating system (if that's not a contradiction in terms!) which can therefore ensure a minimum of delay between the application generating data and it being flushed to disk. Note this is never perfect, but can be close enough that you'd only notice one in a million power-offs.
eliminate the disk entirely, silly person.
boot from flash or from a CD. if you really
need to store more data than you can keep in
flash between power-cycles, then use CDRs.
when one fills up, eject it, and they can
pop in a new one. *bam* instant permanent
audit trail, in a compact format.
-I like my women like I like my tea: green-
I have been using EXT3 since Linux Kernel 2.4.12, a couple revisions before it got folded in...
It has severed me well since, with no hitch. My server goes down probably one or twice a month due to power outages(Yes I have a APC, but let just say its a little under powered and I am strapped for cash). The server goes down hard and ugly when this happens. It always come back with out issue however, and quickly! A normal boot after a proper shutdown is about 2 Minutes(it starts lots of services)...when in ext3 recovery it only takes about 20 seconds extra.
Power Corrupts,Absolute Power Corrupts Absolutely, leaving one person(group)in charge is absolutely corrupt.