Advanced Filesystem Implementors Guide Continues

Tom writes: "This is part six of the Advanced filesystem implementor's guide. I've been following an outstanding series of articles about implementing the advanced filesystems that are available with Linux 2.4. The author really knows his stuff and has done a great job with explaining Reiserfs, XFS, GFS, and the other file systems that are available." The series gets into greater depth as it goes on; you may want to start with Part One and work on from there.

Comments

Encrypted loopback root example.

[Adam J. Richter]

I am posting this from a notebook computer that has all partitions encrypted except for a boot partition at the front of the disk. The kernel boots an initial ramdisk with an /sbin/init script that does essentially the following, using cryptoapi, the successor to the linux "kerneli" patches.

modprobe cryptoapi
modprobe cryptoloop
modprobe cipher-aes
losetup -e AES /dev/discs/disc0/part6 /dev/loop/0
Password:
mount -t ext2 /dev/loop/0 /newroot
cd /newroot
exec ./bin/chroot . ./sbin/init $@

This should work with any disk file system, not just ext2.

I have been using this arrangement for several months now on a couple of computers, the slowest of which is a Kapok 1100M that uses a 233MHz Pentium II process and, I believe, PC-66 SDRAM. On that computer, the change in interactive responsiveness is hard to notice, but it is noticible for disk intensive activities. I have not timed it, but I think that big rsync runs are at least a factor of two slower.

I do not run swapping on these computers, as I've seen claims that there are more potential deadlocks when attempting to swap to an encrypted partition than when attempting to swap to an unencrypted partition.

I hope this information is helpful.

Re:File systems obselete?

[Peaker]

Persistency: Data's "survival" throughout time, power breaks, etc. Persistent memory is non-volatile memory (disk, for example).
Persistency in operating system is usually achieved by writing things to disk, in order to persist them.

... could you explain what you mean when you say objects in a filesystem are forced to be serialized?

Not all data in a file system can be stored as it is in memory, because pointers, and other information must be converted to persistent form. Often objects are stored in very difficult ways to write to disk (by being spread on many small linked objects, for example). This means you must serialize the data into the disk, by converting it to a stream of 1's and 0's, that allows reconstructing the objects' structure. This requires a lot of work for every application and object implementor, as they have to create methods to serialize, and de-serialize the objects, from their normal repserentation to a persistent streamed representation.

And what orthogonal persistency is. It sure sounds good, but I would really like to know what it means.

Orthogonal persistency is persistency implemented by the underlying operating system, rather than every application writer.
The entire system state is saved to disk every once in a while, in a checkpoint.
Mechanisms are used to ensure there's always a stable/reliable checkpoint to go back to. Some schemes even let you roll back to any checkpoint in the past. Typically, checkpoints are done every 5 minutes.

Orthogonal persistency is totally transparent to applications. They seem to 'live forever', and do not need to explicitly persist or serialize their information. They can keep it represented as objects, or whatever representation they choose for their own simplicity.

Orthogonal persistency treats RAM as a cache to the disk, and thus achieves two purposes.

Simplicity: There is only non-volatile memory, rather than volatile, and non-volatile memory, that are allocated and managed separately

Performance: It is much easier to optimize this system, as there are no file caches, and memory swap areas on disk. Instead, you treat the entire RAM as a cache to the disk, allowing simpler and more powerful page caching algorithms, that do not have to guarantee things such as quick disk writes for files, as file systems do.

An amazing advantage for orthogonally persistent systems, is that due to the entire chunk of dirty pages from memory being copied to disk at once, it can sequentially move the disk heads across the disk to update all necessary areas. This process is called migration, and is a far more efficient method of updating the disk from the volatile state, than the explicit update used by current file systems.

Yet another advantage, is that due to the entire system state being preserved as a whole, more powerful security schemes can be used. The whole load-from-file process can be avoided, and with it, the security problems of identifying who has access to what file, and why.