State Of The Filesystem

Skeme writes "Have you heard of Plan 9 or Reiser4 but don't know much about them? Are you curious about the improvements free software is making to its filesystems in general? Read my summary of the current developments in the filesystem: namely, what improvements we can expect (a lot), and what Linux and the BSDs can do to improve on the filesystem."

Comments

Transferring Files

[jhunsake]

I've always wondered how these filesystems with metadata handle transferring files between different systems. It would suck to have all your MP3 info in filesystem metadata and then lose it all when you transferred to a system without fs metadata. Anyone have any insight?

Re:Transferring Files

[ThogScully]

I would think that would be prevented by the filesystem implementation. A filesystem's features are only as good as the implementation. So if you have fs metadata and are transferring it to another filesystem, whatever tool you are using to move the files, whether it be at a BASH prompt or drag and drop in Konqueror, should figure some way of properly translating data from one system to another without losing anything. Moreover, the tool should be able to do it without knowing any particulars about the actual types of filesystems in use. It's all in the implentation.
-N

Re:Transferring Files

[warrax_666]

Anyone have any insight?

Are you mad? This is slashdot.

Re:Transferring Files

[Surak]

Well, remember Reiser4, for instance, is a TOTALLY DIFFERENT system than the standard POSIX filesystems (Reiser3, Ext2/3, UFS, etc.)

The idea is that data is stored as in a database, and small files can be handled efficiently.

Think of it this way: What *is* a filesystem? Really, it's a relational database. We don't tend to think of it as one, because it isn't implemented as one, but that's what the data it represents is. If you think of each file and directory and associated metadata as a record, with directories being pointer records to other records that show relationships.

Anyways, code needs to be implemented to handle this.

We use metadata NOW. A files permissions, owner/group info, date and time stamp, and even the filesize are metadata. We transfer those between different systems right now. With the correct transport method, this metadata doesn't get lost at all. A tarball can sit on any system no problem. Inside the tarball is all the POSIX metadata. That gets transferred from system to system.

Of course, some OSes (read: Windows) don't handle Unix-style permissions correctly, so these are translated into rough-equivalent ACLs with systems such as Samba.

Re:Transferring Files

[zatz]

A filesystem is nothing like an relational database. I wish people would stop making this comparison, because it's completely misleading and unhelpful. A filesystem is not a set of user-defined tables, each of which contains an unordered set of rows. Queries and joins are not possible. Constraints and null values are not supported. Files within a directory have an inherent order. Files are variable-length and byte-addressable. Duplicate "rows" are not permitted. The principle relationship modeled is hierarchy... ever heard of a hierarchical database?

Sorry, but

[Morth]

This article seems to just be the author brainstorming or feeling excited about reiserfs. It's hardly a "summary of developments in the filesystem". Now if he was asking about opinions on his article it'd be fine, but he's not, so I'll just discard this as another non-news.

No need for LDAP?

[jackalope]

It seems that the author presumed that the only use of LDAP is to provide passwords for user authentication. While that is a common use of LDAP it is not the only use.

It would seem that having a file system that is LDAP aware could be extremely useful. Imagine if your LDAP tree were reflected as a tree in your file system. You wouldn't need to embed LDAP calls in your application, it would just be data in your file system. So looking up an attribute for the current user, or a user, would be as simple as reading a file that holds the value of the attribute.

Re:How about fixing the current filesystems?!

[spaic]

From the Linux-ntfs FAQ

3.8 How was the Linux NTFS Driver written?

Microsoft haven't released any documention about the internals of NTFS, so we
had to reverse engineer the filesystem from scratch. The method was roughly:

1. Look at the volume with a hex editor
2. Perform some operation, e.g. create a file
3. Use the hex editor to look for changes
4. Classify and document the changes
5. Repeat steps 1-4 forever

If this sounds like a lot of work, then you probably understand how hard the
task has been. We now understand pretty much everything about NTFS and we
have documented it for the benefit of others: http://linux-ntfs.sourceforge.net/ntfs/index.html

Actually writing the driver was far simpler than gathering the information.

Some of these ideas are VERY short sighted

[irw]

I wish people with clever ideas to redesign POSIX namespaces would spend ten years in system administration first so they realise what's involved with managing REAL WORKING SYSTEMS.

Some of the ideas might well lead in useful directions, but some (at least as described in the paper) are plain silly. viz:

1) with overlayed mounts:

suppose my home dir is mounted read-write over a read-only system root, and I do not have a "/bin/prog" in my home dir. Consider:

cp /bin/prog /bin/prog

First time, it copies the system /bin/prog into my home fs - Counter-intuitive to the path semantics. If I run this a second time it copies my copy of /bin/prog over itself - Inconsistent.

2) Attributes in the namespace

We have a rather carefully written setuid chown/chgrp/chmod replacement which can be run by users in an "admin" group, and allows devolution of 1st-line support tasks to nominated users. It won't touch files whose uid/gid is 100, so they can only touch non-system files.

If attributes (file uid) is file/..uid and cp is supposed to handle what chown does, the above breaks big-time. We now need a custom cp replacement. Either that or we have to add an ACL for the admin group to every file we want them to manage, which is a great deal of effort, and likely end up inconsistent.

Contrary to the paper, setuid and PARTICULARLY setgid is NOT going to go away in the real world any time soon, as far as files are concerned. Ports less than 1024 are a different matter and I agree with the document.

3) Consider the number of file descriptors involved if /etc/passwd becomes a hierarchy of files. Just logging in one user will involve multiple open()-read()-close() operations. Whilst these might be efficiently implementable at fs-level, it is still very inefficient in user space, or will at least require a dramatic rethink of unix tools.