Slashdot Mirror
State Of The Filesystem
Skeme writes "Have you heard of Plan 9 or Reiser4 but don't know much about them? Are you curious about the improvements free software is making to its filesystems in general? Read my summary of the current developments in the filesystem: namely, what improvements we can expect (a lot), and what Linux and the BSDs can do to improve on the filesystem."
will ReiserFS be ready for the 2.6 kernel? Just curious.
Any sufficiently advanced libertarian utopia is indistinguishable from government.
As far as i understand this.
The concept of reducing primitives is a good one, and based in sound mathematical theory. As already pointed out though, you need some container format that can handle some of these new ideas, things like very small files, files as directories and so on. This is needed, because when you transfer files through lossy mediums like the internet, or older filing systems, you don't want to lose the structure.
As far as I know, there isn't a container format that can do this. Tar is showing its age already, I wouldn't like to see it hacked yet again. Zip is alright, but you'd need to break compatability to add in all those extra features, and then it wouldn't be zip anymore. It'd also be inefficient.
So, what I propose is rather than reinvent the wheel to solve this problem, we add support for "boxing" to the Linux kernel.
A box is a filing system in a file. We already use them, to some extent - it's been possible to mount ISO images using the loopback filing system for a while. What's needed is to take this to the next level. The first thing is that we need the ability to use files as mount points, not just directories. When files and directories are the same, well, I guess that should be easier.
The .box file format simply contains a short header with some useful metadata, like maybe a checksum, and the filing system it contains (maybe that isn't needed). The fun part is the UI. What you need is the ability to right click on any dirfile (for want of a better term) and choose the "Box it" option. You'd need a better label. What essentially happens then is that the heirarchy below this point is sucked up and transformed into an ISO containing perhaps a "Reiser4-Lite" filing system. You can forgo the journal and other things that are redundant purely for storage.
The user has then converted their file or directory into something that can be transferred across the net, on Windows compatible CDs and so on, without losing the inherant structure of the original.
At the other end, choosing the "Unbox" option mounts the contents of the box using the loopback FS, mounted at the point of the file. To the user, it is seamless, far easier than zips or tarballs.
Of course, there are lots of complications. You have to agree on the format to use inside the box, for one, because the need to have kernel mods and so on makes it more complex than just installing tar.
I think MacOS has something a little bit similar with disk mountable images (.dmg) files, but the MacOS filing system is rather poor, and I don't know how easy it is for users to create them. Also the OS unfortunately applies some magic to them - for instance Safari will automatically extract the contents of the DMG file then destroy it when you download one (but other stuff does not, oops).
Anyway. That's one way to prevent loss of vital structure when transferring across lossy mediums, that I can think of. There are probably others.
Yes. They don't have access to the NTFS specs. Also, NTFS is a very complex filing system, with many different versions. You don't want to get that wrong. Resizing was a more important goal, and that has been working for many months now.
Of course, it might be included in all distros when completed anyway, due to patents MS hold on the technology.
Uhh... Informative?
Whatever.
the file is "..owner" not "../owner" you aren't going backwards out of it.
I'm surprised at the negativity of some of the comments here, moaning that POSIX semantics are perfect and nothing else can possibly be countenanced...
Plan9 namespaces and Reiser4 really do bring a lot more to the table in terms of useful expanded semantics and utility than all the posix filesystems. Posix extended attributes are very limited, and some filesystem implementors seem to be keen to implement them in the most restricted way possible ( eg size limitations in ext3).
The annoying this with Reiserfs is that the VFS will lag it by a few versions, and very very few apps will make any use of its special system call. Sigh. We'll be stuck with databases in a file for a long while yet.
One thing I would like to know about reiserfs is how attributes are attached to directories? If they are just small files in the "directory" bit of a file, what distinguishes them from children of the directory? Or are attributes just banned from dirs? Seems limiting.
Nobody (apart from perhaps this guy) has ever claimed that this syntax will actually ever be used, or needed. There are other possible syntaxes available, and in fact one long term blue sky plan for RFS is to allow many different types of syntax within the same file path, including for instance things that vaguely resemble database queries.
So, don't get hung up on the syntax given in this article.
We had plan9 machines here 10 years ago...
I don't think any exist anymore, in fact I don't even think the inferno install works anymore.
But anyway, it isn't a "new advance" anymore.
I've always wondered how these filesystems with metadata handle transferring files between different systems
Usually it doesn't. Take for example VxFS (Veritas File System); they support attributes like preallocating space for a file or forcing a file to be contiguous. When you move a file that uses these attributes to another type of file system (VxFS to UFS) you lose them since the target file system doesn't support those attributes.
Another example is ACLs. If they are in some way propriatary they won't transfer and sometimes won't work well together. The situation I ran into was a HP-UX NFS server's ACLs not working correctly with a Solaris client. The server used and enforced the ACLs however the client couldn't view or modify the ACLs on the files that were stored on the server.
If the implementation is truly at the file system level there's nothing you can do. However, as stated below, if there is a layer above the file system that handles metadata then you can more than likely keep the metadata intact.
3.8 How was the Linux NTFS Driver written?
Microsoft haven't released any documention about the internals of NTFS, so we
had to reverse engineer the filesystem from scratch. The method was roughly:
- Look at the volume with a hex editor
- Perform some operation, e.g. create a file
- Use the hex editor to look for changes
- Classify and document the changes
- Repeat steps 1-4 forever
If this sounds like a lot of work, then you probably understand how hard thetask has been. We now understand pretty much everything about NTFS and we
have documented it for the benefit of others: http://linux-ntfs.sourceforge.net/ntfs/index.html
Actually writing the driver was far simpler than gathering the information.
9p is a protocol. the file systems in Plan 9 are venti and fossil
9p in itself is worthy of including in the linux kernel, if only the guys there would do it right (their track record isn't too good with Plan 9 things).
More about 9p dould be found in section 5 of the man pages
> If you think of each file and directory and associated metadata as a
> record, with directories being pointer records to other records that
> show relationships.
Sorry to be snippy, but what you're describing is an *hierarchical*
database. But you're right in your main point: an FS like ext2 is an
example of an hierarchical database.
Posix file systems aren't required to be hierarchical they just of sort of look that way. Remember hard links? The name space is hierarchical but files and directories can appear in many locations. link (no ln) has been able to create hard links for directories since Unix version 2.
To create one, use dd to get the size you want (from /dev/zero) or copy from some real partition (ie, a floppy or a hd), run the desired mkfs if it was empty, and mount with the proper options. I have been using this to check ISOs and to clone floppies and store them into CDs for ages. I have used it to get perfect FS images too (machine updates, ie) and then extract the contents anywhere (eer, ok, Linux machines :) ). If you have access to a RH, check /sbin/mkinitrd, it is a script that builds the initrd in a similar fashion.
BTW, Linux also supports mounting dirs into other dirs, with -o bind.
Windows support for metadata has always sucked, recognised by every Mac user who moved to a PC and discovered that you had to tell the system what a file did by appending a clumsy tla to the end
As opposed to Mac OS, where you have to tell the system what a file does by setting an even clumsier 4-character code hidden deep within a file's metadata?
and passing gently over the inconsistencies of the support for long and short filenames.
What inconsistencies? Correctly-written modern applications (say, those written after 1995) support long filenames. Older apps do not, but you can still use files with long names with them by using the equivalent short filename.
The 8.3 compatibility layer is a benefit for those with legacy apps.
Moving a directory from Linux to a Windows system may be possible but the programming to do it will become increasingly painful and the risk of data loss will rise.
And whom would that benefit? Possibly, Linux. But right now, Linux isn't facing a situation where customers are threatening to migrate to Windows. Linux vendors are more likely to lose customers they don't even have yet, because who wants to move to a system they're locked into and can't back out of?
It's hierarchical because they inherit the fields of the levels above, like permissions and path.
As the poster noted, it's possible to run your filters in user space, not kernel space, so buffer overflows and segfaults should be sources of irritation, not sources of vulnerabilities.