File System Round-Up Interview

Little Sheep writes: "An interesting round-up interview regarding modern Linux filesystems is published by OSNews, featuring the developers behind IBM's JFS, ReiserFS and SGI's XFS filesystems."

Comments

Hmmm...

[mirko]

Not much about Ext3, even though Redhat seems to prefer it to the others...

Re:Hmmm...

[Billly Gates]

Perhaps this might explain it. I don't know about you but I think I will keep reiser away from my mission critical servers for now, regardless if this is just a flame war.

netbench recent results

[johnjones]

there are results from
http://lse.sourceforge.net/benchmarks/netbench/res ults/august_2001/filesystems/raid1e/README

I quote

Hello all,

I recently starting doing some fs performance comparisons with Netbench
and the journal filesystems available in 2.4: Reiserfs, JFS, XFS, and
Ext3. I thought some of you may be interested in the results. Below
is the README from the http://lse.sourceforge.net. There is a kernprof
for each test, and I am working on the lockmeter stuff right now. Let
me
know if you have any comments.

Andrew Theurer
IBM LTC


seems that its all pretty much rocking and they turn out the same ish even tho they do things differant except riser which sucks and alaways has in my eyes (each to their own)

regards

john jones

Re:netbench recent results

[jsse]

Amazingly, in the article Hans Reiser quoted another benchmark tests which show Reiserfs is superior.

The results too perfect to believe.......may be I'm thinking too much....

Have to give credit to Hans

[wowbagger]

Of the three interviewees, Hans knew more about the other guys, he was better able answer what was better about the others then his, and how his was better than the others. The others came off as suits, he came off as an engineer.

Re:Have to give credit to Hans

[Tet]

The others came off as suits, he came off as an engineer.

From following the posts on the Linux Kernel mailing list, I've come away with exactly the opposite opinion. Reiser is the suit, and the others are the engineers...

BeOS' BFS uber alles!

[deusx]

I really wish someone would include BFS-style attributes in an Open Source file system. Hell, I really really wish my Mac OS X installation had it. Steve Best kind of dismisses the Live Queries it as "similar to a change notification mechanism," since he admitted he wan't really familiar with it... but it's more than that.

In BFS, (although I'm probably going to butcher this explanation) the system actually retains the optimized parsed tree of the query, and monitors the modification times of the individual indices used in the search. When one of those times changes, the system re-queries just that branch of the tree rather than re-processing the entire query. Is very neat. Oh, yeah, and there's some notification going on.

Bah... I just really want a native file system with arbitrary indexed attributes so I can run SQL/LDAP-like queries against my non-Be machines too...

Re:BeOS' BFS uber alles!

[Adnans]

I really wish someone would include BFS-style attributes in an Open Source file system

It's called XFS :-)

XFS supports extended attributes just like BeFS. no wonder, since BeFS is based, at least in part, on ideas originated in XFS. You will need some userland tools to set extended attributes and probably a modified tar if you want to preserve attributes inside archives. The web page on Access Control Lists and Extended Attributes might also prove useful. Now, combined with fam & imon , which provide (i)node monitoring you have everything except indexed searching, all in Open Source (imon is actually a kernel patch that adds node monitoring at the VFS layer)

-adnans

Re:BeOS' BFS uber alles!

[deusx]

Hmm.. that might be getting close, though the indices with maintenance and searching are pretty integral to BFS, along with the whole query and live query system.

Though... maybe it wouldn't be too hard to work that into XFS. Most of what I know comes from undergrad Comp Sci classes and a few readings of the BFS file systems book.

Re:BeOS' BFS uber alles!

[fm6]

I just really want a native file system with arbitrary indexed attributes so I can run SQL/LDAP-like queries against my non-Be machines too.

I'm all for extended attributes. It makes a lot of sense to put the file system in charge of metadata. Grossly superior to using filename extensions.

Problem is, extensions are what people know, and what almost all Windows, Linux, and Unix apps are built around. Getting everybody to change won't be as hard as retiring the QWERTY keyboard, but it's up there.

And feature creep makes matters worse. Why does a filesystem need to support arbitrary relational queries? Why make every single app deal with complex access issues that only a few will ever use?

Re:BeOS' BFS uber alles!

[ReelOddeeo]

Why does a filesystem need to support arbitrary relational queries? Why make every single app deal with complex access issues that only a few will ever use?

Why would every single app be affected? If you don't want to use a feature don't.

On my Mac, I can search large file systems across multiple hard disks for a file with a certian name or attribute, very fast. For instance, find me all files that end in ".jpeg", are bigger than 3MB, and are in a folder that is locked. I can get results quick.

Much better than using locate/updatedb or find.

This seems like an advance to me.

Re:Comments...

[l0wland]

Just put your Forum-treshold to +1 and you won't notice any of them.

collaboration

[A moron]

Hans Reiser: I'd like to thank the XFS developers for taking the time to personally explain to me why delayed allocation is the way to go.

Now that's cool. Projects that appear to be in direct competition, but they all have great respect for each other and actually communicate with each other. And in the end, each product ends up being better.

Corporations take note.

Re:collaboration

[mikeage]

I admit I don't know if that's true or not... but I just can't shake the thought that this is not really true...
(-1 Insulting)
(+2 But it's to a big company)
(-3 They have an awesome employee server)
(+4 But they shut it down (reality.sgi.com)

Re:collaboration

[linuxpng]

perhaps you haven't notice the pure hatred Hans has for redhat. Ever opportunity he gets he criticizes redhat's choice for GCC (which is undefendable as far as 7.0 is concerned) and seems to be at war with Alan Cox. I'd hardly call it friendly.

Production Use

[Marvin_Runyon]

I think the clearest choices for a production environment are XFS and JFS, as they have many years of proven reliability.

Generally, datacenter environments demand reliability over speed, and a good track record wins the day in selecting technologies.

When we implemented our database backend, we decided to go with SGI's JFS based on years of production use. So far we havn't encountered any problems!

-Marvin

Sad technical loss

[sphealey]

One of the highest-performance, most managable, most securable file systems on the market for the last 15 years has been Novell's NCP file system for Netware (I am sure it has a name but can't think of it at the moment!). The current versions (Netware 4 and 5) are supurb techncial achievements.

Now, it seems pretty clear that Novell is doomed, and when it goes Netware and NDS will evaporate. I just hope that whoever turns out the lights in Provo has the foresight and generosity to release the details of those two technologies under some sort of open source license, so that even if the products disappear the technology might be saved.

But I doubt that will happen.

sPh

JFS in the kernel?!?!?

[Ignacio]

Yeah, right! Have you seen the code for it? It is a horrid mess. I tried digging through it, and soon found that I would have to rewrite it from the ground up. I'm not surprised LT et al. don't want it in Linux. It's a nice FS, but it's NOT ready to go into the kernel.

JFS pulled from Mandrake 8.1

[LinuxGeek8]

Interesting review.

On MandrakeForum the latest news about filesystems is that JFS will be pulled from Mandrake 8.1.

There was done a test with a buildup/takedown of 100.000 files.
In the case of JFS the deleting of those files caused a hard kernel crash.

Seems there is some work to be done, despite it being a 1.0 release.

And hey, what's up with this html here?
Seems only plain text works right for me.

Re:BeFS and new file systems

[Sloppy]

It tells us that it's a real neato piece of work that everyone wants, but that few could actually use, because it only ran under one OS (and it happened to be an OS that saw very little use as a server).

Filesystems, just like data formats and communications protocols, must be open in order to be generally useful.

(Novell's filesystem was able to sort of skirt this problem for a long time, because even though it was closed, it was implemented on a server, so that other OSes could "use" it without actually knowing anything about it. But they did their best to kill it anyway, by keeping NCP closed so that only a few platforms could use the server. If it weren't for sniffers, Netware would have become obsolete a lot sooner.)

reiserfs on 2.4 vs 2.2

[TheGratefulNet]

I've been running the latest reiser on the 2.4 series, both at home, at work and on my laptop (especially laptop, since the battery can go out at any time with little notice. fscking a 32gig notebook (slow) drive sucks...)

I had some problems I thought were 2.4 related so I tried the 2.2.19 kernel with reiser. it didn't seem to be able to locate superblocks when it booted up and just froze ;-( so I had to go back to 2.4

this is the only downside I see. I know 2.2/2.4 reiser should work but it didn't for me. and I think I did all the right things when building the kernel AND userland tools.

I've not tried xfs and I did have some hangs with jfs when it reached 1.0.0 - but that was while on an IDE system with the infamous VIA chipset. I'd probably blame the chipset bugs before JFS, but it did shy me away from JFS.

reiser seems stable on 2.4 and I'm quite happy with it. give it a try.

Re:reiserfs on 2.4 vs 2.2

[TheGratefulNet]

thanks for the info - I just assumed it was backwards compat. my goof.

now, if there was a convert util that did the re-mkfs in-place, I'd be ok with that. but I'm not about to reconvert (by hand) my entire set of reiser partitions. oh well - guess I'm stuck with 2.4 then.

the reason is small files

[johnjones]

hey no those results are right you just have to put them into context

Reiser is desgned for large amounts of Dir and files which is what this tests for

and yes it achive very high marks but I have yet to see a TB on a Reiser in a live platform yet I see one every day for XFS (streaming video) and I know someone who has AIX with 3TB on it

most large files seem to be of the video nature or are database yes while MP3s are comman and thata is what Reiser did (remember mp3.com was a sponsor and came up in the boot up) but I have to say round here most peoples dir do not contain over 1000 files most have some MP3s and video, documents and the like

really I have been running XFS on a i686 for a while and have not had anything go wrong (we havent exactly pushed it tho)

what XFS and JFS need are ports to other archs and JFS seems to recogise this

remember benchmarks are written to test certain things but we commanly relie on quantum chaotic things and are unable to test for this (because they dont really have random things)

Metadata APIs?

[Sloppy]

One thing that pleases me about this, is that it looks like all three of these filesystems have (or will have) metadata support. This has been a pretty serious (imho) weakness in traditional Unixes. (BTW, isn't it interesting that the platforms with the best GUIs (MacOS and OS/2 WPS) happen to depend heavily on metadata?)

I haven't used or programmed for any of these filesystems so far, though, so I'm wondering if the APIs for getting at the metadata stuff, are all the same. This is something that will be absolutely necessary before Linux app writers will be able to start using this stuff.

It looks like the teams have good attitudes toward one another. I hope they're coordinating on the APIs so it'll all be consistent.

Re:Metadata APIs?

[spitzak]

ReiserFS is supposed to suppor this.

However I feel that metadata is a bad idea. The problem is that one of the main things done with a file is to copy it from one place to another. In fact with the web, I would say that non-system files are copied hundreds of times more often then they are "used".

This requires the metadata to be encoded and transmitted with the file. These encodings tend to be complex, kludgey because they are never designed for completeness, and require both ends to understand the encodign. And the end result is a stream of data that most programs (all those concerned with copying files) will treat as the file's data anyway.

There is also the annoying fact that all programming languages need functions added to them to manipulate the metadata.

So how about imbedding all that metadata into the file's data? I have thought that a system could be designed where a file is *only* a block of data. Even the file's name and protection is imbedded in the file. A filesystem would be designed so the first 1K or so of a file is very quickly accessible. There would also be simple rules so that the data could be located and it would appear as comments to whatever reads the file.

Since anybody with write access could change the permissions, permissions would actually be some and/or combination of that file and all the parent directories. This is the only complex part I see to this.

Re:Metadata APIs?

[Skapare]

The very fact you might need a new API would be one major reason why metadata is a bad idea. While the concept of having metadata is not in itself bad, implementations of it that require special access are. For example, if I have a filesystem with metadata that uses special API system calls, how will I be able to backup and restore that filesystem with the metadata using ordinary tools like tar?

A simple way to have metadata is to define another file. Just append ".meta" to a filename, or replace its extension with "meta". You can now read and write as much metdata as you want to have. Then any filesystem backup or transfer tool can do the job.

Re:Metadata APIs?

[ReelOddeeo]

filesystems should be for named unstructured data.

I agree that the data part of a file should not have any structure known to the filesystem. The structure of the content of a file is unstructured.

But I disagree. You said "named unstructured data". The name is metadata. There seems to be no disagreement that the name is useful. (C'mon, why can't we just all use inode numbers and do away with names?)

The metadata is to make files easier to find, and to store more information about what is in that unstructured data.

We do need to know what format the data is using, so the only additional peice of information that I need that is not in current metadata is the MIME type.

You mentioned filetype. I agree. But I don't think we should stop right there. I think "creator" is another good one. (i.e., which application will launch when I click on this file? e.g. multiple "jpeg" files might launch with different applications, and accordingly display different icons.)

Another good piece of metadata about files is their X,Y location within the window that displays them. Think about this. The Macintosh is the only OS I know of that has a geographic memory of where you put a file. I can remember, "oh, yeah, when I open that folder, I left my business plan in the mid-lower-left corner of the window."

There are other useful metadata as well.

This does not mean that existing apps have to be changed at all. On the Mac you can still write a C program that is totally ignorant of the Mac's extended capabilities. You end up with users saying "why is this program so stupid? why do all it's files have plain white icons?" But the program works. Your C program doesn't have to set the X,Y location of files, the system does that, when the user moves the icon for the file.

On Mac, every file and folder can even be given a custom icon that overrides the icon the file would display based on the combination of it's type/creator. This is another good use of metadata. In fact, it begs the question, if the fielsystem can associate metadata with a file, then why not just make the metadata completely arbitrary? The FS doesn't need to know what attributes KDE or GNOME want to tack onto a file. A window manager could tag a file as having a certian "X" attribute, and certan "Y" attribute. I custom icon attribute. etc.

But you could still type: vim mypict.jpeg, and open a jpeg file using a non-aware application.

Just because the filesystem supports extended attributes doesn't mean you have to use them. Nor do you have to run a GUI that takes advantage of them. Nor do you even have to run the particular filesystem that has extended attributes.

Some of us want better GUI's.

Re:Metadata APIs?

[Sloppy]

if I have a filesystem with metadata that uses special API system calls, how will I be able to backup and restore that filesystem with the metadata using ordinary tools like tar?

Well, unfortunately, you can't, unless you modify tar. (This isn't all that far-fetched; I used to use an OS/2 port of tar that saved and restored extended attributes.) Of course, that only works when tar knows it's working with files, and the approach falls on its face when you're just doing streams. So yes, there's a problem.

Just append ".meta" to a filename, or replace its extension with "meta".

This does mostly work, and it's what my Amiga does (.info files), and OS/2 does something a little like this when you try to use extended attributes on filesystems that don't support it (e.g. FAT). The main problem with this approach is that the files can too easily end up getting seperated from their metadata. You still need non-standard (from a Unix point of view) tools that know that when they copy a file, they have to copy the file's metadata too. "cat foo > bar" isn't going to copy foo.meta.

Tux

[johnjones]

this looks to be a great system

I use a nettapp all the time and a filesystem that had the same sort of funtionality would be great

I heard about this in the write up for linuxconf.au but heard little about it since

regards

john jones

http://people.nl.linux.org/~phillips/tux2/

References on filesystem design?

[torpor]

One thing that's always interested me is the technology behind filesystem designs - are these guys operating on any references, that might be worth studying? "Filesystem design 101" sort of books?

Anyone know?

Re:References on filesystem design?

[be-fan]

Practical File System Design by Dominic Giampalo.
Great book, has a detailed overview of the BeOS structure, some comparisons and explanations of other filesystems (like ext2, NTFS, XFS, and HFS+), and a cool file system builder kit with full source to pretty nifty example filesystem.

Re:ease of and quickness of use

[Billly Gates]

My choice would deffinetly not be reiser. Its too cuting edge and imature for server use. This particular bug may have been fixed but in a server environment, reputation for stability needs to be there. Its not like I can bring the server down and download a patch to fix a fs bug in a mission critical environment. XFS has been around for years and would be my choice if I had to bet my job on a fs.

Re:BeFS and new file systems

[be-fan]

Except not really. BFS is very well documented, and some of its code actually appears in Giampalo's fskit. Plus, all of the data structures are cleanly laid out, so implementing a clone (eg. AFS) is very straightforward.

Re:All journall FSs useless on Linux

[Skapare]

So what is your solution?

All your journal are belong to us

[Skapare]

So when do we get to see a complete round up of all journaling filesystems (I noticed ext3 was missing), including comparisons of features (including implementation features) and benchmarks on a variety of hardware configurations (SCSI, IDE, USB, RAID)?

Re:BeFS and new file systems

[megaduck]

I find it interesting that BeFS is mentioned so prominently by each of the developers as a goal for an FS to aspire to, yet the OS itself has basically died even though it was given away for free. What does this tell us?

This tells us that no matter how technically brilliant your OS is, you need developers. We went through this before with OS/2. OS/2 was much better than Windows in all sorts of ways. IBM failed to get third-party developers on board. No developers = no apps = no users. When Windows 95 came out, it sucked but was universally adopted because Microsoft was able to leverage their existing Windows developers.

Personally, I really wanted to see the BeOS succeed. It was fast, pretty, versatile, and thoroughly modern. I couldn't make the switch, though, because the apps and drivers I wanted simply didn't exist. From what I understand, this was because Be didn't work on cultivating those third-party developers that are so important.

For an interesting contrast, look at Linux. Linux is so darn developer friendly that we've got 3+ full GUI environments, 4+ journaled file systems, a bunch of web browsers, a couple of Office suites, and device support up the wazoo. We've even got some decent games (thanks, Loki!). Be didn't have any of that, despite their incredible technology. Again, developers = apps = users.

I'm really sorry to see the BeOS gone. It really was ahead of its' time. Fortunately, the whirlwind nature of Open Source development means that a lot of the good ideas in the BeOS will probably make it into another OS eventually. Good ideas never die, they just go into hibernation every now and then.

Re:Pity ext3 is the de facto standard

[ReelOddeeo]

For what it's worth, what RedHat says generally goes.

Really?

I tried RH 5.2 as my first Linux. Hated it. No KDE. Other complaints.

A friend pointed me to SuSE. Boy am I glad. I might never have given Linux a second try.

I've been using SuSE for years now, completely ignoring what RH says. I'll confess I've been using KDE for several years. And ReiserFS for awhile too. But don't tell RH. Or MS.

It's nice to have choice.