Slashdot Mirror
Interview With Reiser4 Author Hans Reiser
An anonymous reader writes "KernelTrap has an interesting interview with Hans Reiser, the author of two revolutionary Linux filesystems, Reiser3 and Reiser4. Reiser3 was the first journaling Linux filesystem. Reiser4 is a complete rewrite that is claimed to offer amazing performance and a new plugin architecture offering semantic enhancements to rival Microsoft's WinFS and Apple's Spotlight. Comparing Reiser4 to WinFS, Reiser says in the interview, "Reiser4 is a much more mature design, representing a 10 year effort"."
WinFS exists and is still in development. It's just not set to debut as part of Vista. Whether that means it will debut significantly after that I don't know, though. I think there's an alpha or beta version of WinFS available to developers now.
Honor Among Slackers. A veri
Here's a link to the page that hides the asshats making the pages super-wide with lame comments.
This one is pretty old review back when Reiser4 was still Experimental. More recent one would be here, but it too is over a year old.
-----BEGIN PGP SIGNATURE-----
12345
-----END PGP SIGNATURE-----
Anything more specific? I'm the first to admit that he can be rather immature, spoiled and inflammatory but a quick look at the link you offered showed none of these attitudes. Actually the discussion sounded quite civilized, so what's the problem?
Don't think of it as a flame---it's more like an argument that does 3d6 fire damage
rye-sir.
It's my last name, too.
There's a beta version available to developers now. There was a lot of criticism of it when it was first unveiled so they went back to the drawing board and released a new version that claims to address those concerns.
This is my sig.
XFS beta came out 9/22/2000 (its source code was first publically available on 3/30/2000) The first journaling version of reiserFS was release in november of 1999.
JFS came later than XFS though I can't find the date.
Of course officially ext3 came out before reiserFS in september of 1999, though ext3 is the real winner. Which produced workable code first I have no idea.
The german magazine iX ran benches on Reiser4 a while ago, and the benchmarks indeed were impressing with two huge downsides however, one is already mentioned in the interview, a reallocator is needed because Reiser4 has a tendency to fragment. And the other one being a much higer CPU usage than every other filesystem.
Now having read all of the ML discussion in the link GP posted I still don't see how it shows Reiser's immaturity. He may be a bit overconfident about his ideas but that's not necessarily a bad thing. That said I wanted to add that the link is a very good read and it is about features of Reiser4 that may be more important than raw speed (especially raw speed according to benchmarks on Reiser's own homepage =).
Don't think of it as a flame---it's more like an argument that does 3d6 fire damage
My understanding is that the kernel developers have pointed out flaws in the benchmarks and he has accepted the criticisms but points out that they are just benchmarks and all benchmarks have flaws. This would not be a problem if he didn't keep referring to the benchmarks when trying to ram a change into the kernel. You can't have it both ways.
It's also my understanding that the key reason kernel developers don't want to accept his patches is that they don't like big megapatches that affect many systems or replicate functionality that is already in the kernel -- it's bad for maintenance. It's also my understanding that he doesn't want to break up the patches himself and he has refused help from others who are eager to do it for him. For him, it's an all or nothing deal -- take it or leave it. The kernel developers say "fine, we'll leave it", but he doesn't accept their decision and continues to complain. Again, you can't have it both ways.
Reiser may be a genius, but even geniuses have to (*gasp*) live in the real world and negotiate with real people. Even if Reiser is smarter than all the kernel developers (doubtful), it pays to treat your so-called "inferiors" with respect. Even janitors and garbage collectors can have wisdom that we don't have and things they can teach us.
People who think they need to implement spotlight, HFS+ xattrs or Reiser should read Practical FileSystem Design (pdf) then just go away and use BeFS instead. :-)]
[Actually, the person who implemented HFS+ xattrs and worked in the Spotlight team was the guy who wrote Practical FileSystem Design, so I think that counts
There are 2 other significant players in the FS field that Hans doesn't mention:
XFS (from SGI) and GPFS from IBM.
GPFS has a different focus, but XFS seems to be aimed at solving similar problems as ReiserFS (scalability, high performance, journaling).
Simpy
It was not Hans who wrote that, it was Linus in response to Hans' claims that Reiser4 is important because it can compete with WinFS and Spotlight.
This is a type of question that, unfortunately, cannot be answered correctly. Well, it can: it depends. But that's not what you are after. As Hans himself pointed out, there are some fsync performance problems with ReiserFS. If you look at PostgreSQL config files, you'll notice a "fsync" setting, and if you look at pgsql-performance mailing list, you'll see frequent mentions of fsync. Obviously, fsync affects DBs (not just PostgreSQL), and ReiserFS may not currently be so great for DBs. However, it is apparently good for large directories (1 directory, lots of files in it). So, it depends how you use your FS.
Describe how you use your FS, and maybe somebody can provide good feedback.
Simpy
A new review available in the German magazine this month:1 0
http://www.linux-magazin.de/Artikel/ausgabe/2005/
Unfortunately not free...
Are you sure you understand what ReiserFS is suppose to do? What about WinFS? I don't think compairing WinFS to ReisierFS is quite like compairing Qt to Explorer. The functionality of WinFS is (as I understand things) a proper subset of the functionality of ReiserFS.
Again, there purposes are not really different. Sure, ReiserFS is a full blown filesystem, and WinFS is not a filesystem, however the functionality of WinFS is included in ReiserFS. The fact that they operate at different levels is the reason for Reiser's remarks concerning ReiserFS and WinFS. That's his point. WinFS works on a user level to provide functionality that ReiserFS provides at the filesystem level, and Reiser feels that this is a more mature design.
...interesting if true.
I never said the above words attributed to "Rieser".
I am sure of it becuase I would absolutely never say that "Linux kernel developers do what's right because it is _right_, not because somebody else does it.".
I am just not that nice a guy that I would say such a thing.:-/ I am guilty of saying the opposite at various times. I am known for this, and not particularly liked for it.:-)
This is a forged quote. Note the false defensiveness put into it in the sentence "So there's really no point in trying to push your agenda by trying to scare people with MS activities." That really sounds like someone at MS posted this.
It does not matter so deeply that MS put it into or out of the kernel, what matters is how they layered the code relative to itself --- that is, do they use the FS API, which lacks an insert or excise operation, to repack small objects that they squished together within a file, and does that layering make things slow. I think it almost certainly does make it slow, and it definitely is inelegant.
Hans Reiser
Reiser4 Architect
Namesys
``None of that has anything to do with the FS being in kernel or user space, there's no reason why I cannot create a FS module that allows people to mount SSH or FTP or anything else just like they can mount NFS or ReiserFS''
I don't know if you've actually tried this, but I have. Between the fact that the VFS system is poorly documented and all Linux kernel internals are subject to change without notice, writing filesystems for Linux is a total nightmare. And then you have to do the work again for FreeBSD, NetBSD, and OpenBSD.
Having things in userspace provides some form of shielding, and practice has shown that it lowers the barrier to implementing filesystems. Witness the great number of useful "filesystems" available through LUFS, FUSE, KDE, mc, GNOME, and others.
Please correct me if I got my facts wrong.
Sure, ReiserFS is a full blown filesystem, and WinFS is not a filesystem, however the functionality of WinFS is included in ReiserFS. The fact that they operate at different levels is the reason for Reiser's remarks concerning ReiserFS and WinFS. That's his point. WinFS works on a user level to provide functionality that ReiserFS provides at the filesystem level, and Reiser feels that this is a more mature design.
Just in case anyone else has been watching this all go over their head: WinFS wikipedia entry
Just fyi: there actually is a ssh filesystem called shfs. And from what I know isn't LUFS deprecated in favor of FUSE now?
Those are the words of Linus Torvalds in response to someone suggesting that Reiser4 should be merged, in order to stay competetive with WinFS and Spotlight. To counter the reasoning, Linus Torvalds stated the following seperate points:
1. WinFS is not the real filesystem, the real filesystem (NTFS) still runs in kernel mode. WinFS is "merely" a set of libraries in user-space, like gnome-storage. So, you can't derive a need to push such functionality into the Linux kernel.
2. Trying to push some functionality into the kernel with the reason to compete with Microsofts development won't work, because they do what they think is right, not in order to compete with someone else.
"Between strong and weak, between rich and poor [...], it is freedom which oppresses and the law which sets free"
Well, what I want to know is: How do I get to this metadata? ...Once there's an application which can find all pictures of my dog, or songs with piano in them, and store THAT in the metadata, which I can search somehow, call me.
I take it you have not actually tried to use any of these new filesystems and their metadata. Metadata comes from lots of places. It comes from an internet database of music CD and movie DVDs. It comes from the OS intelligently reading the text contained within various file types (like text, rtf, .doc, PDF, PS, etc. etc.) and extendable by a plug-in type architecture. It comes from applications who assign it based upon given criteria, or from applications that create files which are now starting to assign more and more metadata to those files. It comes from hardware, like when your digital camera or PVR assigns dates to files it creates. It comes from users inputing it by hand, like when they go through their vacation photos and add a description for each picture.
I use this metadata and perform searches on it every day. Why shouldn't I be able to do an easy search on my computer for every document, application, library, etc. that has the string "vpn" in it? Shouldn't I be able to find all references to MPLS in my files, whether or not they are in in text, .doc, .pdf, or some other file format? Shouldn't applications on my system be able to find and edit this data as well? Well, now I can (and they can) and I really, really like it.
For some reason you are looking at the current limitations of metadata, i.e. optical recognition can't reliably identify my dog, instead of the advantages, which is all the information that can be reliably searched. Maybe right now I can't search for all my mp3's with a piano in them, but I can automatically tag all the audio coming in over the mic I have attached to the piano with metadata that says it is piano. Now fast forward 10 years and suddenly all of your files have a wealth of automatically generated data associated with them. In 10 years I will be able to search for all the mp3's that have piano in them, because my audio mixing program labels all the files with input from the piano mic with the proper metadata and why not. For a few seconds work up front I, and everyone using my files, gets additional functionality. Now apply that to all files from all sources and suddenly metadata has greatly improved the computing experience.
Get with the times, metadata in the filesystem is here and it is very useful and it is becoming more and more useful every day.
The "unit bastardization" is the A:, B:, C: thing
It breaks the filesystem namespace, it's ugly, there's a nice paper from Rob Pike and Peter J Weinberg where they compare and explain different namespace choices for several operative systems (unix, plan9, dos, VMS): The Hideous Name
ext3 with the dir_index feature does just fine by me, although I'm not sure what kernel version it was added in, if at all (I've got it by default in my Red Hat Enterprise Linux 4 installs).
last week, I formatted my new USB hard drive with reiser4. It works well with both 32 and 64 bit (kernel and userspace), no problems so far.
A better question is: if you're a home desktop user what benefit does a journaling file system give you?
The thing is, Reiser ist not only about journalizing but also about performance because of stuffing things together in a more intelligent way. Therefore you don't waste harddisk space and the access of the data is also faster (especially with many small files).
Therefore you get benefits even on your home desktop
The quote is genuine, but it is from Linus Torvalds. Gee, he really believes that Linux kernel developers do things because those things are right? How'd have thought that?
we had problems with amd64 for up until this may when one nice company http://t-platforms.ru/ lended us dual amd64 mobo for couple months which was enough to get rid of couple annoying bitmap operation bugs....
... i guess it works... real evil is an oil war.
so
sshfs/ftpfs/anything you want-fs: take a look at http://lufs.sourceforge.net/lufs/
Fun stuff.
Roger
Do you have any better hostages?
yes, the native hpfs in os/2 did store the EA as part of the fs, not hashed off like it was for FAT.
... i dont this design choice (internal vs external storage of EA metadata) was not due to any inherent limitation in the orginainating platform at apple - because the HFS eventually supported multiple forks, which would have easily supported a 'slot' for something like extended attributes. But that was a while ago, so it's hard to say clearly.
man, those were the days - sweet sweet os/2.
surprisingly, while ibm made many useful improvements in the joint venture with apple on TALIGENT (eg the collection classes in SOMObjects were derived from taligent), one of the issues IBM did _not_ bring to the table at Taligent was it's meta-data experience with os/2.
this was a disappointing ommission.
to counter the perfidious microsoft 'traitors' inside ibm, os/2 was always looking for a "killer app" -- ironic that the building blocks were always right there in the fs staring everyone in the face.
a bit of trivia: surprisingly, the workplaceshell was not orginally mandated to use somobjects; thus the wps might have had to access the meta-data in the extended attributes directly. perhaps becuase SOM did such a wonderful job of abstracting access to the EA, developers never truly appprecaited its elegance & power.
i dont reacall how the BENTO-based compound storage mechanism for OPENDOC made use, at all, of the extended attributes; my recollection of the bento API is that each content model had its own semantics, privately represented inside bento - so perhaps there was no need/ability to publish this information externally in the core fs
Anyone have any more direct recollections?
cheers:dlf
Sorry, but ReiserFS v4 is a tiny subset of WinFS. I would say that the grandparents analogy is apt.
To clarify: ReiserFS adopts database concepts in two ways. It uses a B-tree variation to allow efficient hierarchical storage layout and it uses it to allow single-process atomicity. That is all.
WinFS on the other hand is an actual factual relational database (actually parts of MS SQL-Server) built on top of NTFS; a NTFS file is a special datatype among many others. One can specify relations and constraints as one would in a relational database. It has full atomicity (ACID), whereas ReiserFS's atomicity just gives a rollback local to a process. That is, any read or write may be transactionally inconsistent if there is more than one process dealing with the data whereas in WinFS one gets full support for transactions.
In addition to the above WinFS of course provides the obvious big end-user features; Fast queries over all the data in a relational way, and triggers that allow actions performed when data is changed or queried (both of which OSX provides in a slightly more ad-hoc way on the side).
So where does ReiserFS fit in? Well, the complaint that Hans Reiser has against WinFS is that not everything is a file, rather files is a special case mapping down to NTFS streams. Small auxilliary data is handled by other data-types in an SQL-like manner in WinFS, which to me at least seems like a quite sane approach. ReiserFS instead just optimizes the heck out of small files, planning to just take the UNIX principle to the limit, everything is a file, even if you just want to store a small integer. This makes everything fit into the classic filesystem namespace in a good way, but it is really less structured than the WinFS approach. WinFS relates whereas ReiserFS just provides really fast unstructured primitives.
Overall I don't expect ReiserFS to be overly relevant to the OSS answer to WinFS, mostly because I don't really expect a OSS answer to WinFS on that level. While Microsofts approach is a great piece of engineering is is also a huge amount of work. More likely is that the Apple approach will continue to be the important one; Just add the stuff needed to get the actually important end-user features today, work on the low-level programmer features later.
On the other hand ReiserFS v4 really appears to be very very fast. No doubt great features can be built on top of it, but on the other hand the classic VFS abstraction is a good thing. Swapping filesystems without impacting the userland is a feature that is really useful, so it seems fairly likely that the simpler approaches like Beagle will get to add the basic end-user features instead of fattening up the file systems (I highly disagree with Reiser that it is on the actual FS-level things should happen in the future). People who need databases can, *gasp*, use a database instead.
In fact, I am suprised to not yet have seen systems like Gnome and KDE fire up a stripped down PostgreSQL process or something similar to provide applications with an excellent database for whatever their needs might be. For Microsoft doing a thing like that hurts the bottom line (they do charge a fair bit of money for SQL-Server after all), but there is no reason why an OSS platform should not utilize the best technology it has available at every turn.
Oh well, getting off track. Summary: ReiserFS v4 is a tiny subset of WinFS feature-wise, it just does it very very fast.
www.namesys.com
How about the semantics described in The ReiserFS Future Vision Whitepaper
Regarding how to compute it, the standard algorithm traditional to boolean algebra works reasonably well, in which you take the intersection of all of the sets matching the subnames, starting the computation with the smallest of those sets.
Hans