Slashdot Mirror
Ask ReiserFS Project Leader Hans Reiser
Hans Reiser leads a successful Free Software project that has attracted plenty of attention, many users, and even that Holy Grail of so many who have started their own Free or Open Source projects: Big-time funding from DARPA, SuSE, and others. How did he do it? What's his advice for other project leaders? Ask him! And ask him any other question you have in mind. Please stick to one question per post, and avoid questions that can be answered with a few minutes' worth of research. We'll publish Mr. Reiser's answers as soon as he gets them back to us.
When you look at your Linux installation scripts today, you see that ext3 is all over popular distrobutions like RedHat. ReiserFS used to be the most popular journaling filesystem for Linux, but how will it shape up given the mass adoption of ext3?
--------
Free your mind.
What specific benefits would I see by switching my box to ReiserFS as opposed to EXT3 or one of the many other journalling file systems that Linux now supports? At this rate, seems like I could be reformatting my hard drives every week.
Now you just need to hire someone to desire a modern, more "commercially pleasing" website. =)
My main question(s) is this(these): Is ReiserFS tunable to reduce usage of a laptop HD? If not, will it? As is, it's great on low capacity (under 1 gig) drives (I'm not sure how much overhead it imposes in comparision to ext2/ext3). However, it also likes to keep the drive active, which drains laptop batteries.
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
Having obtained financing for the project, how does that impact the future direction of development? How do you balance the interests of developers, users and sponsors to choose which updates to pursue?
Stop by my site where I write about ERP systems & more
You've probably noticed that there are quite a few journaling file systems for Linux and other Open/Free/Unixie operating systems that are fairly open or completely open source. Have you ever thought about getting together and making "THE" journaling filesystem for these free OS's? I know that open source is all about choice, but as you probably also know there are certain things EXT3, XSF, etc. can do that ReiserFS cannot. If their could be a hybrid made from the big journaling filesystems would you be part of it?
Ignore the "p2p is theft" trolls, they're just uninformed
Although its speed and (otherwise) good level of data integrity is of great interest to my employers, before I can recommend it to such a large company, I would like to know how and when you intend to put in a fix.
Comment removed based on user account deletion
What was so lacking in other file systems that you felt compelled to write your own? How do you think yours compares with the others?
I've been a fan of Reiserfs ever since I started using it, and I've been very impressed by its speed. I'm really anticipating Reiser4, and I hope it'll get merged into 2.6. However the last I heard about this is that Linus hasn't read your docs yet, and the status page on kernelnewbies shows Reiser4 to be "pending". Can you give us any insight on what's going on?
"Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it." -- Linus Torvalds
Can you clarify your long term goals with respect to ReiserFS and file system design in geek lay man's terms?
I'm refering to your Future Vision paper in particular. How would this kind of system be used on a day to day basis?
Right now any mention here of adding database like functionality to a file system is met with crude comments about SQL... Some explaination would really help.
cheers,
--
Simon
As I understand, what ReiserFS offers that "enterprise-class" journaling filesystems like JFS and XFS don't is the ability to treat the filesystem more like a database and less like a tree, but I haven't really heard of anyone getting serious about using this functionality. Why did you bother to add it, who do you think will get the most use out of it, and how would you promote it, given the chance?
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
I've used reiserfs for years and was always very satisfied with its performance and stability. Recently, I convinced myself that ext3 had gotten as stable as reiserfs. So I switched to ext3, mainly because I ext3 == ext2 + journal, so should the need arise, I can mount my ext3 partition as an ext2 one. I haven't run any test on my new ext3 partition, but they look like they're doing okay and ext3 behaves well in case of a crash.
:-)
So fo me, today, in terms of usability ext3 == reiserfs + ext2 backward compatibility. As much as I hate to admit it, I have now given up on reiserfs.
So my questions are : can you compare reiserfs and ext3 in terms of performance and reliability, i.e. are there good reasons to keep using reiserfs ? Do you think ext3 might become the de-facto standard for Linux journaling filesystems ? How would you feel about it, after all the work you've done on reiserfs ?
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
Beside the finding and organizing files, the biggest problem for desktop users today is probably that changes on the file system are not recoverable. It is easy to accidentally overwrite a file and lose your work, and the only only sane way to solve these kinds of problems would be to make it possible to revert changes.
Several research systems have been created, like the Elephant File System, but none of them made it into the mainstream free and commercial operating systems. Are there any specific reasons why nobody offers recovery (high complexity in implementation, very bad effect on performance, etc) or is it just because FS designers don't see the need for it?
We ran reiserfs for a short period a few years ago because we were saving data to 10's of millions of small files, and reiser seemed like a perfect fit. We had to stop however because as we added more and more files, disk access slowed down more and more. It eventually became almost unusable and we had to switch back to ext2. How much faster is it now than it was then? Have you completely solved this problem?
(1) Software project management techniques seem to vary by discipline. Some groups follow the traditional âoewaterfallâ(TM) model others follow Rapid Programming models â" Do you try to follow a particular style? (2) Classic project management spends a lot of time accounting for resources. I can imagine that an open source project has people signing-on and dropping off. How did you keep assignments & scheduling under control? (3) Have you come up with any tips or tricks that work for managing your project that might be useful to other software development Project Managers?
In your Future Vision white paper, last modified in January 2001, you outline several very interesting ideas about metadata.
Several developements have taken place since : the extensible attributes of BeFS has been burried with BeOS, the database-like metadata of Longhorn (aka Yukon) may actually be a separate layer from the filesystem altogether, and Apple is also moving all metadata out of the filesystem to XML files shared between applications (see iLife package).
My question : What is your current take on the metadata debate ? Do you still think the filesystem is the right place to handle metadata ? Any predictions ?
As a leading expert on linux filesystems I was wondering whether you were aware of any snapshot implementations for linux similar to those found in Network Appliance filers. Since working in a shop that used Netapp's I have always found myself wishing for the admin time saving feature of snapshots without the high cost of a filer. Veritas's snapshoting is whole volume only as are the other solutions I have seen outside of Netapp, is this a technical hurdle that no one else has been able to jump or is it a patent issue? Thanks for your time.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Will dynamically-loadable filesystem plugins be feasible, or practical? What scope will the plugins have? Encryption/decryption is obvious, but what about more interesting things, like scan-on-access Antivirus, or perhaps a mail spool could be mounted with some kind of spamassassin plugin.
- Minimizing Inner/Outer fragmentation
- Maximising locality of reference
- Minimizing the cost of file extension
I'd like to know how Reiser FS approached these conflicting goals. Has Reiser FS used any mathematical models to evaluate long-term stability/performance?Reiser FS is already a pretty mature, stable, usable product. Once V4 is done, is there really much work left to be done on ReiserFS proper? Do you have a giant to-do list that'll keep you and the guys occupied for years, or do you intent to work in a diffent direction (SAN, networkFS, databases, etc.)?
(or perhaps you'll just retire to Portugal and play lots and lots of golf)
## W.Finlay McWalter ## http://www.mcwalter.org ##
When you began a file system project as a free software project, you must have known that (assuming it worked) it had the potential to turn into a big project. How did you determine how long to work on it as your own project before making the first release? I imagine there must have been a strong temptation to just get it "out there" knowing its potential, yet certainly releasing too soon would make it look unprofessional and thrown together.
One question I always have with regards to successful (meaning funded, wide acceptance, large user/developer community etc.) is how did you raise the awareness of your project to get it from just a side project to something that it is today?
Did you use traditional PR techinques, or just through a community of connections?
sad robot making broken music
Apologies in advanced for a simplistic and obvious question but...
What's the best way to get funding for a project, assuming the project is at some sort of usable prototype stage already (i.e. who should you approach, what ground work should be done)? How should one court potential sponsors? Is it better to wait for them to come to you, or should you present a formal funding request in a format similar to that of a traditional business plan?
From what I have understood of file systems, data safety is priority number one. Considering that, are there any super speed algorithms you have dismissed because of "unsafe" behaviours?
On the ReiserFS 4 page you say that the new filesystem uses algorithms allowing the FS to do transactions only databases could do previously. Do you envision ReiserFS 4 being something comparable to MS' "SQL-FS" plans in Longhorn? How would ReiserFS 4 affect regular (oracle, postresql, mysql) database projects/companies?
Clearly Unix filesystems are moving in the direction of extended attributes. This isn't necessarily a smooth transition for Unix, where many utilities are modeled after streams of bytes.
I personally have pondered about this a lot. What do you think needs to be adapted and why? Should the VFS be changed? Should traditional Unix tools like "cat" be adapted somehow, or given attribute-aware equivalents? Should file attributes be accessable at the file descriptor level? Will we need a new "tar"? What should be done to standardize these attributes, or make them compatible across filesystems?
The directory layout of the typical unix filesystem has grown over the years to what has become the FHS 2.0.
If you were in the position of designing an entirely new FHS, more suited to ReiserFS (lots of small files, database-like access, etc) and without backward compatibility, what would it look like?
This is your sig. There are thousands more, but this one is yours.
In your future visions paper early on you talk about Reiser's Rule of Thumb #2. However, I can't find Reiser's Rule of Thumb #1 -- what is it? Is it a secret? Does it contain the sum of all human knowledge?
TELL ME!
Carpe Daemon
I've spoken to a few people (not naming names) who appear convinced that the design of ReiserFS is fundamentally flawed, and that it would lead to inevitable deadlocking. I've read your white papers now several times (took me a while to figure them out), and think the work you're doing is great, but a few smart people seem to think it can't be trusted. How would you reassure those people that their data is safe?
the redmond hegemony lost the internet protocol battle and decided to fight back with semantically rich but proprietary file systems (think 'halloween memo', 'longhorn', 'groove'). are you thinking about how to mutate ReiserFS to support arbitrary object attributes, full-text indexing, callbacks, namespace syntax for queries, etc?
Mr. Reiser, first off I have no complaints about ReiserFS (which is a high compliment), I use it on almost all my machines, except a couple are running EXT3 because they're not heavily used and I'm lazy at times. But thats neither here nor there.
You fall into an interesting subcategory of project managers or whatever you want to call them. I'll call it the "outspoken genius" category (even though the first word might be understated and the last is probably hyperbole). Basicly your work is technically interesting, applicable, etc. That's a give in. But there are quite a few people who have personal issues with you and your manner and usually cite some exchange or another. Sometimes this is the basis of an argument to reject the use of your work, which I think is somewhat silly. You're not the only one, and certainly not the first to be interviewed here.
So what do you think about this? ie. Do you think you made interpersonal mistakes that landed you here or do you think you've been misunderstood? Does it bother you? Why do you think people enjoy egging on folks such as yourself and then citing the moment you get annoyed with them? Do you think this question ever has a prayer of being moderated higher than someone following the method of the previous question?
Jeeze, I realize I just wrote an essay question in the style of one of my old Philosophy professors. You know the kind, here's a statement now write some stuff (I guess I'll give you a few ideas of where to go).
I recall a discussion at lkml about the licensing issues with bitkeeper. Someone said that the future versions of your FS will have some version control built-in, and thus be a sort-of competitor to Larry McVoy's bitkeeper. Larry's comment was that if that happens, then you have to buy bk in order to use it.
You suggested that indeed version control is going to be in. Can you say more about these plans now, and the eventual conflict with bitkeeper.
Thanks.
I'm preparing by reading everything I can find: I'm working on Tanenbaum & Woodhull's "OS Design & Implementation"; I've read "Design and Implementation of the Second Extended Filesystem"; Steve Pate's "UNIX Filesystems" is waiting on my shelf; and of course, there's the FAQ and ReiserFS v.3 Whitepaper at www.namesys.com.
Specific questions: what branches of math are useful in this line of research? Any books, articles, etc., that I haven't listed that are a 'must read' or 'should read'? Those who have succeeded in building a better filesystem: what have they done that I should also do? Any mistakes I should avoid? Anything that no one told you about filesystems that you wish you had known up front? And are there any special tricks (above and beyond mastering your subject) to getting hired in this field once a degree is in hand?
Thanks!
Having seen the BeOS development APIs, Reiser4 strikes a familiar cord with its metadata handling and querying. However, while BeOS had a captive audience (there were no BeOS clones/work-alikes), you are making ReiserFS a part of an established operating system and all of the inertia that "establishment" brings with it.
Where BeOS failed (among other reasons) because of the perceived lack of applications, do you have any thoughts on how to seed development for ReiserFS.
Case in point, where in other filesystem setups you would have a mail spool and/or mailbox as a custom file format (and indexes) on top of the filesystem, Resier4 allows for custom queryable file attributes (metadata) and the ability to efficiently store each message in the queue as a separate file. The mail handling program becomes proportionally less complex as that complexity has been shifted toward the common filesystem layer.
That said -- mostly for the sake of folks that haven't been following Reiser4 development -- do you see a way of encouraging Reiser4-aware programs? After all, it's a chicken and egg issue. Without common usage of Reiser4, application developers will be less inclined to commit their time to it (perhaps?). Without applications that take advantage of its unique features, what reason does the user have to put it on their workstation or server?
Are we looking at custom Linux distributions with this in mind or just piecemeal applications? Personally, I'm looking forward to dropping Ogg Vorbis files into a directory and, through the use of plugins, having their ID3 info automatically searchable. Hmmm... Where was that Fugees song again?
- I don't need to go outside, my CRT tan'll do me just fine.
There seems to be a lot of FUD against reiserfs being spread by a small but vocal minority of people.
Among other claims I have heard by the FUDsters - 'no large production systems (eg terabyte or larger) trust their data to reiserfs', 'xfs is superior to reiserfs because it has been well tested on irix', 'reiserfs has chronic data corruption problems', etc. Spin tactics that would make microsoft's propaganda teams proud.
How do you respond to the FUDsters? What claims are bogus, and what valid objections to reiserfs are there? (The only legitimate problem I can think of - lack of full data journaling - is shared by xfs and jfs as well).
There also seems to be a resistance by commercial distros to adopting reiserfs.
What do you attribute the resistance to?