Slashdot Mirror
Ask ReiserFS Project Leader Hans Reiser
Hans Reiser leads a successful Free Software project that has attracted plenty of attention, many users, and even that Holy Grail of so many who have started their own Free or Open Source projects: Big-time funding from DARPA, SuSE, and others. How did he do it? What's his advice for other project leaders? Ask him! And ask him any other question you have in mind. Please stick to one question per post, and avoid questions that can be answered with a few minutes' worth of research. We'll publish Mr. Reiser's answers as soon as he gets them back to us.
Where did you come up with the name for your project? :)
When you look at your Linux installation scripts today, you see that ext3 is all over popular distrobutions like RedHat. ReiserFS used to be the most popular journaling filesystem for Linux, but how will it shape up given the mass adoption of ext3?
--------
Free your mind.
What specific benefits would I see by switching my box to ReiserFS as opposed to EXT3 or one of the many other journalling file systems that Linux now supports? At this rate, seems like I could be reformatting my hard drives every week.
Now you just need to hire someone to desire a modern, more "commercially pleasing" website. =)
And how do you accidentally add a filesystem anyway?
My main question(s) is this(these): Is ReiserFS tunable to reduce usage of a laptop HD? If not, will it? As is, it's great on low capacity (under 1 gig) drives (I'm not sure how much overhead it imposes in comparision to ext2/ext3). However, it also likes to keep the drive active, which drains laptop batteries.
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
How's reiser4 shaping up these days?
Having obtained financing for the project, how does that impact the future direction of development? How do you balance the interests of developers, users and sponsors to choose which updates to pursue?
Stop by my site where I write about ERP systems & more
You've probably noticed that there are quite a few journaling file systems for Linux and other Open/Free/Unixie operating systems that are fairly open or completely open source. Have you ever thought about getting together and making "THE" journaling filesystem for these free OS's? I know that open source is all about choice, but as you probably also know there are certain things EXT3, XSF, etc. can do that ReiserFS cannot. If their could be a hybrid made from the big journaling filesystems would you be part of it?
Ignore the "p2p is theft" trolls, they're just uninformed
Once Reiser4 comes out, should I switch or just stick to ReiserFS? Why, why not and so forth?
Although its speed and (otherwise) good level of data integrity is of great interest to my employers, before I can recommend it to such a large company, I would like to know how and when you intend to put in a fix.
Comment removed based on user account deletion
Is it save to ditch EXT3 in favour of ReiserFS ?
Owner of a Mensa membership card.
What was so lacking in other file systems that you felt compelled to write your own? How do you think yours compares with the others?
I've been a fan of Reiserfs ever since I started using it, and I've been very impressed by its speed. I'm really anticipating Reiser4, and I hope it'll get merged into 2.6. However the last I heard about this is that Linus hasn't read your docs yet, and the status page on kernelnewbies shows Reiser4 to be "pending". Can you give us any insight on what's going on?
"Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it." -- Linus Torvalds
Can you clarify your long term goals with respect to ReiserFS and file system design in geek lay man's terms?
I'm refering to your Future Vision paper in particular. How would this kind of system be used on a day to day basis?
Right now any mention here of adding database like functionality to a file system is met with crude comments about SQL... Some explaination would really help.
cheers,
--
Simon
When will reiserfs have undelete facility (a la) debugfs for ext2/3?
As I understand, what ReiserFS offers that "enterprise-class" journaling filesystems like JFS and XFS don't is the ability to treat the filesystem more like a database and less like a tree, but I haven't really heard of anyone getting serious about using this functionality. Why did you bother to add it, who do you think will get the most use out of it, and how would you promote it, given the chance?
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
I've used reiserfs for years and was always very satisfied with its performance and stability. Recently, I convinced myself that ext3 had gotten as stable as reiserfs. So I switched to ext3, mainly because I ext3 == ext2 + journal, so should the need arise, I can mount my ext3 partition as an ext2 one. I haven't run any test on my new ext3 partition, but they look like they're doing okay and ext3 behaves well in case of a crash.
:-)
So fo me, today, in terms of usability ext3 == reiserfs + ext2 backward compatibility. As much as I hate to admit it, I have now given up on reiserfs.
So my questions are : can you compare reiserfs and ext3 in terms of performance and reliability, i.e. are there good reasons to keep using reiserfs ? Do you think ext3 might become the de-facto standard for Linux journaling filesystems ? How would you feel about it, after all the work you've done on reiserfs ?
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
Let me have a look at that tooth. Oh, you've got a cavity. Here let me ...
Aaaaaghhhhhh!!!!!!!
Now, is it safe?
Beside the finding and organizing files, the biggest problem for desktop users today is probably that changes on the file system are not recoverable. It is easy to accidentally overwrite a file and lose your work, and the only only sane way to solve these kinds of problems would be to make it possible to revert changes.
Several research systems have been created, like the Elephant File System, but none of them made it into the mainstream free and commercial operating systems. Are there any specific reasons why nobody offers recovery (high complexity in implementation, very bad effect on performance, etc) or is it just because FS designers don't see the need for it?
We ran reiserfs for a short period a few years ago because we were saving data to 10's of millions of small files, and reiser seemed like a perfect fit. We had to stop however because as we added more and more files, disk access slowed down more and more. It eventually became almost unusable and we had to switch back to ext2. How much faster is it now than it was then? Have you completely solved this problem?
(1) Software project management techniques seem to vary by discipline. Some groups follow the traditional âoewaterfallâ(TM) model others follow Rapid Programming models â" Do you try to follow a particular style? (2) Classic project management spends a lot of time accounting for resources. I can imagine that an open source project has people signing-on and dropping off. How did you keep assignments & scheduling under control? (3) Have you come up with any tips or tricks that work for managing your project that might be useful to other software development Project Managers?
way to grab an old smelly sock out of the closet, trouble is, the sock isn't so old or smelly. To me, Hans seemed rather civil in his approach of this matter. I didn't notice any direct name calling either. only a rather thoughtfull comparision of Debian's disregard to the software author's copyright violation. Hans original email was direct and to the point. "Hey guys, give credit where credit is due, don't change MY license", even other debian-devel users resort to more childish flaimbaitish type communication in this thread than Hans.
In your Future Vision white paper, last modified in January 2001, you outline several very interesting ideas about metadata.
Several developements have taken place since : the extensible attributes of BeFS has been burried with BeOS, the database-like metadata of Longhorn (aka Yukon) may actually be a separate layer from the filesystem altogether, and Apple is also moving all metadata out of the filesystem to XML files shared between applications (see iLife package).
My question : What is your current take on the metadata debate ? Do you still think the filesystem is the right place to handle metadata ? Any predictions ?
As a leading expert on linux filesystems I was wondering whether you were aware of any snapshot implementations for linux similar to those found in Network Appliance filers. Since working in a shop that used Netapp's I have always found myself wishing for the admin time saving feature of snapshots without the high cost of a filer. Veritas's snapshoting is whole volume only as are the other solutions I have seen outside of Netapp, is this a technical hurdle that no one else has been able to jump or is it a patent issue? Thanks for your time.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
and avoid questions that can be answered with a few minutes' worth of research.
Now if only you were to visit the said website namesys.com, You would see it in HUMONGOUS font "Raiser4 is due June 30 2003".
When will it be stable enough for early-adoption users?.
What kind of early adaptor are U any way ? asking questions about stability ?
for the last time people, I am "frodo from middle eaRTH", not "middle eaST".
"Reiser4 is due June 30, 2003!"
Will dynamically-loadable filesystem plugins be feasible, or practical? What scope will the plugins have? Encryption/decryption is obvious, but what about more interesting things, like scan-on-access Antivirus, or perhaps a mail spool could be mounted with some kind of spamassassin plugin.
- Minimizing Inner/Outer fragmentation
- Maximising locality of reference
- Minimizing the cost of file extension
I'd like to know how Reiser FS approached these conflicting goals. Has Reiser FS used any mathematical models to evaluate long-term stability/performance?Given the problems encountered by Mark West when DARPA pulled its funding on the OpenBSD project, are you concerned about what strings may be attached to such funding for ReiserFS, be they implied or explicit?
I'm very appreciated your effort in bring enterprise elements to Linux, it's people like you make Linux success nowaday. However, I see that you or your project team seems to push the journalling fs concept too far: from root to /usr, to any single partition in a system. May be I'm a bit old-school, but journalling fs should be used only on fs which require failover protection. It's a plain waste of resource to make, say, boot and /usr journalling, for example, as they are supposed to be running readonly in normal circumstance.
Shouldn't you do something to educate people journalling is just part of the system and shouldn't be used for everything like Windows NT, and waste precious processing power?
Reiser FS is already a pretty mature, stable, usable product. Once V4 is done, is there really much work left to be done on ReiserFS proper? Do you have a giant to-do list that'll keep you and the guys occupied for years, or do you intent to work in a diffent direction (SAN, networkFS, databases, etc.)?
(or perhaps you'll just retire to Portugal and play lots and lots of golf)
## W.Finlay McWalter ## http://www.mcwalter.org ##
uhhhhh IBM has mainframe swith Linux right NOW!
Great Linux Site
I had the same reaction -- Reiser didn't seem so far out of line to me.
When you began a file system project as a free software project, you must have known that (assuming it worked) it had the potential to turn into a big project. How did you determine how long to work on it as your own project before making the first release? I imagine there must have been a strong temptation to just get it "out there" knowing its potential, yet certainly releasing too soon would make it look unprofessional and thrown together.
Obviously, you named ReiserFS after yourself. Why after yourself? Were there any runner-ups? What will happen when you create another great project? :)
-
ping -f 255.255.255.255 # if only
Is it just me, or does that thread remind anyone else of "CRIPPLE FIGHT!!"?
One question I always have with regards to successful (meaning funded, wide acceptance, large user/developer community etc.) is how did you raise the awareness of your project to get it from just a side project to something that it is today?
Did you use traditional PR techinques, or just through a community of connections?
sad robot making broken music
Is there any effort to port ReiserFS to *BSD systems?
{{.sig}}
Apologies in advanced for a simplistic and obvious question but...
What's the best way to get funding for a project, assuming the project is at some sort of usable prototype stage already (i.e. who should you approach, what ground work should be done)? How should one court potential sponsors? Is it better to wait for them to come to you, or should you present a formal funding request in a format similar to that of a traditional business plan?
While Reiser's posting was far from friendly, the only hostility and name-calling seems to come from debian developers.
These things simply happen when ego's clash. Debian developers seem to have a collective huge ego, and Reiser has some prettig large ego by his own.
Live with it. It helps you enjoy the wonderful filesystem called ReiserFS.
This is your sig. There are thousands more, but this one is yours.
From what I have understood of file systems, data safety is priority number one. Considering that, are there any super speed algorithms you have dismissed because of "unsafe" behaviours?
On the ReiserFS 4 page you say that the new filesystem uses algorithms allowing the FS to do transactions only databases could do previously. Do you envision ReiserFS 4 being something comparable to MS' "SQL-FS" plans in Longhorn? How would ReiserFS 4 affect regular (oracle, postresql, mysql) database projects/companies?
Clearly Unix filesystems are moving in the direction of extended attributes. This isn't necessarily a smooth transition for Unix, where many utilities are modeled after streams of bytes.
I personally have pondered about this a lot. What do you think needs to be adapted and why? Should the VFS be changed? Should traditional Unix tools like "cat" be adapted somehow, or given attribute-aware equivalents? Should file attributes be accessable at the file descriptor level? Will we need a new "tar"? What should be done to standardize these attributes, or make them compatible across filesystems?
The directory layout of the typical unix filesystem has grown over the years to what has become the FHS 2.0.
If you were in the position of designing an entirely new FHS, more suited to ReiserFS (lots of small files, database-like access, etc) and without backward compatibility, what would it look like?
This is your sig. There are thousands more, but this one is yours.
Hans original email was direct and to the point. "Hey guys, give credit where credit is due, don't change MY license"
But the license he is using is the GPL. Hans wants to further restrict the GPL so that he can spam users of his filesystem with advertizements for his sponsors and company. Further, it seems that he's incredibly interested in making sure HE gets credit but doesn't really care for acknowledging the work of others.
So my question is, if you don't like the way the GPL is worded and don't feel that it allows you to demand enough credit for your work, why in the world did you choose the GPL for a license?
In your future visions paper early on you talk about Reiser's Rule of Thumb #2. However, I can't find Reiser's Rule of Thumb #1 -- what is it? Is it a secret? Does it contain the sum of all human knowledge?
TELL ME!
Carpe Daemon
Ext3 deals in multiples of whole blocks, and its dir structure is simplistic. It's been my experience that this plays nicer with slow drives and crypto loop filesystems. ReiserFS by contrast tends to thrash around writing itty bitty pieces of files and metadata.
2 words - Bull, china.
You don't just charge in with all-out accusations, you politely ask and see if something can be worked out.
"To any truly impartial person, it would be obvious that I am right."
How about adding better methods for repairing partitions with bad blocks?
I lost three partitions of data that were formatted with ReiserFS and I was unable to repair them. Creating an image of a partition is simply unacceptable IMHO as part of a data recovery process, suppose you have 600GB of data?
Okay... I'll do the stupid things first, then you shy people follow.
[Zappa]
this question is aimed at satisfying the linux newbie.
What features do you feel are most important/would convince a newbie to make ReiserFS the system to use as compared to the many other alternatives (XFS,EXT3,etc...) out there?
In the interest of accuracy, could you restate "huge" and "pretty large" in the current Proposed Standard of Ego Measurement?
--
"Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next.
I've spoken to a few people (not naming names) who appear convinced that the design of ReiserFS is fundamentally flawed, and that it would lead to inevitable deadlocking. I've read your white papers now several times (took me a while to figure them out), and think the work you're doing is great, but a few smart people seem to think it can't be trusted. How would you reassure those people that their data is safe?
the redmond hegemony lost the internet protocol battle and decided to fight back with semantically rich but proprietary file systems (think 'halloween memo', 'longhorn', 'groove'). are you thinking about how to mutate ReiserFS to support arbitrary object attributes, full-text indexing, callbacks, namespace syntax for queries, etc?
Mr. Reiser, first off I have no complaints about ReiserFS (which is a high compliment), I use it on almost all my machines, except a couple are running EXT3 because they're not heavily used and I'm lazy at times. But thats neither here nor there.
You fall into an interesting subcategory of project managers or whatever you want to call them. I'll call it the "outspoken genius" category (even though the first word might be understated and the last is probably hyperbole). Basicly your work is technically interesting, applicable, etc. That's a give in. But there are quite a few people who have personal issues with you and your manner and usually cite some exchange or another. Sometimes this is the basis of an argument to reject the use of your work, which I think is somewhat silly. You're not the only one, and certainly not the first to be interviewed here.
So what do you think about this? ie. Do you think you made interpersonal mistakes that landed you here or do you think you've been misunderstood? Does it bother you? Why do you think people enjoy egging on folks such as yourself and then citing the moment you get annoyed with them? Do you think this question ever has a prayer of being moderated higher than someone following the method of the previous question?
Jeeze, I realize I just wrote an essay question in the style of one of my old Philosophy professors. You know the kind, here's a statement now write some stuff (I guess I'll give you a few ideas of where to go).
Sorry - That was not my intent. I don't know how to start a new thread.
I don't know how to start a new thread.
Just click the Reply button to the right of the dropdown boxes for changing display options -- between the article and the start of the comments.
That's awful big talk from a
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
I have read that Reiser4 will allow plug-ins. Will more functionality wander into user-space and will that influence kernel stability?
If I had moderator points, I'd mod this what it deserves, but I don't. So I will instead burn a little karma to point and laugh at the parent poster and the moderators who were befuddled enough to moderate this "Interesting".
Look folks, if you want to waste your own time trolling, that's fine...but don't waste Hans' time on this drivel by moderating trolls up.
I was wondering how complicated is to get funding for an Open Source project from big companies like IBM, SuSe or even RedHat. I mean, probably you have to create a "business plan" kind of document to explain why someone should invest money on your project, but what important things should be included as a part of such proposal? (or this is just a matter of have a proper "networking").
:)
Also, do you known if there is any company that dedicates itself to help Open Source projects, besides de FSF?
Thanks in advance
Jose Vicente Nunez Zuleta RHCE, SJCD, SJCP
Anecdotal stories of ReiserFS having issues with disappearing files have been very concerning. It is my understanding that the point of the journaling is basically to prevent this from happening (more elaborate I know, but I don't care to type lengthy right now). While I've been quite happy with using ReiserFS on some of my machines, even anecdotes of this happening leave me concerned there is too much risk. What's happening here, why, how will it be fixed? Is it something wrong with the filesystem / drivers or user implementation? Or can we deny it all at once and say somebody was doing something stupid? I believe in order for this project to be really successful despite its funding, it needs to get past these rumors.
SIG: HUP
Filesystems like Coda and InterMezzo, network RAID applications like ENBD/FR1 and DRBD, and filesystem synchronizers like Unison provide some partial solutions for the need to have a distributed filesystem with replication and disconnected operation. Do you think Reiser4 or its successor will provide features which make it easier to implement a robust usable distributed filesystem?
I've been reliably using ReiserFS on high-load mail servers for two years. No self-induced glitches here.
If at first you don't succeed, skydiving is not for you
I recall a discussion at lkml about the licensing issues with bitkeeper. Someone said that the future versions of your FS will have some version control built-in, and thus be a sort-of competitor to Larry McVoy's bitkeeper. Larry's comment was that if that happens, then you have to buy bk in order to use it.
You suggested that indeed version control is going to be in. Can you say more about these plans now, and the eventual conflict with bitkeeper.
Thanks.
Probably because even anyone proposing this in a Unix FS would get spit on. "Oh no, we're not conforming with POSIX!"; "Oh no, this is not the way Unix does it!"; "No, this just won't work with the rest of the kernel".
VMS didn't have to go through the Unix interfighting and standards compliance (which are broken in many places anyway, and stifle innovation).
No question, but as a recently-joined member of the unofficial Hans Reiser fan club, I'm required to say: What a guy!
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
I'm preparing by reading everything I can find: I'm working on Tanenbaum & Woodhull's "OS Design & Implementation"; I've read "Design and Implementation of the Second Extended Filesystem"; Steve Pate's "UNIX Filesystems" is waiting on my shelf; and of course, there's the FAQ and ReiserFS v.3 Whitepaper at www.namesys.com.
Specific questions: what branches of math are useful in this line of research? Any books, articles, etc., that I haven't listed that are a 'must read' or 'should read'? Those who have succeeded in building a better filesystem: what have they done that I should also do? Any mistakes I should avoid? Anything that no one told you about filesystems that you wish you had known up front? And are there any special tricks (above and beyond mastering your subject) to getting hired in this field once a degree is in hand?
Thanks!
ReiserFS has got a large amount of features in a short time. Also, new features are added regularly, and new, massively changed releases were issued several times. There were concerns regarding how this could affect the reliability of your filesystem. What's your take on that?
It used to be late 2002. Then around late 2002, I noticed that the homepage had been changed to say the end of this month.
Was it an unforeseen debugging issue? Was there an algorithm change in the mid-to-lower levels? Was it a lack of coding resources due to excessive meetings? Or was it simply an underestimation of the total time required to get to the first public milestone?
I'm sure I'm not the only one who would like to know the type of things that delay releases in the free software world. When you have more info about what slows a project, you can see better how to avoid it in the future.
- I don't need to go outside, my CRT tan'll do me just fine.
Actually, it says by the time you reach a million or more files the chance for hash collision is approximately 100%. However, you can still potentially hit hash collosions way before then. The handling of hash collosions is the question - does it prevent the new file from being created, or does it overwrite the file, losing data. My understanding is the ReiserFS does the latter. I could be wrong though, and I hope I am.
You are in a maze of twisty little relative jumps, all alike.
Non-Journalled, or Unsure:
Journalled:
Network file-systems
To summarize: We have a horribly-large number of filesystems, most of which are incompatiable, many of which do not support the Linux security module extensions, one (e2fs) provides defragging under Linux, and none at all provide support for conversions.
Hey, diversity is good! I -like- diversity! I want MORE diviersity! I also want ways to efficiently move data around.
Will future versions of ReiserFS include additional userland tools for defrag, fs conversion, scope of logging (eg: none, meta, full), pluggable hashing algorithm, etc?
Ultimately, all the choice in the world is no choice at all if there's no way to make use of those choices.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
A while back, there was a Slashdot article about the Conquest File System, a filesystem for machines with large amounts (gigabytes) of battery-backed RAM. It seems likely, with falling RAM prices, that large quantities of persistent RAM will be a part of fast servers in the future. Do you have any plans to make ReiserFS take advantage of such systems, beyond just putting the journal in RAM?
Having seen the BeOS development APIs, Reiser4 strikes a familiar cord with its metadata handling and querying. However, while BeOS had a captive audience (there were no BeOS clones/work-alikes), you are making ReiserFS a part of an established operating system and all of the inertia that "establishment" brings with it.
Where BeOS failed (among other reasons) because of the perceived lack of applications, do you have any thoughts on how to seed development for ReiserFS.
Case in point, where in other filesystem setups you would have a mail spool and/or mailbox as a custom file format (and indexes) on top of the filesystem, Resier4 allows for custom queryable file attributes (metadata) and the ability to efficiently store each message in the queue as a separate file. The mail handling program becomes proportionally less complex as that complexity has been shifted toward the common filesystem layer.
That said -- mostly for the sake of folks that haven't been following Reiser4 development -- do you see a way of encouraging Reiser4-aware programs? After all, it's a chicken and egg issue. Without common usage of Reiser4, application developers will be less inclined to commit their time to it (perhaps?). Without applications that take advantage of its unique features, what reason does the user have to put it on their workstation or server?
Are we looking at custom Linux distributions with this in mind or just piecemeal applications? Personally, I'm looking forward to dropping Ogg Vorbis files into a directory and, through the use of plugins, having their ID3 info automatically searchable. Hmmm... Where was that Fugees song again?
- I don't need to go outside, my CRT tan'll do me just fine.
Do you have any updated performance metrics for reiser4 in comparison to ext3, xfs, jfs, and reiser3?
I'd be curious to see both small file test, large file tests, and a mixture -- as is the case for websites with movies where the HTML and graphics are 3K but the media files are 50MB and up.
- I don't need to go outside, my CRT tan'll do me just fine.
How much do you cherry-pick (and get cherry-picked) ideas from other modern filesystems? I understand you had some debates with the XFS developers over the "best way" to handle certain storage/algorithm cases. Was this the norm or are things far more insular than that with folks keeping their noses only in their own projects?
- I don't need to go outside, my CRT tan'll do me just fine.
There's some interest from users in seeing ReiserFS ported to other operating systems, like FreeBSD. Is this something the ReiserFS developers are interested in doing/encouraging, or are they focusing their efforts on the big moneymaker (Linux)?
Do you see the work being done by Microsoft with regard to the "databasefying" of the filesystem as a boon or a hindrance to the adoption of programs that allow complexity to fall to the filesystem?
In other words, do you see a Windows developer who has worked with Longhorn (or whatever the name is now) being more inclined to develop for/port to Linux because of the parity in functionality? And would the converse be true in your opinion?
- I don't need to go outside, my CRT tan'll do me just fine.
As someone who is very interested in filesystem design, I've been following your progress and goals for a while, especially when it comes to fundamental changes in v4 and v5, and new API's. After a great deal of discussion, you managed to get a new system call all to yourself (reiser4()), so that you could experiment with new API's for advanced features. This seems like a good solution for now, because you can test stuff and refine your ideas. However, it seems that Reiser5 is going to need a complete new API.
My question is twofold.
1) Are you going to try to write a new API from scratch, even if(when) Linus doesn't like the idea of a complete new filesystem interface?
2) Would you consider porting to another plaform if necessary to have the control over the API that you need? Or even just to expand the number of supported platforms?
As someone that dual-boots windows and linux, I've always been looking for a good cross-platform filesystem. Currently, this limits me to fat32 and a lesser extent ext2. Are there any plans for a port of ReiserFS to Windows 2000 or XP?
How is/was your business in Moscow?
My exception safety is -fno-exceptions.
In the past few years there has been an enormous amount of effort made extoling the virtues of journaling filesystems. But do I really need a journaling filesystem? Under FreeBSD I get by just fine with SoftUpdates, so I'm wondering why there is no effort in the Linux world to create a similar solution, instead of the mad rush to journaling.
After all, your Aunt Tillie isn't running a bank or a stock exchange on her laptop, but instead browsing the web and reading email. Something lightweight and fast sounds better for her.
A Government Is a Body of People, Usually Notably Ungoverned
There seems to be a lot of FUD against reiserfs being spread by a small but vocal minority of people.
Among other claims I have heard by the FUDsters - 'no large production systems (eg terabyte or larger) trust their data to reiserfs', 'xfs is superior to reiserfs because it has been well tested on irix', 'reiserfs has chronic data corruption problems', etc. Spin tactics that would make microsoft's propaganda teams proud.
How do you respond to the FUDsters? What claims are bogus, and what valid objections to reiserfs are there? (The only legitimate problem I can think of - lack of full data journaling - is shared by xfs and jfs as well).
There also seems to be a resistance by commercial distros to adopting reiserfs.
What do you attribute the resistance to?
or are they too cowardly to stand behind their claims?
it's easy to throw rocks while hiding in the shadows. but will they still make their claims in public?
First of all anyone who hasn't read the Future Vision for Reiser4 should. From the comments so far it seems like most people are unaware of the main goal of the project, which is too bad because it is probably the most ambitious free software project in existence.
/proc or /etc? Wouldn't it be nice to be able to change your resolution in X with /etc/X11/Resolution?
/etc/password, for example would look like this
/etc/passwd/
/etc/passwd/root/
/etc/passwd/root/password => sldk7w9xh3
/etc/passwd/root/uid => 0
/etc/passwd/root/gid => 0
/etc/passwd/root/name => Sir Walter Raliegh
/etc/passwd/root/home => /root
/etc/passwd/root/shell => /bin/bash
/etc/passwd/root/luser/
...
/etc/luser/password), without adding the complication of a half-assed database (e.g. our friend the Windows registry).
This doesn't really do it justice, but what the project is trying to accomplish is to push a great deal of the functionality of databases into the filesystem. In a better world than ours you wouldn't need a table with keys and values, you could just have a folder (table) with filenames (keys) and file contents (values). The problem is that storage is allocated in large blocks, and so the kind of small files that one would use as values end up wasting enourmous amounts of space. This is the virtue of ReiserFS: it has efficient small file performance. With future version the namesys crew seems to intend to add all the wonderful searching abilities that we would want directly into the system calls.
Ask yourself this, which is less hassle to use,
echo "800x600" >
The general idea is that settings could be stored hierarchically, so that
where => is some sort of operator that displays the contents of a file.
Essentially this adds all the benefits of a registry (e.g. you change your password with echo "opensesame" >
Think for one second about how easy this would make adding a GUI configuration to your favorite program ("You mean I don't have to write a seperate parser for every program"), without, of course, requiring a GUI. It has simplicity, it has elegance, it has closure.
My question for Hans is this: it seems that you have put in place a system to let the filesystem take on all the functionality that I was going on about above, how do you intend to get people to use it? Making reiser a filesystem does add a high level of commitment: users have to reformat their drives, and people writing applications can't really assume that their users will have it, and so will be reluctant to rely on it. Obviosuly it is immediatly useful for in-house projects that can have very specific requirements, but how can we get the rest of the world to switch over?
Hans,
Do you find it annoying that 75% of the Slashdot questions could be answered by actually reading the opening page of your company's website?
11*43+456^2
And you're apparently also the reason they have to specify "one question per post" in every damn interview posting.
Admins who have worked on NetApp filers know the joy of filesystem snapshots. Are there any plans to add this feature to ReiserFS?
I know that reiser primarily uses balanced trees to store data, which I think is great. I decided to make my inodes structured this way for an operating system course. Kudos to an excellent file system!
However, how does reiser's algorithms deal with the dynamic allocation problem? What scheduling algorithm does it use to queue jobs? What replacement policies does the cache use? And finally, have you ever considered writing a file system in a functional language, such as ML (I think its a great idea)?Keep up the good work!
Carmelo"Recursive bipartite matching"- try it!
Why can't I use the one-partition hint with LFS builds with Reiser? LFS notes some weird kind of recursion error if a recursive mount is performed on the file system. So, as such I must use ext2 or ext3 when moving a new LFS into place of a canned distro.
REISERFS WARNING:
If you're using reiserfs, this hint is almost unusable for you due to deletion errors on a recursive mounted volume with reiserfs (the files will be deleted, but the space wouldn't be freed). You could still use the hint to make everything ready for a transfer using a bootdisk/CD, but don't try the "Delete the previous System" section or further. You've been warned!
I don't know much about file systems I admit, but I'd like to know about this one area where ext2 and ext3 have a additional feature Reiser does not.
Do you have any plans to add BeOS style live queries to ReiserFS? They were IMHO the best feature of the BeOS, and they are the only thing I really miss when using linux. Could this be done as a plugin, or does it require more fundamental changes?
I know that linux has the dnotify mechanism, but that is only useful for watching for changes in a single directory, so it is basically useless for most purposes.
Private property is the central institution of a free society (David Friedman)
Hans, how come everywere i go i always see people saying reiser has no 'undelete' function. i recently installed slackware and wrote down my partition numbers wrong and ended up remaking the reiser fs over a directory of 18 years of pictures three times! i was able to recover all but 5 pictures by replaying the journal.
why is this not considerd an undelete feature? or is this not the intended use?
-- botsex is {grep;touch;strip;unzip;head;mount}
No mod points though :(
Get your own free personal location tracker
Didn't this go out of style with VMS?
I can see the need for Versioning and Transactions, though. I've been pondering these myself.
One short-term solution to versioning (in user space on a single user workstation) would be to run a private CVS and CO/CI your home dir for each login session.
Being and x509 buff, I'm appalled at the non-flexible logical structure built in to (and expected of) 'nix operating systems and their numerous filesystems.
I'm looking at a project to pull together a good unified directory for security, ACLs, configuration, networking, etc... The onerous task of converting utilities and user-land apps will come after a solid and fast system is built.
OpenLDAP is decidedly out of the question, along with NDS, ADS, and the other also-ran and proprietery directory solutions.
--==-- I've found Karma to be a relative thing... Ya know, the kind you invite to Christmas...
Does anyone know the default cluster sizes for the various file systems? I'm a little confused on that, since when recently doing a rebuild of my box, I seemed to be limited to a certain (albeit, efficient) size. Ideas?
Up to 127 filenames per directory (MAX_GENERATION_NUMBER, defined in include/linux/reiserfs.h) can have the same hash value. After this, creating more filenames with this hash is impossible (the EBUSY error code is returned). ReiserFS does NOT blindly overwrite files because of hash collisions.
/* there is no free generation number */ /* /* I think it was better to have an error code with a name that says /* adjust offset of directory enrty */ /* update max-hash-collisions counter in reiserfs_sb_info */
/* we need to re-search for the insertion point */
/* Following line is 2nd line touched by Alan Cox' trivial fix */ /* I think it was better to have an error code with a name that says
You can choose from multiple hash algorithms when you create the filesystem (faster hashes have a greater probability of hash collision). But collisions aren't a reason to avoid ReiserFS - most other filesystems (including ext2/ext3) won't get anywhere near a million files in a directory before suffering huge performance losses.
The following code was taken from linux-2.4.21-rc1/fs/reiserfs/namei.c and demonstrates the handling of hash collisions.
gen_number = find_first_zero_bit ((unsigned long *)bit_string, MAX_GENERATION_NUMBER + 1);
if (gen_number > MAX_GENERATION_NUMBER) {
reiserfs_warning ("reiserfs_add_entry: Congratulations! we have got hash function screwed up\n");
if (buffer != small_buf)
reiserfs_kfree (buffer, buflen, dir->i_sb);
pathrelse (&path);
* Trivial changes by Alan Cox to remove EHASHCOLLISION for compatibility
*
* Trivial Changes:
* Rights granted to Hans Reiser to redistribute under other terms providing
* he accepts all liability including but not limited to patent, fitness
* for purpose, and direct or indirect claims arising from failure to perform.
*
* NO WARRANTY
* This is one of two lines that this fix consist of.
*/
return -EBUSY;
what it means, but I choose not to fight over it. Persons porting to
other operating systems should consider keeping it as it was
(return -EHASHCOLLISION;). -Hans */
}
put_deh_offset(deh, SET_GENERATION_NUMBER(deh_offset(deh), gen_number));
set_cpu_key_k_offset (&entry_key, deh_offset(deh));
PROC_INFO_MAX( th -> t_super, max_hash_collisions, gen_number );
if (gen_number != 0) {
if (search_by_entry_key (dir->i_sb, &entry_key, &path, &de) != NAME_NOT_FOUND) {
reiserfs_warning ("vs-7032: reiserfs_add_entry: "
"entry with this key (%K) already exists\n", &entry_key);
if (buffer != small_buf)
reiserfs_kfree (buffer, buflen, dir->i_sb);
pathrelse (&path);
return -EBUSY;
what it means, but I choose not to fight over it. Persons porting to
other operating systems should consider keeping it as it was
(return -EHASHCOLLISION;). -Hans */
}
}
Another big reason why a lot of people implement snapshot differently than NetApps, is to avoid shooting yourself in the foot. With NetApps, the snapshot data is kept on the same volume as the data itself, which leads to situation where you jump from say 50% usage to 99% just like that overnight (the snapshot area is allowed to run over the data area). This is quite a delicate situation as deleting files makes things worse (you have to get rid of old snapshots to free up space). I have seen big production database taken to their knees because of this.
On the other hand, the other implementations are a bit slower because the blocks are copied instead of being just not deleted, but snapshots never take space from data. The implementer has to make the choice, space control and simple understanding of space vs. speed to snapshot and recovery.
Absolutely!! I have looked high and low for anything approaching the beauty of the VMS filesystem for Linux. I'm currently using glastree and some custom code to create a backup versioned filesystem with Samba/MSDFS, but it is only an approximation of what VMS had.
Will this be possible with plugins?
Regards, tEtra
I throughly understand the file-system is everything independent of any overlying infrastructure in terms of applications and files, but an interesting thought runs through the head. How about having file-system providing hooks to track what portions are heavily used? The basic idea is some sort of real-time information system. File-system provides the lowest level where 'things happen' and working on that level will cut through all the crap, so to say. I understand that all files are not plain text and file-formats or all the executables loading, but I am talking in terms of only hooks and do not expect the file-system to do that.
Random Stuff
If memory serves there has been a quite a bit of speculation that Microsoft has been having a really hard time with their new FS, WinFS. Have you thought of approaching them to port Reiser4 to Longhorn (or some later Windows release)? IIRC Reiser4 and WinFS had some major design goals in common (i.e. lots of metadata in the FS and a more database like design).
Have you had any issues with the fact that Namesys is a Russian company? I think we all remember the recent story about DARPA pulling their funding of OpenBSD. Do you worry about this happening to Namesys?
When using ReiserFS I often find some newly-(over)written files contain garbled content after a power loss. Filesystem integrity is okay according to reiserfsck. When using Ext3 such problems don't happen that often. From what I have read, this is because ReiserFS does metadata journaling like Ext3's "metadata" mode, while Ext3 uses "ordered" mode by default, which prevents writing metadata before the actual data that may cause garbage to show up if a power failure happen in between ("in between" can sometimes mean a minute or so).
The question is: will reiserfs support something like Ext3's ordered mode? Otherwise, it is quite unsafe to store frequently modified files (like those in the home directory) on ReiserFS partitions, if power failures happen often.
If you read the thread, and I do mean read it, you might notice that Reiser doesn't actually tell anyone what the problem is. Is it the accidental removal of credits from a file? Or is it the deliberate disabling of the long credits list during a program execution? Reiser doesn't tell anyone, because he'd rather throw a tantrum than actually solve whatever problem he's having. There's nothing direct and to the point about what he wrote at all.
Oh... and no one changed his license. It's the GPL, you can change it how you like. Since Reiser decided to implement ReiserFS in Linux, he was bound by the GPL and all it enatils. If he doesn't like its terms, he can write his own kernel or use another. The Debian people were fully within their rights to change the program however they wanted. It might not have been perfectly civil, but then again, it's tough to correct your mistakes when you aren't being told what they are. Besides, the credits list generally belongs in the README or copyright files, not at the launch of the fucking program. Would you like ls or cp to display a credit list every time they're run?
And finally, Reiser decided to come out and bash the whole of the Debian project for something the maintainer himself did. Reiser never directly contacted the maintainer about the issue, but rather chose to vent on debian-devel, calling everyone in the project plagiarists. He can throw out Stallman's name all he wants, but that doesn't make him any more correct. His little tantrum certaintly doesn't represent the people he claims to be standing up for very well.
"I may not have morals, but I have standards."
It is true that even Ext3's ordered data mode cannot absolutely prevent breakage during a power loss, but keep the probability of lossage low is also important. It is impractical to backup my term papers every minute, and the lower the chance of losing one day's work (or more if I forgot to backup yesterday) because of a power loss (because of garbled content... I do save every minute), the better.
Did you perform any benchmarks of ReiserFS/Linux vs. UFS/FreeBSD (on the same machine)? My personal experience shows that UFS/FreeBSD is twice faster than ReiserFS/Linux or XFS/Linux. Perhaps the fault is in Linux Kernel? - because obviously ReiserFS is performing very good when confronted to any filesystem on Linux platform. If it were the case - would you consider porting your excellent FS to *BSD?
You can defy gravity... for a short time
I don't know how much detail you want anyway. All I was told is that the potential for nasty deadlocks is almost built in to the design... I pointed out that this was surely a property of the implementation, not design. But apparently not.
Of course this is 2nd hand advice, and I believe the person who originally said that was in fact a FS designer. So maybe there were some sour grapes there.
--Dude, Reiserfs is TINY compared to XFS: (lsmod output)
Module Size Used by Not tainted
xfs 511026 0
reiserfs 161008
ext3 62880
> With ReiserFS you often stumble into incompatibilities with other things in the kernel, such as NFS and LVM.
--Patently false. My SCSI and IDE LVM setup both use Reiserfs, and I have had no problems with using NFS to access Reiserfs partitions or volumes. Some older SuSE NFS userspace implementations had to have a hashed_inodes entry in /etc/exports, (I'm talking SuSE 6.4 and 7.3 here) but I'm using nfs-kernel-server in Debian/Knoppix and it hasn't had any issues.
--If you haven't tried Reiserfs lately, give it a shot. Most of the "problems" people think it has have been fixed for a long time and it's been rock-solid stable for me for ~3 years.
.
== WolfriderV6 == I'm willing to admit that *I just might* be wrong... Are you??
don't forget lazyfs, lufs, autofs
> --Dude, Reiserfs is TINY compared to XFS:
I wasn't comparing with XFS.
> Patently false. My SCSI and IDE LVM setup both use Reiserfs, and I have had no problems
Doesn't mean other people haven't had disasters with RiserFS. See the mailing list archives.
> --If you haven't tried Reiserfs lately, give it a shot. Most of the "problems" people think it has have
You can't ignore the crappy historic stability record.
> You can't ignore the crappy historic stability record.
--(shrug) Sure I can. As I said, I've been using it for ~3 YEARS with *no problems.* What are the *dates* on those archives you're talking about?
.
== WolfriderV6 == I'm willing to admit that *I just might* be wrong... Are you??
Some AC recommended this reply (also from an AC) which had been languishing in score 0 territory be modded up. I'll go ahead and post it so it gets noticed by people reading at a threshold.
:)
I'm still not a great fan of it simply refusing to create the file and then returning an error code that isn't that explanatory (tho it doesn't take a rocket scientist to realize that EBUSY on creat is an overflowed bucket), but a failure is definitely the preferred behavior here.
I personally think filesystems need about 10 years to shake out all the bugs. NTFS is just now getting to where I can trust it, for example (and it still has serious warts). ReiserFS is definitely getting there, but til then I'm a fan of good ol ufs -- with softupdates of course, yes I'm a BSD bigot
-------------------8<------------
Oh hey I WOULD have posted it, but the god damn lameness filter stopped it. Can't have a site populated by NERDS actually having CODE in its discussions. So go adjust your threshold, read the parent, and it's time for a new sig .
I've finally had it: until slashdot gets article moderation, I am not coming back.
I am curious what your opinion is on using ReiserFS as opposed to XFS on very large data stores (say 3+TB). I have done a great deal of benchmarking on my current system (x86 box with 1.75TB external RAID array on an Adaptec U360 controller) to compare XFS and Reiser and gotten very different results when changing the file sizes. Reiser seems much happier with zillions of small files where XFS seems to excell with very large average file sizes. Is there a way to tune Reiser to balance it or have I simply provided myself with flawed findings?
I've tried XFS a couple times over the years and one thing I've noticed every time is that it didn't seem that great for an average desktop machine. It seems to hang the system while doing lots of file operations. It feels like the file system is taking precedence over the OS, therefore making an interactive system seem sluggish.
For a server this might be OK because you may get a performance increase due to the filesystem getting lots of CPU time but on the desktop it just makes your apps hang when the filesystem kicks into high gear. I didn't like it at all.
It could be that I have something configured incorrectly, but I've tried tuning it as best as I could and it still ran the same. I welcome any ideas.
I find that Reiser or EXT2/3 run mostly as fast as XFS anyway and don't hang up my system while doing stuff.
The ratio of people to cake is too big