Reiser On ReiserFS's Future And More
Steven Haryanto writes: "This one's from Indonesia.
InfoLinux did an email interview with Hans Reiser, in which he explained about the ReiserFS project plan and the new Namesys business model. Mr. Reiser told me that Namesys recently received $600K funding from DARPA to include encryption in ReiserFS v4.0." Dig this quote: "We are going to add plugins in our next major version, and we hope that plugins will do for filesystems what they did for Photoshop." Mmmm -- encrypted, compressed, journaling, extensible filesystems. Reiser also touches on issues of international software development and how programmers can achieve fame.
http://www.namesys.com/whitepaper.html is much better.
Cheers,
--fred
The most famous example of a file system that grew into journaling was Solaris UFS. I have heard some rather harsh criticism (from SGI) on the design. I don't know enough about filesystem design to discuss the merits, but I do know that Sun was the number one US server vendor last year, so they must be doing something right.
Here is the man-page entry:
logging | nologging
If logging is specified, then logging is enabled for the duration of the mounted file system. Logging is the process of storing transactions (changes that make up a complete UFS operation) in a log before the transactions are applied to the file system. Once a transaction is stored, the transaction can be applied to the file system later. This prevents file systems from becoming inconsistent, therefore eliminating the need to run fsck. And, because fsck can be bypassed, logging reduces the time required to reboot a system if it crashes, or after an unclean halt. The default behavior is nologging.
The log is allocated from free blocks on the file system, and is sized approxima tely 1 Mbyte per 1 Gbyte of file system, up to a maximum of 64 Mbytes. Logging can be enabled on any UFS, including root (/). The log created by UFS logging is continu- ally flushed as it fills up. The log is totally flushed when the file system is unmounted or as a result of the lockfs -f command.
RiserFS should fall under export restrictions since technically it's imported from Russia and other places. That's where most of Riser's staff works.
--
Quidquid latine dictum sit, altum viditur.
Whatever is said in Latin sounds profound.
--= Isn't it surprising how badly I spell ?
Or his uncle, Chow Yun Fat. Author of the original FAT file syustem.
--
Quidquid latine dictum sit, altum viditur.
Whatever is said in Latin sounds profound.
--= Isn't it surprising how badly I spell ?
Perhaps I'm just not up on the latest compression techniques (most likely), but those questions just popped in my head.
Either way, this is just further down the road of increasing CPU requirements just to drive the friggin disk. Ick. I miss SCSI.
--
Care about electronic freedom? Consider donating to the EFF!
I don't see how it is x86 specific. It is specific to CPUs that use caches, which is pretty much all non-embedded CPUs, and many of the embedded ones. I would imagine this would not be faster on something like the Terra, which is a non-cached CPU, but since it relies on proper management of 1000s of hardware level threads, I don't think Linux is ready to run there at all. Even on those systems it should be faster then indirecting through a pointer, faster by exactly one memory access (the pointer).
I do agree that it is untested (in terms of benchmarking). I don't think it has entirely wrecked extensibility. One needs to modify that file to add a filesystem, so you can't do it through a run-time loaded module...unless you use the generic void*.
I'll go for "that is really really ugly", and maybe even "does that make a noticeable difference when you are probably going to schedule a disk I/O?", and even "I'm glad FreeBSD doesn't do that". There is no way I'll buy "it's x86 specific" though.
This has been discussed on the LKML. Using a Union for all the various FS specific inode info is a performance hack. You could have a pointer there to a FS specific inode, but you are going to lose memory locality benefits, and cause cache line thrashing. Inodes are touch OFTEN, that makes thes memory issues important.
Further this is a performance hack that takes advantage of the fact that the Linux Kernel is open source, and easily compiled. You can't get this advantage in closed source without arbitrary FS specific inode info size limits.
But, you are right there is an ugliness to it, but don't make the mistake that Linux Kernel developers are idiots or poor coders. This has been considered.
Remember the truism: The Perfect is the enemy of the Good.
-- I am not a fanatic, I am a true believer.
Just because something is modular in the kernel doesn't mean it can only be a module. The only case that this exists, AFAIK, is the protocol-specific masquerading modules.
Maybe against the current recommendations, anything that I don't have to load as a module (my AWE32 and Masq mods) gets compiled into the kernel. Why? Because it's not like I won't need the features - that's why I selected them for compile in the first place.
If you encrypt all of your main filesystems, then you'll just have a /boot partition with vmlinuz on it, and the encrypted filesystem mods already loaded. Load the kernel, find the encrypted root, and *Bam* there's your newly-readable filesystem. This isnt' rocket science.
This space for rent. Call 1-800-STEAK4U
What if you did something silly like placing your filesystem encryption module on your encrypted filesystem... Get outta that one Houdini ;)
(Wavy flashback lines)
I remember when the Linux kernel introduced modules and in the race to out-module one another, a lot of newbies rebuilt their kernels with every single filesystem as a module. Ahh, those were the days...
Learn to spell: nickel, missile, lose, solely, amendment, speech, kernel, probably, ridiculous, deity, hierarchy, versus
> Stackable vnodes, which are supposed to address this same need, are ancient history
I think I gave up on the idea that Linux would ever know what it was doing with filesystems when I saw this abomination. That's right, Linux not only has no concept of vnodes, it actually uses a union of every single filesystem type for its inode data structure. Search lxr for "vnode" if you don't believe me. So much for generic, to say nothing of modular.
--
I've finally had it: until slashdot gets article moderation, I am not coming back.
> This has been discussed on the LKML. Using a Union for all the various FS specific inode info is a performance hack
... however much faster is unknown, since this is totally untested.
So, they basically wrecked the extensibility of the architecture so a completely theoretical untested performance hack for the x86 architecture *might* go faster
As for that truism, here's another: penny wise, pound foolish.
--
I've finally had it: until slashdot gets article moderation, I am not coming back.
Not to start a filesystem war, but my experience with XFS on linux has been very positive. The install .iso is *polished*. The fs tools are plentiful and high quality. Performance is excellent. If XFS on MIPS is any guide, reliability is top notch too.
I'm glad to see ReiserFS aggressively pushing the technology envelope, but I have nothing but good things to say about XFS, and would recommend it to anyone using a recent kernel who wants a robust journaling filesystem.
As others have said I think there is room for both filesystems going forward.
"Lens flare" - does nothing useful, but hey, you can't be a l33T web designer without it.
How many plugins have you seen that fail (crash or fail to complete the operation) when you try them on a large image?
;)
Erm. None, really. Based on the architecture of the file system, I think plugins would be an interesting and, most importantly, plausible concept.
No one wants to take a risk of widespread data corruption or data loss.
You've obviously never seen the Linux zealots running the -dev releases of Debian. Sheesh! Talk about trashing a system sometimes...
Cheers,
levine
I suppose you could make "zipmagic for linux" (letting you treat .tar.gz files as directories). What else could you do?
.. to make the same kind of mistake.
/etc a partition of its own.
/etc/fstab to mount /etc.
I once in my very newbie-days made
Too bad I needed
I really hope that when Hans and Co. think about how to extend filesystems they'll put their weight behind some generic facility and not just pull some new ReiserFS-specific interface out of their collective ass. Stackable vnodes, which are supposed to address this same need, are ancient history. NT has had filter drivers - not just for filesystems but for just about anything - since day one, and they have proven invaluable as a way to implement new functionality without writing a whole new filesystem. Erez Zadok has done some really cool work on a system called FIST for this sort of thing. All of these should serve as sources of inspiration at the very least, and of code in some cases.
On a different note, journaling is not really something that can be added as a plugin, extension, or filter. Even if you have a generic journaling interface or layer separate from the "core" filesystem code, you have to make sure that that core code is "journaling-friendly". This involves thinking about ordering and atomicity, plus callouts and hooks between the main FS and the journaling code. It's pretty pervasive, and not something that can be done "stealthily" by adding a plugin or filter to a filesystem that was never designed (or, as in ext3, redesigned) with journaling in mind. Encryption, compression, virtual namespaces, all sorts of other things can be added as plugins or filters, but not journaling.
Slashdot - News for Herds. Stuff that Splatters.
I've had my /usr and /home partitions on ReiserFS for a long time, and had zero trouble with them (Mandrake 7.1 came with ReiserFS integrated into the 2.2 kernel).
The model of Red Hat, Caldera, et. al. isn't selling software. It's selling support. Many years ago (15-20) Jerry Pournelle, writing in Byte, said that in the future (today) money wouldn't be made from selling software, it would be made by selling support and documentation. Certainly there are many companies that already make money from all or part of this model. They also provide support for open source developement.
Best Slashdot Co
Nice to see that DARPA (the Defense Advanced Research Projects Agency) is still funding useful things like this. Remember that they funded the internet when it first started. They're usually up to something interesting.
Best Slashdot Co
Hans said:
"SQL has been crippling the database industry for decades"
I'll admit it's not the best syntax to manipulate
a database but before SQL there was no uniformity
in database access. At least you don't have to
learn a new "language" for each database you
maintain/access. I don't see how it's crippling.
I am curious about any solution he would propose.
the pedant notes that when going whoring, getting screwed and getting out ok are not mutually exclusive.
Hans Reiser is also going to speak at LinuxTag 2001 in Stuttgart, Germany. From the LinuxTag website:
GPLing the code is also bad policy because people should be able to use technology that's developed with their tax dollars for any purpose. The GPL prevents enterprising programmers from using the code in their own products and making money from those products. It therefore seems to me that DARPA should either NOT fund such a project or insist that the code that is generated be placed in the public domain -- or at least licensed under the BSD License or the MIT license. After all, it's our tax dollars.... None of us should be denied the use of the code.
--Brett Glass
The export restrictions on cryptography have been lifted.
Yes, the crashing of the plugins is because of bad programming. Exactly my point.
While the plugins for Photoshop are great and very powerful, there is one problem. How many plugins have you seen that fail (crash or fail to complete the operation) when you try them on a large image? I think it will be very hard for enough high quality, stable, trustworthy plugins to be available. I'm sure compression, encryption, and a few other basics will be great. But will they do the same as for Photoshop? I doubt it. No one wants to take a risk of widespread data corruption or data loss.
I think alot of people are to focused on XFS, Ext3 and ReiserFS competing with each other. Why can't Linux have three filesystems???
I like ReiserFS and I found that it's completly stable, it has never crashed. Others like XFS and that is fine by me, just don't tell me that my choise is wrong, I'll decide that for myself.
The great thing about Linux is that it let us choose what we want and don't give us what some dude i Redmond thinks we want. Let us have have 3,4 or 8 filesystems, then the user can choose what filesystem he or she want to use.
I didn't say it was your fault. I said I was going to blame it on you.
Quote: "we hope that plugins will do for filesystems what they did for Photoshop."
What, make them run 55% faster on a Mac than on an equivalent PIII?
From the interview
Not every day a big name in software grants your wishes...
Special Relativity: The person in the other queue thinks yours is moving faster.
For almost ten years I worked on networked filesystems, and the US government prevented us from adding any kind of "hooks" on the basis that they might be used by furriners to add prohibited crypto.
Now the US Defense Department is paying a bunch of Russians (oh, the irony, the irony) to do exactly this!
And the Bush administrations is paying Osama bin Laden 40 mil for his valiant efforts on behalf of the War on (some) Drugs?
What's next? An invitation for Fidel Castro to spend the night at the White House and drop E with President Junior? Saturday Night Live quits doing White House satires because "we just can't keep up"?
It's www.reiserfs.com
They way I see it is that Reiser is looking to make a mark with having the latest & greatest feature list, XFS is looking to have the the most stable & reliable filesystem, and ext3 is a way to have people keep their existing ext2 data and add journaling support.
Reiser is planning on selling their modules in the future, make a new feature to be sold and change the previously sold module to be free. Their entire business model depends on them having newer and newer features, which is great for people who are wanting/needing feature over stability.
XFS is leaning more towards the datacenter type of situation, it may not have the latest and greatest, but it will work reliably, constantly, and with great performance. XFS is looking towards Linux as their OS platform, they have to give the same quality of filesystem they had on Irix to their customers who demand that quality. (when buying a multi-million dollar 512proc numa system they tend to require lots of stability).
On the competeing filesystems, Steve Lord from the XFS mailing list said it probably best:
"...I have never regarded the different filesystem on Linux as being in direct competition with each other, there will always be benefits to using each different filesystem for their strong points. Plus having several filesystems under active development means that there will be a tendency for the developers to make theirs the best, the implementations improve, and everyone wins."
Reiser will be used for the things it's good at (squid, mail spool, new features) and XFS will be used for the things it's good at (larger files, NFS server, stability). They compete only that they are filesystems, but what they are designed to be good at are two different things.
I won't be rushing again to stick ReiserFS on an NFS system any time soon....
Actually he feels that it's relational databases that are crippling. In his whitepaper advocating a unified namespace, he proposes a hierarchical model that fits to data rather than fitting the data to the model.
- "Random noise" - randomly copy from
/dev/urandom instead of writing the actual file
- "Motion blur" - bit rot on very fast disks
- "JPEG compression" - lossy compression for your data!
- "Mosaic" - increase fragmentation
- "Colorize" - rot13 all
.so files
- "Watermark" - digitally sign all files (also system files)
How about just making a toolkit for porting Gimp plugins?^]:wq!^M
That should be pretty easy - create a kick-ass piece of software that everybody uses & name it after yourself (like he did :).
Blue skies... Barthie burgers... girls.
okay....
And you expect to boot off of your encrypted/compressed drive how again?
Forgive me if I'm wrong, but you need to read the module files off the hard drive in order to load them and use them, but you need the modules loaded to read the drive... Kind of a chicken before the egg problem?
Lol. (Yea, I know. Boot partition is a standard FS the kernel and bootstrap prog can read and keep your FS module files on a non encrypted/compressed drive. But still, it's funny. I'm sure that more than a few newbies are going to forget about this little problem, move their module files to the encrypted drive, and then when they next reboot it won't work. *grin*)
"I now tell people that I firmly advocate making the names of the programmers as prominent on the code as possible, especially when it is free software, and the sponsors should also be prominent on the code. People who do something should get either fame or money as their reward for it, and for free software to succeed we have to maximize the amount of fame that goes around. I think it is a complete rip-off that Stallman's name is not on every Linux distro (or at least the name GNU). It hurts his ability to command attention from people outside the industry who don't know much about us, and it is simply unfair. Failure to cite properly is considered an offense in academia, and it should be considered an offense in the free software community."
There's always sufficient, but not always at the right place nor for the right folks.
How stable can a FS with plugins be? Some third party's bad code can make your files go bye-bye. I'm sure the security implications would be heavy too. Viruses planted in plug-ins would be too easy to implement and distribute.
Before going into implementing new features, I'd prefer if they made ReiserFS rock solid first.
At the moment, and after hearing recent scary stories about problems with ReiserFS (but I personnally haven't ever had any big trouble), I switched to XFS and enjoy it just as much.
I also find SGI much more quick at getting things stable and finished, which is fairly important IMHO.
Matthias
-- Life wasn't meant to be easy...
... or James V.Fat, author of the VFAT file system. Or Sid Mips, who has a whole architecture ... the list is endless.
port named after him. Or Georg Keyboard, Enzo Tpqic02,
Every bloody emperor has his hand up history's skirt [Peter Hammill/VdGG]
Encryption debuted in Win2k
RE: ReiserFS, revolutionary...
From what I have read thus far on ReiserFS and how they are planning on charging for plugins in the future along with ports, XFS seems like a slightly better choice for those concerned with the aesthetics of a project. I don't mean to be a "FREE!" bigot, but I wouldn't want to run something on my system next to the very core of Linux that didn't match the philosophy of it's neighboring code. I hope XFS finds a place in the kernel -- it deserves it a lot more -- and politics casts ReiserFS out. I know the argument: "People have to eat!" Well, that's fine and dandy, Certainly ReiserFS should be available. I just don't think XFS should be left out of the tree if it is embracing more platforms and has no commercial "options" (thus, making it more free.)
From the ReiserFS FAQ: "15.Can I use ReiserFS on other architectures than i386? Yes, on DEC Alpha. You may be able to easily bribe us to do the port though...."
From the XFS FAQ: "Q: Does it run on platforms other than i386? The current XFS tree seems to work just fine on ppc now (aside from some trivial compile fixes). It also runs well and is getting sporadically tested on the alpha, sparc64 and ia64. But on all of those platforms it is
not as well tested as on i386, but so far there are no major problems on those platforms known. All in all it looks like XFS will be running across a lot of platforms fine soon (with all the platforms above we have 32/64bit and little/big-endian architectures supported. If you run it on a platform not mentioned here please
let me know so that i can add it. Also an important note is that XFS is inherently platform independent in the on disk layout - so it should be possible to move a XFS disk from one linux platform to another out of the box."
mwtr / THIS SIG HAS BEEN PRAYED OVER AND MAY BE USED AS A POINT OF CONTACT (ACTS 19:12)
What are the disadvantages of putting it into kernel space?
Maybe ReiserFS is the best we are going to get and the only way in which people are going to migrate to something less impoverished than the current UNIX file system data model. But I still think it's not a good engineering tradeoff overall.
She has a darn good point on this. If all that Linsux companies can do is produce pieces of plastic to buy, then I am definitely going to take my money elsewhere. The reason I buy software that is not open source is because I can't get the sources.
Is open source bad? No, just the business model. It's a decent community, it just doesn't work in the corporate world. I know very few people who, faced with a free choice and a paying choice, both of which are perfectly legal, will take the payment route.
This