Reiser On ReiserFS's Future And More

Steven Haryanto writes: "This one's from Indonesia. InfoLinux did an email interview with Hans Reiser, in which he explained about the ReiserFS project plan and the new Namesys business model. Mr. Reiser told me that Namesys recently received $600K funding from DARPA to include encryption in ReiserFS v4.0." Dig this quote: "We are going to add plugins in our next major version, and we hope that plugins will do for filesystems what they did for Photoshop." Mmmm -- encrypted, compressed, journaling, extensible filesystems. Reiser also touches on issues of international software development and how programmers can achieve fame.

Comments

Mmmm... CPU cycles

[Lally Singh]

Mmmm -- encrypted, compressed, journaling, extensible filesystems.

May ask for a moment of silence for the CPU? First IDE, now CPU heavy filesystems? Also, one of my first questions is, how do you compress a filesystem without having problems with either:

• Making seeks tough to calculate, or worse, requiring many disk accesses to get the right address
• Performance problems with things like writes in the middle of the file, causing the dictionary for that part of the file to overfill and the whole portion of the file to need recompression

Perhaps I'm just not up on the latest compression techniques (most likely), but those questions just popped in my head.

Either way, this is just further down the road of increasing CPU requirements just to drive the friggin disk. Ick. I miss SCSI.

--

Re:Mmmm... CPU cycles

[Hater's Leaving, The]

Very good post. You ask all the right questions.

Compressed FSs have always had this problem. The best solution that people have comeup with is one that we have already implemented - caches.

In particular, the hairy writing to disk stage (where the compressions and on-the-fly construction of dictionary etc. takes place) only needs to be done on file closure.

For some files, such as those that are frequently opened for writing, it's perfectly possible to have backgrounded compression. Basically you don't compress the file until a later point in time, you store it uncompressed on the hard disk and mark it as such. It's a seperate house-keeping job to actually compress the files (when they are 'stable' for some heuristically defined stability function (e.g. closed >10 seconds ago, or yesterday, or whatever))

Another helpful technique is to chunk the files, so that you only ever have to seek from the nearest chunk boundary. This simply shrinks the amount of data that suffers from the seek problem, and often (though _not_ always) reduces the compression ratio. (e.g. a large executable could possibly be made smaller, as the code and initialised data could end up in different chunks, and the compression model could only need to adapt to one type of data.)

Note, however, that we are not talking about _a_ compression plug-in. We are talking about compression _plug-ins_ (i.e. plural). You can chose your plug-in depending on the requirements.
e.g. A FS for infrequent writes and frequent whole-file reads, such as a document management server, and could use a 'slow' compression, fast decompression, no-seek algorithm.

From purely personal opinion, I believe with current HD transfer rates giving the CPU the decompression task is a better than reading bigger files. However, I'd not swear to that until I've played around with it and tested it thoroughly, with all my favourite file types. (I tend to have 500MB highly-compressible log files from what I do, so this really pushed my buttons!)

I do go on sometimes...
THL.
--

Ooops...

[Psiren]

What if you did something silly like placing your filesystem encryption module on your encrypted filesystem... Get outta that one Houdini ;)

Re:encrypted, compressed, journaling....

[jimhill]

(Wavy flashback lines)

I remember when the Linux kernel introduced modules and in the race to out-module one another, a lot of newbies rebuilt their kernels with every single filesystem as a module. Ahh, those were the days...

Plugins do the same as for photoshop?

[stilwebm]

While the plugins for Photoshop are great and very powerful, there is one problem. How many plugins have you seen that fail (crash or fail to complete the operation) when you try them on a large image? I think it will be very hard for enough high quality, stable, trustworthy plugins to be available. I'm sure compression, encryption, and a few other basics will be great. But will they do the same as for Photoshop? I doubt it. No one wants to take a risk of widespread data corruption or data loss.

Has the world gone totally mad?

[peccary]

For almost ten years I worked on networked filesystems, and the US government prevented us from adding any kind of "hooks" on the basis that they might be used by furriners to add prohibited crypto.

Now the US Defense Department is paying a bunch of Russians (oh, the irony, the irony) to do exactly this!

And the Bush administrations is paying Osama bin Laden 40 mil for his valiant efforts on behalf of the War on (some) Drugs?

What's next? An invitation for Fidel Castro to spend the night at the White House and drop E with President Junior? Saturday Night Live quits doing White House satires because "we just can't keep up"?

You got the url wrong

[iplayfast]

It's www.reiserfs.com

Re:great news, xfs

[InsaneGeek]

They way I see it is that Reiser is looking to make a mark with having the latest & greatest feature list, XFS is looking to have the the most stable & reliable filesystem, and ext3 is a way to have people keep their existing ext2 data and add journaling support.

Reiser is planning on selling their modules in the future, make a new feature to be sold and change the previously sold module to be free. Their entire business model depends on them having newer and newer features, which is great for people who are wanting/needing feature over stability.

XFS is leaning more towards the datacenter type of situation, it may not have the latest and greatest, but it will work reliably, constantly, and with great performance. XFS is looking towards Linux as their OS platform, they have to give the same quality of filesystem they had on Irix to their customers who demand that quality. (when buying a multi-million dollar 512proc numa system they tend to require lots of stability).

On the competeing filesystems, Steve Lord from the XFS mailing list said it probably best:

"...I have never regarded the different filesystem on Linux as being in direct competition with each other, there will always be benefits to using each different filesystem for their strong points. Plus having several filesystems under active development means that there will be a tendency for the developers to make theirs the best, the implementations improve, and everyone wins."

Reiser will be used for the things it's good at (squid, mail spool, new features) and XFS will be used for the things it's good at (larger files, NFS server, stability). They compete only that they are filesystems, but what they are designed to be good at are two different things.

Photoshop plugins?

[egjertse]

Looking forward to these filesystem plugins:

• "Random noise" - randomly copy from /dev/urandom instead of writing the actual file
• "Motion blur" - bit rot on very fast disks
• "JPEG compression" - lossy compression for your data!
• "Mosaic" - increase fragmentation
• "Colorize" - rot13 all .so files
• "Watermark" - digitally sign all files (also system files)

How about just making a toolkit for porting Gimp plugins?

Fame

[3G]

Reiser also touches on issues of international software development and how programmers can achieve fame.

That should be pretty easy - create a kick-ass piece of software that everybody uses & name it after yourself (like he did :).