The Linux Filesystem Challenge

Joe Barr writes "Mark Stone has thrown down the gauntlet for Linux filesystem developers in his thoughtful essay on Linux.com. The basic premise is that Linux must find a next-generation filesystem to keep pace with Microsoft and Apple, both of whom are promising new filesystems in a year or two. Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"

Comments

New FS

[stecoop]

Linux must find a next-generation filesystem to keep pace

What are the winds of change saying? R..E..I..S..E..R...4...

Re:New FS

[AstroDrabb]

Reiser4 is going to be great. Here are some of the features for those who don't like to click-n-read:

1. * Reiser4 is the fastest filesystem,
2. and here are the benchmarks.

1. * Reiser4 is an atomic filesystem, which means that your filesystem operations either entirely occur, or they entirely don't, and they don't corrupt due to half occuring. We do this without significant performance losses, because we invented algorithms to do it without copying the data twice.

1. * Reiser4 uses dancing trees, which obsolete the balanced tree algorithms used in databases (see farther down). This makes Reiser4 more space efficient than other filesystems because we squish small files together rather than wasting space due to block alignment like they do. It also means that Reiser4 scales better than any other filesystem. Do you want a million files in a directory, and want to create them fast? No problem.

1. * Reiser4 is based on plugins, which means that it will attract many outside contributors, and you'll be able to upgrade to their innovations without reformatting your disk. If you like to code, you'll really like plugins....

1. * Reiser4 is architected for military grade security (sponsored by DARPA). You'll find it is easy to audit the code, and that assertions guard the entrance to every function.

Notice the plugin feature. This will create endless possibilities for what you can do with the file system. Want to tie a DB/SQL search function in to it? Write a plugin, want special security? Write a plugin. Tons of possibilites with ReiserFS4 and it is _very_ fast. This is hands down better then the MS "a filesystem as a DB" approach. ReiserFS4 will be like Firebird, lean-n-mean-n-fast. Want more features, grab _your_ favorite plugins!

Re:New FS

[prisoner-of-enigma]

I've been shouted down before about this, but I'm going to keep asking for it because it's a useful feature for my company: what about per-file compression in the file system? Now before anyone has a hissy fit, let me explain.

We output a lot of digitally-created video files that are huge (think HDTV resolution). Most of these files are output uncompressed because either (a) the file format doesn't support compression or (b) the multimedia program doesn't support compression. Either way, a few minutes of HDTV-quality uncompressed video will absolutely destroy a few hundred gigabytes of space in no time.

We have to hold on to some of this video for quite some time, but we only need to get at it infrequently. It's too big to fit on DVD-R's, tape is too slow, ZIPping it up hinders easy access later, and removable hard drives are expensive. File system compression, on the other hand, does wonders. We routinely get 60%-80% compression on archived video files, and it's allowed us to stretch our disk capacity a long, long way because of it.

We've considered archiving our video in some kind of compressed streaming format like AVI, Quicktime, or MPEG-2, but none of these offer lossless codecs that are appropriate for us, and we're unwilling to accept using a lossy compressor.

So, I ask the question again: when, if ever, is anyone going to implement file compression on a Linux file system? Or does it already exist but is buried somewhere in some arcane HOWTO or website?

Re:New FS

[Anonymous Coward]

ReiserFS4 will be like Firebird, lean-n-mean-n-fast.

um ya but they're still behind in that crucial name changing area. Hey maybe we can have a plugin change file names at random, that would surely out-do firebird.

Re:New FS

[sisukapalli1]

Quick googling got these. They may provide you with some leads. If you find more, please post :)

http://sourceforge.net/projects/e2compr/ (Ext2 Compression)

http://squashfs.sourceforge.net/ (Squashed - Read Only, don't know what that means)

S

Re:New FS

[Lifewolf]

We've considered archiving our video in some kind of compressed streaming format like AVI, Quicktime, or MPEG-2, but none of these offer lossless codecs that are appropriate for us, and we're unwilling to accept using a lossy compressor.

It doesn't sound as though "compressed streaming format[s]" are what you're really looking for, and AVI isn't a streaming format in any case. However, there are archival-type video codecs that may suit your needs:

• BitJazz SheerVideo is an excellent, relatively quick, lossless, 4:4:4 video codec. The encoder is fairly inexpensive, the decoder is free, and they're working on a 10-bit version.
• Digital Anarchy Microcosm is a lossless, 64-bit, RGBA codec. It's excellent if you're not working in YUV. The encoder is even less expensive than SheerVideo, and the decoder is free.
• Apple Pixlet is quite nice, though not completely lossless.
• Motion JPEG might be considered if you need pretty-much-visually-lossless but not mathematically lossless.

In a perfect world, you'd have one of these working behind the scenes in some sort of network storage device in a manner similar to the dpsVelocity VTFS. If you haven't worked with an editing system that uses VTFS, I recommend getting a demo.

Re:New FS

[prisoner-of-enigma]

It doesn't sound as though "compressed streaming format[s]" are what you're really looking for, and AVI isn't a streaming format in any case.

If you consider "streaming" to mean something like RealMedia or other web-based streaming codecs, you are correct. However, working in the DVD/Digital Video/Multimedia fields, we do refer to MPEG-2, AVI, and so forth as a "streaming" format because it is composed of one or more "streams" of content. Basically, the different between what we have now (tens of thousands of individual files, each one representing a single frame of video) and a "streaming" file is that it compresses all those individual files into one big file.

However, there are archival-type video codecs that may suit your needs:

Thanks for the listing! I will check into these.

Re:New FS

[FleaPlus]

If you're concerned about compression speed, you may want to take a look at LZO. It's got incredibly fast compression, and even faster decompression. I think it was even used on the Mars Rovers.

Re:New FS

[vandan]

Damned right.

With Reiser3, doing `emerge -up --deep world` on my Gentoo box would usually take about 10 seconds after the progress spinner had started.

Now with Reiser4, it takes about 2 seconds after the progress spinner starts.

The speed really is absolutely amazing.

And from what I've read of Reiser4, it has all the database niceties for managing files and contents of files that WinFS is promising. Of course Reiser4 currently exists and is working on my home gaming maching and 4 machines here at work. WinFS is just marketing speak.

Re:New FS

[prisoner-of-enigma]

It's a pity no one has ever fully finished the NTFS filesystem module for Linux. I understand that payware solutions are available, but the "EXPERIMENTAL" read-write NTFS module has been around for years and years with nobody finishing it up. Right now you can write to an NTFS volume so long as you don't change much of anything...yeah, uh huh. That's real useful. And if you do accidentally change something, you can totally fsck up the volume, or at the very least you have to do a CHKDSK before anything else can access the volume. That's about as useful as a square bowling ball.

Re:New FS

[iamacat]

Who said I encrypt first?

Re:New FS

[Clover_Kicker]

>Thanks but no thanks. I'd rather my /etc/fstab not be corrupted in the
>event I edit /etc/hosts , try to save and sh*t happens.

Relax, this is old news! The Novell Netware filesystem did this 10 years ago, they called it "sub-block allocation". I never had a problem with it on my servers. I've never heard of anyone else having problems with it, either.

Re:New FS

[mingot]

I believe this is where someone is supposed to call you a newb, tell you to shut the fuck up, and then yell you to write it yourself.

Or did I get the order wrong?

Hans Reiser's vision of the future

[alanw]

Hans Reiser has written a white paper containing his thoughts on the design of the next major version of ReiserFS.

Re:Hans Reiser's vision of the future

[minginqunt]

In addition to Reiser4, there are a whole whost of projects that aim to provide all or part of what BFS achieved, Spotlight (MacOS X Tiger) and WinFS will achieve.

This includes Beagler/Dashboard

http://www.nat.org/dashboard
http://www.gnome.o rg/projects/beagle/

And of course, the ambitious Gnome Storage project, being pushed by Seth Nickell. He recently wrote a paper comparing all the technologies, found here:

http://www.gnome.org/~seth/blog/document-indexin g

Re:Hans Reiser's vision of the future

[MouseR]

Apple's Spotlight isn't a file system. It's a search engine that uses and maintains metadata stored in the file system.

HFS+ is the current OS X file system, and that of Tiger (next revision of OS X) as well. Spotlight uses HFS+'s built-in metadata support to enhance it's search capabilities. What Tiger offers more to application developers is an API to add metadata to documents, something that was limited until now.

Re:Hans Reiser's vision of the future

[addaon]

What Tiger offers more to application developers is an API to add metadata to documents, something that was limited until now

What Tiger offers is a way for application developers to DECLARE metadata in their document formats... most formats have metadata of some kind already (in an mp3, id3 tags; in a image, resolution etc.; in a source file, dependencies and exported symbols); what Tiger lets application developers do is tell spotlight how to find the information that's already there.

Now, this may lead to future formats that have more comprehensive metadata, since there's now more power to that metadata... but that's not the direct idea.

Re:Hans Reiser's vision of the future

[ninejaguar]

Here are further thoughts on that, from 3 years ago.

= 9J =

Don't try to keep up with Microsoft and Apple

[suso]

Instead, try to keep up with the demands and needs of users.

Re:Don't try to keep up with Microsoft and Apple

[Anonymous Coward]

Don't try to keep up with Microsoft and Apple. Instead, try to keep up with the demands and needs of users.

In this case, they're one and the same.

Re:Don't try to keep up with Microsoft and Apple

[jilles]

Actually that involves keeping up with the rest of the field as well. Not every feature MS adds to their OS should be duplicated. But some features are useful and should be considered.

MS has basically announced/demonstrated most of the new features that are in longhorn. Effectively that has given the linux community two years to come up with competing features. Adding database features to a filesystem makes sense, beos has demonstrated that you can do some nifty stuff with it and both apple and MS have anounced to do this.

The linux community however is divided. You can install reiserfs, maybe develop some tools that use some of its more advanced features but that doesn't fundamentally change anything if openoffice, KDE and Gnome and other programs don't coevolve to use the new features.

The same goes for stuff like avalon. While everybody is still talking about how such technology might be used in OSS projects like mozilla, Gnome, MS is well on their way of implementing something that may actually work.

Filesystems with rich metadata were already a good idea ten years ago. The OSS community has talked about them where others have implemented them. Two years of more talking would be fairly consistent. IMHO the OSS community is underperforming in picking up new technology and good ideas.

Re:Don't try to keep up with Microsoft and Apple

[kfg]

Users often demand and think they need all sorts of pointless, worthless, daft shit. Commercial companies, of course, have to cater to this, and the less ethical directly exploit it ( I'll sell you speaker cables that I've meditated over while sitting under a mystical waterfall to infuse them with energy and align their molecules, only $2000 a set. If you can't hear the difference it's because your chakras are blocked, but don't worry, I've developed a homeopathic remedy, only $20 a bottle. Oh, they only work while listening to my taped lecture series though, just $499. Remember to sit on my special magnetic pad at the same time (available to members only)).

How's about this for a better idea, instead of trying to keep with Microsoft try to keep up with sound software engineering principles in designing our file systems?

There may even come a time when the required action to impliment this idea is to do nothing.

KFG

Re:Don't try to keep up with Microsoft and Apple

[dasunt]

As a linux user, I don't sit back and think "this filesystem sucks". For the most part, I'm happy with ext3.

When I do try to make a wishlist, the only things I really want is KDE's IO Slaves integrated into the system at a lower level so that all programs can use it, and a more secure version of NFS. That's it. Perhaps some sort of revision control on certain files, but RCS works fine for me.

I don't want data forks -- it creates more problems (with transferring files) then it solves.

For a similiar reason, I don't want my filesystem to be a DB. I'm happy with files. Damn happy. I don't see what problems a database solves.

Just my $.02.

Re:Don't try to keep up with Microsoft and Apple

[vsprintf]

Well, the key to a database filesystem will be seamless data entry and simple, powerful access to search and reporting features.

I'm not sure what you mean by "seamless data entry." Maybe I missed something in the article. Are you suggesting people will be willing to provide meaningful metadata for a file when they aren't even willing to provide a meaningful file name? And "powerful access to search and reporting features"? As opposed to wimpy access to search and reporting features? It sounds a bit like marketspeak to me.

It seems to me that this push for "database filesystems" is from people who want to throw everything in one directory and name their files "1", "2", "3", etc. It just dumps more work on the CPU to handle more filesystem overhead. When you give a filesystem the attributes of a relational database, you also get all the related problems, overhead, and constraints. Records in a database table are very closely related. Files in a directory may have nearly nothing in common.

What I'm guessing is that we'll see a thinly veiled front end to grep -- and as much as I like grep, it's a serious pricker bush.

What is WinFS, except an admission that you can't *grep* or *strings* a Windows *folder* full of files? Files that have secret, constantly changing formats require metadata to make them searchable, which makes the filesystem bigger and slower. Being able to see the same file from two different directories doesn't impress me. I can do that with ln -s (if I really have a valid reason for it). I really don't see a lot of benefit to *nix in following Windows down this road. I'm sure I'll get enlightened shortly. :)

Re:Don't try to keep up with Microsoft and Apple

[e_AltF4]

Instead, try to keep up with the demands and needs of users.

Very true - locate, find and grep do their job quite well, so why should i dedicate 1/3rd of my HDD to index and database space ?

Re:Don't try to keep up with Microsoft and Apple

[mvpll]

Most filesystems all ready handle seamless metadata entry. A files directory, filename, permissions, ownership, ctime, mtime, etc are all metadata stored by the filesystem and few people want to argue that this metadata is unwanted and needless filesystem bloat.

Not all information is text based either. How do you store family photos in an easy to find manner? What directory/filename structure will allow you to quickly find the photo of daughterA at locationB taken sometime in yearC that doesn't contain ex-partnerD?

"Virtual folders" are the _best_ feature of the Evolution mail reader and beat the pants off a one-time search for something. This is simply because no matter how you organise your data, sometimes it just makes sense for it to stored under two places at once. For example, the directory which holds photos of daughterA and the directory which holds photos taken at locationB. It would seem to me that storing this data multiple times (or making lots of soft links) creates far more overheads then storing some extra metadata which is available to every program you run.

Also, relying on metadata created and stored by an application leaves you beholden to that application.

Whether or not people will actually make any use of a FS metadata capabilities is a seperate issue. I don't want to spend all my time re-arranging directories and shuffling my data around to make it easier to find. I bought a computer to do such menial tasks...

Recall that Mark Stone...

[tcopeland]

...wrote "Open Sources", which you can read/buy here. He's a fairly savvy fellow...

easy answer

[dAzED1]

nfs4, with solid integrations for auth servers (ldap to active directory, etc).

We live in a network-based universe. Local filesystems are already good - whether its just continued development in Reiser, or whatever else.

Nfs4, though - its like afs, only without the sucky stuff. AIX is now including nfs4 in its AIX5.3 release, even! With the Big Dog on board, we should realize there's wisdom in that direction ;)

Re:easy answer

[Anonymous Coward]

Man, you totally miss the point. NFS is not a file system (don't be fooled by the name), it's a network protocol. The files provided by a NFS server have to be physically stored on some (real) filesystem, like ext3 or reiserfs.

This is very much like saying "the future of filesystems is apache2, local filesystems are already good, now we have to concentrate on apache2".

ReiserFS is pretty damn good

[bigberk]

Hans Reiser has some interesting ideas about the role of a modern file system. Here's a recent USENET post describing some of the immediately visible features of reiserfs v3. Some people have said that there was corruption in the past, but I think there are no longer any problems in recent 2.4 kernels. Namesys is now developing Reiser4, which appears to be more flexible (still needs time to stabilize though). If I had to put down my money on a future filesystem though, it would be ReiserFS.

Why not use...

[AnthonyPaulO]

... the BeOS file system? I heard that it was supposed to be the latest and greatest, the OS to end all OSes.

Re:Why not use...

[0racle]

Something that exists can't compete with something that doesn't? Given the time between now and whenever the next version of Windows shows up, I think that there might be some time to whip BFS into shape, assuming it is as outdated as you say, personally I don't know and right now think that adding a relational DB to the file system is just going to have a significant impact on the performance of anything less then the absolute bleeding edge.

I want a transparent filesystem/VM

[valen]

I want a disk equivalent of top - something that'll tell me what processes are kicking the shit out of the disks, and by how much.

If Linux could do that - it's more a VM thing than a filesystem - I'd stick with ext3 for years to come.

Who needs a filesystem in a database when you have a database that lives on your filesystem (updatedb). Get that updating in realtime, with more things (like permissions, access times etc.) and a lot of the work is done.

john

Re:I want a transparent filesystem/VM

[Elwood P Dowd]

Who needs a filesystem in a database when you have a database that lives on your filesystem (updatedb). Get that updating in realtime, with more things (like permissions, access times etc.) and a lot of the work is done.

PR & tech journalists to the contrary, that is all that is involved in Spotlight & WinFS. Spotlight runs on HFS+. WinFS runs on NTFS. Both are databases stored as files on existing filesystems. The only difference between those databases & updatedb is that they may be using better database design (dunno) and that they update in real time via background processes.

I'm wrote a journal entry guessing as much about Spotlight, but since then more evidence has arrived, and I'm convinced that both WinFS & Spotlight are implemented that way. The features & implementation details are quite different, but not the filesystem.

We'll probably eventually start calling these databases a part of the "filesystem" much like right now some people will call mspaint.exe & bash a part of the operating system.

Another solution in search of problem

[CarrionBird]

Of all the problems I can spot in the current crop of OSs, filesystems aren't one of them.

Nobody has come up with a compelling reason or feature to make me want to change filesystems.

Re:Another solution in search of problem

[lightknight]

Right, and how often do you misplace files?

More than three times a week, and that's criminal.

I mean, throwing things about in your home or My Documents directory are fairly standard. How often do you put your (picture) files in a \qw3r3et354t\bchnjc8g45\3j4n45g9u98d directory?

While everyone seems to see WinFS (and associated services) as some sort of search panacea, your ability to retrieve those files is linked to 1.) its metadata and 2.) your ability to recall a search term that appears in the metadata. If your search for "bird" and the metadata specifies "hawk", short of a dictionary search, you still cannot find it. It doesn't matter if the uber search capabilities can span the entire hard drive in 5 secs, and run through multi-dimensional data. You still need a search term, and that search term (in whole or in part) must appear somewhere in the file, be it the filename or metadata.

Essentially, WinFS makes data appear more ordered (assuming you take the time to fill out the fields). Otherwise, it's useless.

Re:Another solution in search of problem

[hackstraw]

Of all the problems I can spot in the current crop of OSs, filesystems aren't one of them.

Nobody has come up with a compelling reason or feature to make me want to change filesystems.

I disagree. Why can I go to google and search the entire web for something and get an answer in less than 1 sec, and I can't do that on my computer or lan?

Re:Another solution in search of problem

[kfg]

When storing files in a database retrieval is dependant on metadata. That metadata is not derived by magic, it requires human input. You may automagically determine that a file is a jpeg, but classification as a jpeg of a bird is a cognative decision. Maybe you aren't even interested in the bird at all, but in the hemlock limb the bird is sitting on. Unless someone has supplied that metadata you just as lost finding the jpeg of the hemlock branch as you are in finding randomly named jpeg of a bird.

Filenames are metadata and are just as much under user control as database metadata, no more, no less.

KFG

Re:Another solution in search of problem

[AstroDrabb]

And exactly how would any search/indexer know that it is picture of a bird? If you add 10,000 randomly named imaages and 1,000 of them were of birds to a filesystem, how would any metadata be added that described those 1,000 images as images of birds? Sure you could find all 10,000 images easily, though there would be no way to find all images of birds unless you or someone else added metadata saying that it was an image of a bird. So for many file formats, there is no less work involved in creating metadata then using a good file name. However, the metadata approach is much more flexible and more fined grained then a file name. It will still require work for many file types that cannot be parsed to guess at the type of content. HTML, text, office documetns will all be easy to build metadata from with no user intervention. Though how would you do it for an image or audio file? Some audio files have "tags", though you cannot be guaranteed that they are valid.

Side Note: Cool posibilites like this show how proprietary formats can really suck and lock users out from searching/indexing their own content. Even offerings from MS in this category will suffer from the inability to search/index proprietary formats. I doubt MS will get specs for all possible file formats out-in-the-wild, thus leaving much of an end-users content hidden from searching/indexing.

Re:Another solution in search of problem

[at_kernel_99]

I disagree. Why can I go to google and search the entire web for something and get an answer in less than 1 sec, and I can't do that on my computer or lan?

A fine question. Which makes me wonder: Is google's next killer app a new filesystem? As the search kings, and rumored Linux users, might they be about to enter the hard drive search / filesystem market? Various pundits speculate that gmail is their first foray into searching beyond the web; surely, at some point, their technology will reach our hard drives. Will it be via a stand-alone tool, or a whole filesystem?

Re:Another solution in search of problem

[runderwo]

Finally, what bloat is today, is necesary tomorrow. Imagine an oracle database on hardware from the seventies. Bloated beyond imagination, dog slow. But since the seventies the amount of data stored in a database has grown tremendiously, to the level where we simply need databases like Oracle or SQL Server to store it.

Bloat isn't an absolute metric. Bloat is the ratio of the memory and execution footprint of a program to the useful work it gets done. A program which does the same amount or less useful work than another program, and which is twice the size in core and uses twice as much CPU time as the more efficient program, is referred to as bloated. It would be illogical to refer to a database server such as Oracle simply as bloated, unless it were possible to point out a competing database server which is equally as useful and which has a smaller footprint, either due to careful coding or to better algorithms. In the case of Oracle, this might be true. But just because it doesn't produce useful work on the hardware of 30 years ago doesn't mean it isn't a well engineered piece of software.

A better label to use would be "complex". To respond to your argument that the only obstacles to db-fs are ignorance and blind conservatism, complex software is undesirable. It increases costs in terms of man hours to maintain it, it increases QA overhead, and it increases support calls from users who came to depend on a feature which was included for completeness, but was never audited for correctness or robustness. People don't code complex software unless they are paid to do it (and usually when a manager is making the technical decisions). This is the reason most open source/free software tools seem to follow the Unix philosophy; simple tools which do one task and do it well, but are yet flexible enough to build into more complex systems. A monolithic database filesystem does not appeal to the sort of psyche which produces open source code for that reason: Complexity doesn't make a programmer's job fun. In order to produce large amounts of code at a low cost as in the open source/free software world, the people behind the engineering of the software need to be having fun, and a complex database filesystem is a rather good example of something which is _not_ fun to produce and therefore unappealing to the hacker sort.

Filesystems are tools

[tikoloshe]

Filesytems are tools that will suit different purposes. Some are good for databases, some for lots of small files, some for lots of reading, some for writing, some for networks, some for streaming.
So to develop a one handy "swiss army knife" of filesystems may not be the best route. For the most part one knows what a system will be doing and can build in the most appropriate filesystem for the job.

Re:Filesystems are tools

[beee]

A good filesystem should be capable of handling all potential applications (for example, FAT32 has found its way into grandmother's desktop and production web servers). Specializing a FS is a huge mistake, and any highly-specific FS introduced to date has been a huge flop. This is not the best route to travel for Linux.

Re:Filesystems are tools

[Anonymous Coward]

for example, FAT32 has found its way into grandmother's desktop and production web servers

Wow, your grandmother has production webservers! Cool. ;-)

Re:Filesystems are tools

[kasperd]

A good filesystem should be capable of handling all potential applications
I absolutely agree. And I actually think the current interface to filesystems is good. I don't want any major changes. Because major changes would most likely lead to all new kinds of metadata that no applications know how to deal with. And whenever your files get handled by a program without this knowledge, you are losing metadata which again means new applications that makes use of the metadata get screwed. So most of this inovation will just give us lots of compatibility problems. If anybody really want to inovate, and produce something good, then they should implement a clever implementation of the existing interface, that works well for different cases, that is both small and large files, deep trees, many files per directory, few files per directory. AFAIK reiserfs and XFS are doing quite well.

(for example, FAT32 has found its way into grandmother's desktop and production web servers).
FAT is a horrible example, because it didn't become this widely used because of quality. Minix' FS is simpler than FAT, it have more features, and it is a lot faster for random access to large files. FAT-16 had problems with small files, because on large partitions you were forced to use large clusters, which means lots of disk space wasted (I have seen 30% waste in real life cases). FAT-32 did improve on the problem with small files, because now you could have much larger partitions with 4KB clusters. But since FAT-32 still use linked lists to find data sectors (like previous FAT versions), FAT-32 is worse at handling large files than any previous filesystem. For example seeking to the end of a 625MB file in 4KB clusters requires following 160000 pointers. Most other filesystems use a tree structure, which means you can typically index the entire file with at most 3 or 4 levels of indirection, which means you need to follow 4 or 5 pointers. Would you try to cache the FAT table to speed up the access? Good luck, you would need 4 bytes of FAT table per 4KB cluster on the disk, so for a 160GB disk you would need to use 160MB of your RAM just to cache the FAT. And this doesn't get rid of the CPU time required to traverse the linked list.

Re:Filesystems are tools

[anothy]

...any highly-specific FS introduced to date has been a huge flop.

i call bull. specialized file systems have been a huge success. lots of commercial database applications use specialized file systems to achieve peak performance. google's published a very interesting paper on the design of their custom file system. these things are only a "flop" in that they don't take over the world - but that's not their goal! they're specialized.

Gnome Storage

[leandrod]

Gnome Storage should be a step in the right direction, and it gets it right by not reinventing the wheel, just using PostgreSQL as its database engine.

This way we can test the waters without messing with the kernel. When the concept is tried, we can decide if we make PostgreSQL a required part of a GNU/Linux system, or a Hurd translator, or whatever.

Re:bah!

[Fjornir]

I'll use flat files and grep like god intended.

But...

[sk6307]

Is there anything that a true database filesystem offers that something like a realtime updatedb index and maybe a background updated glimpse index of /home cant offer?

I have about 18GB of files in my main home dir, and I can search it in seconds with slocate and if I need a content search, with glimpse.

I know that this kind of database FS provides a lot of cool opportunities in terms of meta-data, but how useful is it for non-techies, who usually dont name their files coherently, let alone correct ID3 tags or other other meta-data.

Re:bah!

[eightball01]

Agreed. Although Reiser4 looks very handsome, Linux needs to worry more about functionality and ease of use. Not filesystems. Joe Sixpack cares little for the filesystem even if he has a clue of what a filesystem actually is or does. He wants an easy to use, easy to get up and going machine that will play at least a couple of games, view a little porn, access his email, and perhaps get some work done on the side. If the filesystem is robust, then it's a bonus. Joe Sysadmin worries about the filesystem perks and quirks.

Bump it up to WebDAV or an API ?

[nostriluu]

What do you want these next generation features for? Mainly features like access control, security, robustness, and above all organizing and sharing data.

Why not go higher level, use a reliable and simple underpinning such as ext3fs, with something like WebDAV (Distributed Authoring and Versioning) on top of it? Like SubVersion, it is based on HTTP, with specification for versioning and rich access controls.

Or maybe even go to the level of a Java JSR, so you could have a cross platform API for accessing files so it really doesn't matter what the back end is, KaZAA, Google or a DataSette, as long as your programs have a high level view of the information.

You might even end up with something liek the original TB-L Web, with everyone running their own Web server.

Of course, excluded from the above is performance, which would be ok for office type apps but not something that requires direct disk access, but perhaps the simpler file system would be most suitable for that.

Of course, I'm just rambling here, so would be happy to hear more developed responses to this suggestion...

Re:A year or two, or... three?

[bsd4me]

In the early days of Linux (1992/1993ish) a new filesystem seemed to appear each week. Most were pretty unstable, though. My first Linux machine, which started out as v0.11, kept its root partition as minix-fs for a long time for this reason (and also because I didn't feel like recreating my system).

Compatible

[cubicledrone]

Just make sure it is incompatible with all the current applications so we can rewrite everything. Add a cool feature or something too.

Re:Is it?

[furball]

This explains everyting you need to know. But basically FFS is a compatibility thing. Apple still recommends its HFS+.

Linux.com

[maelstrom]

So Joe Barr who writes for Linux.com, submits a story to Slashdot (who owns Linux.com) written by another OSDN guy. Good job for editorial independance.

File versioning

[Alain Williams]

I know that some don't like it, but we need the option of file system versioning, so that if/when you delete half the lines in your letter/program/... you can get them back from the previous copy on disk.

There is an expectation that the application should do it, that means extra code in each application and they all do it slightly differently.

OK: need an O_NOVERSION on open(2) if the app *really* doesn't want this - eg a database.

Re:File versioning

[prisoner-of-enigma]

I know that some don't like it, but we need the option of file system versioning, so that if/when you delete half the lines in your letter/program/... you can get them back from the previous copy on disk.

Interestingly enough, Microsoft has implemented just that very feature in Windows Server 2003. They call it "Shadow Copy Volume" and it's accessed through a "Previous Versions Client" add-on to any file's properties. If you overwrite or delete a file on a Shadow Copy Volume-enabled network share, you can just right-click on the file, select "Properties," and go to the "Previous Versions" tab to see all the prior versions of that file. You can recover any one of them you like and save it anywhere you like. Further, the server only saves the deltas between changes, so it's very space efficient.

This is one feature I'd *love* to see implemented on Linux. I don't think this is in Samba yet, is it?

Keep it all modular, please

[JBMcB]

Make the core filesystem small, robust and fast. Journalling, realtime and not much else. Make add-on modules for fancy things like ACL's, quota, compression, encryption, compatability, extended attributes, etc... Put in shims for calling attributes from a database (db or SQL or whatever)

XFS comes close, ReiserFS 4 is nice, too. The most important thing is keeping the base filesystem simple and FAST. You think NTFS is fast? Try deleting a complete Cygwin install (>30K files) It takes AGES, even from the command prompt. I've deleted 15K files (That's 15 THOUSAND files) on Reiser 3 on the same machine, it took a few seconds.

DO NOT make a database driven filesystem. Some day we will have a true, document based desktop paradigm (OpenDoc anyone?) but probably not for several years, until then we need SPEED.

Re:Keep it all modular, please

[chtephan]

DO NOT make a database driven filesystem. Some day we will have a true, document based desktop paradigm (OpenDoc anyone?) but probably not for several years, until then we need SPEED.

Well, filesystems are more or less some kind of database.

Especially the Reiser (3/4) filesystems come very close to being a database.

The database is one big tree. You can see it as (in SQL view) like a single table, where the primary key is indexed and the actual data (the objects) can be of different types.

These types are:

• file contents ("blobs")
• stat data ("inodes": rights, owners, etc... of a directory entry)
• directory (a list of names)

The root directory (/) has a known key, where it can be looked up. There you'll find a "directory" item. It contains a list of names, each name also has a key. Using this key you can find the stat data for that file or directory list or the actual file data.

This data can be located anywhere in the tree, even small parts of file content (like the end of files that don't fill up a block so it would be a waste of space to store it in a full block).

Using this approach everything becomes dynamic. And also very fast because if you have a lot of file, you can write all the data into a contiguous region on the disk and don't have to update some fixed positions on the disk.

Now, reiser4 takes this approach to an extreme:

• Concentrate on making tree operations very fast
• avoid journaling (need to write twice) when possible, write everything to a new (free) location, update the pointer to that subtree later (commit!) which also invalidates the old subtree
• don't distinguish between data and metadata, so everything becomes atomic
• extents instead of blocks

The clue is:

• the core, described above, is indeed only a flexible and very fast database with atomic operations
• the actual filesystem is completely implemented using plugins
• a plugin for the disk storage (how the tree and extents are stored on disk, how data is organized)
• plugins that tie the database core to the Linux filesystems, plugins like "file data", "directory listing" and "statdata"

These default plugins make a filesystem from the database. It's just like reiserfs3 now, just faster.

BUT: You can now add plugins if you want. Plugins to store compressed or encrypted files. Plugins to store additional metadata alongside the files. It's basically the file system of the future. Because it's extensible without changing the disk format.

Speed and Versioning

[silas_moeckel]

OK we have all these DB things that seem more for meta data and seach realy thats a bit secondary to a filesystem. Most filesystems are access by applications for surprise surprise files with very little user files and lots of application files. While it might make snece to mount /home as some DB is a filesystem with piles of indeed and seachable data so the users can be even more clueless to where anything is. The rest of the system needs faster all around and cluster aware from my point of view. Versioning in the FS ala VMS would be a nice thing as well. Disks are the slowest thing on your average system with Gigabit ethernet moving more data than the highest performing single disk in the real world.

Next generation?

[stratjakt]

Lets get the "this generation" filesystems working correctly, shall we?

Solid, universal support for ACLs, and while we're at it, let's fix the whole user/group namespace mess Unix has with it. Let's use an SID-style id like Windows does.

For example: my small network at home, centrally authenticated through ldap.

Now, windows knows the difference between the user "jim" on local machine A, "jim" on machine B, and "jim" the domain user. They'd be shown as MACHINEA/jim, DOMAIN/jim, etc.. The various SIDs take the domain (or workstation) SID and append the UID. So if his number is 100, his sid is "long-domain-sid" + uid. So when you pass around sid tokens, you know exactly which jim you're talking about.

Now in linux, we just have numbers for users and groups. If user 100 on machine A is "jim", user 100 could be "sally" on machine B. Moving that stuff to ldap becomes messy, now I have to reconcile the numbering schemes of all the machines I want to migrate. Ick. And you get all kinds of screwy stuff sharing folders, if you ls it on one machine it'll show wholly different ownerships.. Is the source of about a billlion and one nfs security holes.

And of course, since a file can only have one permission set - owner, user, group, it sure does make for some sucky shit. The lazy among us would just run as root all the time to avoid the whole damn mess.

I know there's a circle jerk of workarounds, patches and gotchas to avoid this, but it should never be a problem in the first place. The basic unix security model is out-of-date, and is the source of many systemic problems.

Re:Next generation?

[prisoner-of-enigma]

You're right, it wouldn't be traditional UID/GID anymore, it would be something better. We are all in this make something "better," aren't we? Or is this whole OSS thing just one big echo chamber?

Is the Linux/Unix community so "steeped in tradition" (also known as stubborness, obstinance, intolerance, and narrow-mindedness) that it willfully clings to an outdated, inferior way of doing things?

Re:Next generation?

[pclminion]

We are all in this make something "better," aren't we? Or is this whole OSS thing just one big echo chamber?

No, the entire OSS community is not an echo chamber, but the kernel development community is. I've seen flamefests caused by some poor soul suggesting very minor changes to, for example, the semantics of pipes. Unix isn't just written in stone, it's laminated and stored in an evacuated nuclear-blastproof case 500 meters underground.

Is the Linux/Unix community so "steeped in tradition" (also known as stubborness, obstinance, intolerance, and narrow-mindedness) that it willfully clings to an outdated, inferior way of doing things?

Again, it is not the community as a whole which is stuck, but the kernel people. I was simply pointing out the truth, not trying to say it's a good thing. Although I think it is wise to be suspicious of radical new ideas until they have proven themselves, I think that many times ideas are rejected for purely dogmatic reasons, and that really restricts innovation.

Re:Next generation?

[jregel]

I suspect that a lot of the differences between Windows and UNIX are due to their respective histories.

UNIX has traditionally been about big systems with multiple users. Networks have been a standard feature for decades. In this sort of environment, you'd naturally use some network-oriented naming service, be it NIS or LDAP.

Windows has grown from a PC background where everything is traditionally local. In a networked environment there is little need for the MACHINEA/user when there is a DOMAIN/user (some exceptions obviously exist).

I am responsible for a network consisting of an NT domain and a number of Solaris, AIX, Linux and Unixware servers. All the *NIX boxes have the same UID/GID schema because we use NIS; not the most secure solution, but suitable for our environment. We interface those users easily with Windows (and Samba) because we only administer two sets of login credentials - NT and NIS (we could do just one using winbind, but that doesn't seem right...).

The UNIX filesystem permissions schema is easy to understand and it works extremely well. Commercial UNIX has had access control lists for years (part of the POSIX standard), but I'm not aware of anyone who uses them in the real world. They are potentially useful, but most people find the UNIX UID/GID does the job well enough for 99% of the time.

Re:Next generation?

[DunbarTheInept]

Changing the filesystem to store real names is trivial actually. The hard part (and the point you seem to have missed) is that you would also have to alter almost every unix program ever written that ever made calls to setgid() or setuid() or getgid() or getuid(). That means that adapting to this new filesystem of which you speak means chucking most of the utility programs unix has right out the window and starting over again. At that point you might as well start over and make a new OS.

Re:Next generation?

[mattdm]

Is the source of about a billlion and one nfs security holes.

Or rather, it is the source of the NFS security hole. But it's okay. NFS4 (or 3, even) with Kerberos totally solves this problem, much more elegantly.

Everyone's all excited by ACLs, but I'm sceptical of their real world value. The "keep it simple" principle of security can't be emphasized enough. With ACLs, you have to really examine the access rights of a given object to figure out what's going on. With the standard Unix user/group system -- with simple directory-based inheritence -- it's completely transparent.

And, most importantly, I've yet to see one thing worth doing with ACLs which couldn't be set up with user/group permissions instead -- and more simply.

Re:Next generation?

[Edward Faulkner]

I, for one, would flame someone who suggested changing the semantics of pipes. Well-defined interfaces are the heart of reliable software. Changing an interface is a very big deal - especially one as entrenched as, for example, pipes.

Of course interfaces need to change sometimes. But first you need to ask how much you're going to break. If the kernel hackers break existing interfaces too much, they risk alienating the users/distros and forking the kernel.

Is this a check on innovation? Absolutely. But I'll point out that Microsoft has even larger checks on innovation - they promise far more backward compatibilty. And it has to be binary compatibility, which is harder than source compatability.

Re:Next generation?

[Malor]

Properly done, an ACL system will give you a MORE secure system, not a less secure one, because there are fewer chances for mistakes.

In the NT 4.0 days, one of the better ways to handle permissions was the 'AGLP' standard. User A)ccounts go in G)lobal groups, G)lobal groups go in L)ocal groups, and local groups get P)ermissions.

This allows a nice level of indirection. I implemented this standard by specifying that Global groups described groups of people, and that Local groups specified access privileges. I built Local groups on each server describing the kind of access privileges they offered. Generally, I would make four groups for each of my intended shares: Share NA (no access), Share RO, Share RW, and Share Admin. I would assign the appropriate ACLs in the filesystem, and then put Global groups from the domain into the proper Local groups. The Accounting group, for instance, might get RW on the Accounting share. Management might get RO, and the head of Accounting and the network admins would go into the Share Admin group.

What this meant was that, once I set up a server, I *never again* had to touch filesystem permissions. Not ever. All I had to do was manipulate group membership with User Manager... with the caveat, of course, that affected users had to log off and on again for permissions to take effect. But this is also true with Unix, in many cases. (when group membership changes).

Note that Windows 2K and XP have more advanced ways to handle this, so don't use this design in a Win2K+ network.... this is the beginnings of the right idea, but 2K added some new group concepts. Under Active Directory, this idea isn't quite right. (I'd be more specific but I have forgotten the details... I don't work much with Windows anymore.)

ACLs are key to this setup, because I can arbitrarily specify permissions and assign those permissions to arbitrary groups.

By comparison, User, Group, and Other are exceedingly coarse permissions, and it is very easy to make a mistake. What if someone from Human Resources needs access to a specific Accounting share, but nothing else? Under Unix, I can't just put them in the Accounting group, because that will give them access to everything under that Group permission. I'd probably have to make a new group, and put everyone from Accounting and the one person from HR into that, and then put the special shared files into a specific directory, and make sure the directory is set group suid. That is a lot of steps. Everything is always done in a hurry in IT, and lots of steps are a great way to make mistakes. Messing up just one can result in security compromise.

In my group-based ACL system, I'd still have to make a custom group, perhaps "HR People With Access to Accounting Share". But I'd only have to touch one user account, the HR person's, and wouldn't have to disrupt Accounting's normal workflow at all, or touch any filesystem permissions.

Instead of a whole series of steps, any one of which can be done wrong, I have only three: Create new Global group, put HR person in new Global group, put Global group in the correct Local group. All done. Hard to screw this up too badly.

Now, I'll be the first to admit that a badly-implemented ACL setup is a complete nightmare. But a clean, well-thought-out ACL system, in a complex environment, is virtually always superior to standard Unix permissions.

Re:Next generation?

[pclminion]

I, for one, would flame someone who suggested changing the semantics of pipes. Well-defined interfaces are the heart of reliable software.

I fully agree, but I'd hesitate to call pipes a "well defined" interface. POSIX attempts to standardize semantics, but there are still mismatches between Unix-like platforms.

The changes this fellow was proposing (don't remember the details, too long ago) were on the same order as the typical differences in semantics between platforms. In other words, I think there's a region of reasonable wiggle-room but the majority of kernel developers seem intolerant of even these small divergences from the status quo.

In addition, people have a tendency to reject new features even if they do not conflict with traditional features. For example, look at the discussions on the kernel list back when futexes were being discussed.

Incidentally, I came up with a nearly identical idea about a year earlier, but I called it "user wait queues." Threads could block on a queue and be woken by other threads. However, my mechanism for waiting on a queue involved a new system call, whereas futex waits are implemented (I think) by using an ioctl to bind the futex to a fd, and then select()'ing on the fd. I did not submit my idea to the kernel people precisely because I felt it would be rejected as "not Unix enough." Usings fds and select() is traditional, but a syscall would be more efficient, IMHO.

It's that sort of conservatism that I'm talking about, not the perfectly reasonable effort to preserve interfaces.

Why?????

[Libor Vanek]

Can somebody explain me WHY should we put things like database, indexing/previewing etc. into the filesystem => KERNEL SPACE!!!! What advantage does it bring?

Any good (XFS, JFS, ext3) filesystem now has nice feature called Extended Attributes which is intented for STORING such a data (like previews etc.). And using user-space server it's much more easier to add plug-ins for various file-formats, "search" plugins etc.

In a nutsheel

[Sepper]

It keep a journal file of last modifications
(ie "Replacing node 37827 with node 5279867....replaced")
One the modification is done, it erases the entry.

After a crash, the system only need to look in the journal file to know which file 'might' be corrupted and restore the old version of each...

At least, that's how I understand it...

Um, Reiser anyone?

[kwelndar]

Anyone? ... Bueller?

But seriously, even though he mentions Reiser, he doesn't seem to consider it's future direction, which is to allow varying degrees of structure, that could include attributes, as the user sees fit. At least that's how I understand it.

dtrace

[DarkMan]

dtrace, due with Solaris 10 does that. So it's not quite a top equivelent, but it does laet you answear your questions ("What processes are kicking the shit out of the disk", and "By how much"), and long with the also useful "In what way" i.e. many small writes, hugh seek to read ratio, or what have you.

It is, however, expert driven, unlike top, which is simple to use. Still, I think that dtrace shows the furture of performance monitoring apps.

Note that dtrace lives partially in the kernel - it's not portable to Linux.

Re:Next premise, please

[Jeff Mahoney]

Actually, both have journaling filesystems.

hfs+ supports a journal (starting with macos 10.2.2 server and 10.3 panther), and ntfs5 supports a journal (starting with win2k)

Do we all have to make the same mistakes?

[dekeji]

All indications are that Linux, Windows, and Mac OS are moving in a common direction with filesystem innovation

Whether or not it is useful, one thing is clear: this sort of thing is not "innovation". Databases as file systems have been around for decades, as has the question of file system metadata. The UNIX choices in this area are by design, not by an accident of history, and the motivations behind those choices are as valid today as they were several decades ago.

Linux is a ways yet from having a fully attributed, database-driven, journaling filesystem. The direction of future development looks promising, though. Linux will certainly compete as the search wars come to the desktop. Linux's value to the enterprise depends on it.

There are two things one needs to keep apart: what functionality do we want to support and how do we want to support it. Search clearly is important. Metadata clearly is important. However, whether a "fully attributed, database-driven, journaling filesystem" is the best way of implementing those features is an open question. There are many possible alternative designs.

And, in fact, it seems right now as if Microsoft is, in fact, not building what the author seems to think they are building, but is choosing an implementation strategy that combines a more traditional file system with user-level databases.

Re:Encrypted filesystems?

[dcowart]

Here's instructions for RH8.0:

http://bob.plankers.com/other/linux/loopback_efs .h tml

The Linux Doc Project also has a HOWTO in their archive:

http://www.tldp.org/HOWTO/Loopback-Encrypted-Fil es ystem-HOWTO.html

You will want to check around though, a lot of the information appears to be very old. Also, the 2.6 kernel has a lot more encryption routines built into it, so using 2.6 changes how it's done. (but it still is basicly mounting an encrypted file using a loop-back mount point)

Re:Encrypted filesystems?

[iabervon]

The way Linux does encrypted storage is with encrypted devices rather than encrypting the filesystem on the device. This is good because it means you can encrypt any (device-backed) filesystem.

Re:Next premise, please

[sql*kitten]

And neither of whom have a journaled filesystem yet, while Linux has many to choose from.

What are you talking about? NTFS has had journalling for over a decade. And Unicode. And ACLs. And streams. And reparse points (these are amazingly cool). And compression. And encryption. And ... you get the point.

Now, MS doesn't use most of this good stuff, but it's all in there. Even three-letter file extensions on Windows are obsolete, since everything on NTFS can be an OLE server. There's nothing on Linux that comes close to the capabilities of NTFS. About the only major thing NTFS is missing is versionning, which VMS has.

Indestructable is the killer app

[Anonymous Coward]

I've gone well past the point where my data is worth more than the total cost of a new computer, and I don't want to lose it to a HD or computer failure. I'm particularly concerned that we are digitizing our family photos and that they could poof one day. So for me the killer application for a filesytem is 100% reliability. Ideally I'd just run some sort of transparent distributed raid thing and it would just automatically copy everything to one or more of my computers. That way when one crashed I could just plug in another one and be on my way.

The relational database filesystem seems like a big boondoggle to me. We already have several free RDB products (Postgress, MYSQL, etc.) as well as stable programming interfaces to those products in numerous languages. We also have good support for small files (Reiser3 already), and we support hard links. It looks like most of that could be tried out in userland too see how well it worked without any changes to the underlying filesystems.

I'm not a fan of metadata. The WWW has shown that having extra content about the contents of your content simply does not work very well. The metadata is redundant information and is thus prone to many syncronization problems with the original data that cause it to be invalid. There is a reason that we all search the web via GOOGLE instead of some metadata scheme.

Michael

Re:Indestructable is the killer app

[ron_ivi]

I've gone well past the point where my data is worth more than the total cost of a new computer, and I don't want to lose it to a HD or computer failure. I'm particularly concerned that we are digitizing our family photos and that they could poof one day.

Offsite backups are your friend. No matter what your filesystem's software, or the coolness of your raid array, or your battery-backed redo-logs; if a fire or a burglar takes your disks holding your filesystem you're hozed.

Personally, instead of a raid, I do a nightly "rsync" to a "yesterday" drive on a separate server (hense protecting myself from stupid-user failures as well as filesystem/disk failures); a "every time I did something significant" rsync to an encrypted filesystem removable drive kept in my car; and a "once in a blue moon" copy to DVDs in a safe.

An added benefit - upgrading an OS, or a computer is trivial, because the live backups are just that - live, and tested every night.

(Back to the filesystem topic, Reiser's whole naming idea is so much cooler than a heirarchy or a relational system I really hope this is the next big advance for Linux).

been there, had that

[prgrmr]

Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"

The seamless filesystem-in-a-database was created in the Multi-Valued DB structure in the mid-60's and release as the the Pick OS. It is still sold by Raining Data and runs on Windows, Unix, and Linux.

Re:why not improved ramdisk?

[Jeff Mahoney]

If you load everything on the filesystem to memory on boot, you end up wasting a lot of memory, since you typically use only a very small subset of your filesystem at any given time.

The solution would be to load things "on demand," as you've suggested.

Linux already does this, and it does more.

If you've ever looked at the output of free(1) after your system has been running for an hour or so, it will appear as if almost all your memory is in use. See those last two columns, "buffers" and "cached"? That's your "on-demand ramdisk" at work.

Linux will use memory that applications aren't using to cache filesystem data (including executables and metadata) to speed future accesses. If your applications need more memory than is currently free, the kernel will drop cached data rather than swap out application memory to disk. That way, you get the benefits of having your executables on a ramdisk, with the flexibility of not having to sacrifice running application performance in the process.

Re:MOD'd Troll????

[xsbellx]

Funny but I don't remember having to surrender my sense of humour to be able to moderate. Is this a new requirement?

What about granular permissions as in NTFS?

[LazloToth]

Hey, let's admit that Microsoft did a good thing with NTFS. Before I get roasted, let me say I've been working with FAT, NTFS, EXT2/3, Reiser, and others over the last 12 years, and I've had a chance to get a view of reliability, ease of recovery, etc., with several of these in production environments. I think the NTFS permissions model is one that the Linux world would welcome over the old and, I think, inadequate U-G-O scheme we continue to tolerate.

Linux encrypted filesystems not really up to snuff

[0x0d0a]

There are some disadvantages to this approach.

First, it's minimally supported by distros. I can't just set up a Fedora system out of box, and check "use encryption" and have it do an NTFS-style decryption of the file encryption key using the password entered at login for each user to decrypt that users' files. It requires hacking around pam and maybe initscripts.

Second, if that *was* done, it would take a different filesystem per user (per key), which is a pain to maintain.

Third, it can't be enabled by users (would require root dicking around with pam and filesystems) as NTFS encryption can be.

Fourth, it can't be enabled on-the-fly (requires creating new filesystems and copying the contents over, unlike NTFS).

Fifth, it's a pain to maintain -- on NTFS, it's easy for a user to just say "I want the contents of this directory and below to be encrypted" and choose to have things encrypted on a per-directory basis. The equivalent on Linux would be having the root user be creating new filesystems (knowing the appropriate sizes in advance and wasting any excess space allocated) copying over the contents and adding mount points for every filesystem mounted.

Sixth, NTFS supports key recovery with a backup, emergency passphrase (it can maintain two copies of the encryption key, one encrypted with, say, the administrator's password). Dunno about the Linux status of this.

Having an encryption layer above the block layer is a nice idea, but it's not a drop-in substitute for encryption support in the filesystem.

It would be possible to add a layer in which an encryption layer could be *added* (preprocess file/directory contents -- if one *only* wanted encrypted files and not directories, this could already be done with an LUFS or fuse module). Space for such a layer does not currently exist in Linux.

not so fast ...

[vlad_petric]

While I agree with the atomicity part, it's all great provided that the code is bug-free. IIRC reiserfs bugfixes where quite frequent in kernel changelog a couple years ago.

Filesystems are so crucial to OS stability, that I'd say it's worth formally-verifying them to a certain extent (i.e. prove that the algorithms/code work, instead of just observing that they work in normal conditions).

P.S. The whole thing - filesystem as a DB - is complete crap. You can't do a bunch of fs operations in a single transaction and have ACID semantics on the transaction as a whole. Sure - searching is great. But database means much more than just a searching interface.

Re:not so fast ...

[Anonymous Coward]

P.S. The whole thing - filesystem as a DB - is complete crap. You can't do a bunch of fs operations in a single transaction and have ACID semantics on the transaction as a whole. Sure - searching is great. But database means much more than just a searching interface.

The real killer is stored procedures. It'll be a cold day in hell before those are allowed into a kernel.

And how do you email files with attributes or other metadata? They're not part of the regular file data, so all the usual email clients won't see them. Ditto for file compression utilities. The backwards compatibility problems are insurmountable.

Re:not so fast ...

[cortana]

What's that? The ghost of Andrew Tenenbaum... mouthing the word "Microkernel, microkernel" over and over again!

Re:not so fast ...

[AstroDrabb]

ReiserFS 3 had bugs in the early versions just as all software will. That is why reiserFS was not used for productions systems for a while. It will probably be the same with ReiserFS 4. I will use it at home when it first comes out, but not where I don't want to chance data corruption.

Re:not so fast ...

[Sloppy]

Ghost?! He's not even dead yet!

I do think this is really funny, though. The more functionality people want to cram into the FS, the more they're going to look back at that famous Usenet thread, and reconsider... ;-)

Re:not so fast ...

[jbolden]

The backwards compatibility problems are insurmountable

They aren't a problem at all. Every email system can identify file formats it doesn't know how to deal with. Most can get external plugins. The file + attributes can be seen as just a type of file (like say .att). So if you support the .att format you would see a doc plus an icon plus history plus... otherwise you just see a .att file that needs some external app to understand.

Re:not so fast ...

[poelzi]

P.S. The whole thing - filesystem as a DB - is complete crap. You can't do a bunch of fs operations in a single transaction and have ACID semantics on the transaction as a whole. Sure - searching is great. But database means much more than just a searching interface.

Sorry, but you are wrong here. Reiser4 is atomic and you can pack as many operations into one transaction as you like, you just have to use the reiser4 system call. This is, because there is no standard system call for atomic filesystem transactions. Modern filesystems are databases, build to store files and query them trough filenames, reiser4 is the first filesystem where search path can be done through plugins, therefore you can index everything you want.

Re:not so fast ...

[bn557]

I think he something more along the lines of how it does a 'fork' of sorts in the file and the mail program sees multiple objects in 1 file.

P

Re:not so fast ...

[kcbrown]

Ghost?! He's not even dead yet!

MORTICIAN: Here -- he says he's not dead!

CUSTOMER: Yes, he is.

DEAD PERSON: I'm not!

MORTICIAN: He isn't.

CUSTOMER: Well, he will be soon, he's very ill.

DEAD PERSON: I'm getting better!

CUSTOMER: No, you're not -- you'll be stone dead in a moment.

MORTICIAN: Oh, I can't take him like that -- it's against regulations.

DEAD PERSON: I don't want to go in the cart!

CUSTOMER: Oh, don't be such a baby.

MORTICIAN: I can't take him...

DEAD PERSON: I feel fine!

CUSTOMER: Oh, do us a favor...

MORTICIAN: I can't.

CUSTOMER: Well, can you hang around a couple of minutes? He won't be long.

MORTICIAN: Naaah, I got to go on to Robinson's -- they've lost nine today.

CUSTOMER: Well, when is your next round?

MORTICIAN: Thursday.

DEAD PERSON: I think I'll go for a walk.

CUSTOMER: You're not fooling anyone y'know. Look, isn't there something you can do?

DEAD PERSON: I feel happy... I feel happy. [whop]

CUSTOMER: Ah, thanks very much.

MORTICIAN: Not at all. See you on Thursday.

Sorry, I just couldn't resist. :-)

(stolen mercilessly from here). Mad props to Monty Python).

Re:not so fast ...

[squidinkcalligraphy]

As I understand it, bad block checking is obsolete; I thought the hard drive firmware takes care of this nowadays and re-maps bad blocks when it detects them.

Re:ReiserFS corruption

[IamTheRealMike]

The explanation I heard was that there were a lot of bugs in the early 2.4 kernels that ReiserFS exposed and other filing systems didn't due to the way it worked internally. Whether it's finger pointing or not doesn't really matter - people are saying "I can't trust ReiserFS" but what makes you think you can trust the kernel team absolutely either?

Re:Encrypted filesystems?

[pclminion]

use a loop device. We have had encrypted filesystems for several years.

Everyone seems to like this method. But do you also encrypt your swap partition? If not, then whenever the system swaps, unencrypted data gets stored somewhere on the swap partition.

Here's something that might terrify you: run grep on your swap partition and give it a few characters from your password. You needn't list the entire password. Scary, eh? (This won't work for everyone, but it might for you.)

Remember, if you're using loopback with crypto, you're just pissing in the wind unless you encrypt your swap, too.

Why no MS DBFS?

[Doc Ruby]

Exactly why hasn't Microsoft released a SQL-queryable database filesystem? They validated the architecture with their marketing years ago, after IBM proved it technologically. And its advantages are obvious. In addition to better features, it offers Microsoft the opportunity to sell its SQLServer product to serious users, with a natural upgrade path. And its an opportunity to promote the MS version of SQL across the world, raising the tide against Oracle and the rest (including MySQL). It could also make an end-run around the Samba project, which Microsoft initially helped, but now apparently fears. And of course it's a better platform on which Microsoft can offer "open yet proprietary" file formats. Is Microsoft really so incapable of actual innovation, or is there something else wrong with this picture?

Re:Apple isn't "changing filesystems"...

[dasmegabyte]

Why do you copy before you backup?

Copying from three actively used locations, merging, and putting them on a slow external drive. Then I back up from there. I don't want to stop using my drives while I burn 30 DVDs.

so I guess renaming a bunch of files is faster than moving them to another partition

Noooooo, if you read my post I said that copying, moving and renaming, with a large amount of ID3 parsing, on HFS+ was faster than JUST copying on ReiserFS.

My dual Athlon never once locked up since I removed the audigy2 plat

A file system lock up is quite different from a REAL lock up. A file system lockup stops a particular Explorer window's contents to stop refreshing or responding for the period of the lockup. It happens regardless of hardware (happens all the time on our RAID at work, deleting a few hundred thousand bad mail files from Exchange).

An object-persistence system

[SteamyMobile]

If Linux is going to do this, the best way would be in the form of an object persistence system like JDO. What is a file? A file is something that we can access sequentially, by reading and writing, or perhaps random access, by going to a specific location in a file and then reading sequentially from there.

That's great, but is that what we (application developers) want or need? No, it is not. In fact it is extremely different from what we want.

What we want is the ability to make objects (a word processor file, etc) persist. We also want to be able to search for these persistent objects in various ways, by saying "show me all the word processing files that have so-and-so as the author, and were created on this date and contain the word 'nigritude ultramarine'". That's what we want.

A file system that allows sequential access to bytes, and creates a hierarchy of names (/your/file/is/here.txt) is really very different from that. That's why we need to have persistence tools (like JDO, XML, etc) and also search tools (Unix locate, file browsers, etc). But these are just hacks because the real problem of object persistence and retrieval isn't solved in one place.

One problem with solving it is that Linux is all C and has C mentality all through it, even at the application layer. People still scoff at object oriented design. This gets in the way of implementing cool filesystems like this.

Reiser4 does not exactly have these object oriented features, but it's much closer than anything else, and the object persistence could easily be implemented using Reiser4. I'm glad to see that Suse is using it as the default FS. I hope it becomes part of the standard Linux kernel. I also like its plug-in architecture, so we may finally get some advanced FS-layer security features in Linux.

Palm

[gveloper]

Doesn't Palm OS have a database/filesystem hybrid too?

Trust the Kernel team

[nuggz]

I've been using linux since the early 90's.
Always the stable version, didn't have problems until I tried reiserfs, switched back to ext2 ext3 actually, and I didn't have problems again.

The source of my problem appears to be resierfs directly or indirectly I don't know, like most users I don't really care either.

Doctor it hurts when I do this.
Then stop doing that.

Good enough for me.

Re:Trust the Kernel team

[DMadCat]

By that logic I should have gone back to Windows the first time my first Redhat installation died.

I didn't, I looked around until I found something more stable. It turned out to be Slackware using ReiserFS on an AMD900 with "cheap RAM" and I have had 0 problems in the year and a half It's been running (and it's my main desktop PC/workhorse).

Sometimes it is just the hardware.

The assumption is wrong

[njdj]

Linux must find a next-generation filesystem to keep pace with Microsoft and Apple,

This whole article is based on nonsense. Microsoft has a long way to go before it catches up with Linux in the filesystem area. There is no realistic prospect of Microsoft keeping pace with Linux filesystems in the foreseeable future.

(Before dismissing me a Linux fanboy, note that the above applies only to filesystems. When it comes to understanding of GUI issues, I'd make a similar statement but with Linux and Microsoft swapped. But that would be off-topic.)

Re:The assumption is wrong

[swissmonkey]

Well, if you look at NTFS, it has one issue : fragmentation when the disk is near full capacity.

On the other hand, it already does almost everything ReiserFS 4 _promises_ to do, and with NTFS it actually works, tried in the real world, and can be trusted.

Small files aggregation : NTFS stores small files in the MFT directly
Plugin : NTFS reparse points
Encryption : there since ages
etc..

Linux supports a lot of filesystems, but very few come close to NTFS when it comes to capabilities, scalability,...

I'm not the market they are looking for

[Psymunn]

Microsoft has a huge marketing department so they can please the people they are trying to sell to. Linux developers are their own market. By following Linux developers you are, more often then not, finding out what developers, like myself, want and need. By following Microsoft, you are finding what glitzy feature convinces someone that, maybe it's time to retire Office 97. The fact is, I have found few people, if any, who truly want a 3D desktop enviroment. I know others think diffrently, but i found XPs default taskbar seemed childish and condiscending.
frankly, i don't care if the casual consumer uses linux or not (though a larger market share would have some benefits). the people who develop for linux generally want and need the same things as my self and i'm happy.
That being said, faster file searching is definatly a useful tool. But if the registry in Windows is any indication of what the file system is going to look like in order for anything to get found, i don't want it.
On a random side note, App Rocket is a nifty program launcher for windows that finds files very fast.

Re:is this sarcasm

[Detritus]

No. You may not see them very often, but there are a lot of AS/400 systems out there.

Not gonna play with alpha code

[prisoner-of-enigma]

The SourceForge page says:

• Development Status:: 3 - Alpha

Sorry, I'm not about to trust archived video to alpha code, or even beta code. If there's no release-worthy option on Linux, we have to stick to NTFS on Windows.

Re:Not gonna play with alpha code

[Kristoffer Lunden]

Ah come on now, NTFS may possibly not be the best of the best (I don't know such things) but I've been using it extensively since I got my first 2000 install somewhere in early, well 2000, and as far as I can recall it has never failed or lost me any data, and it just works. Some oddities with file locks though when applications don't wanna let go.

Nowadays I am running purely Linux, and I wish I could say the same. Fsck ring a bell? And no, the newer breeds aren't flawless yet. But it is good enough, so I'm using it. ;-)

Just silly to pick on one of the things MS has done that actually works - it may not be perfect, but it is far from bad. Sadly, it also seems far from being writable in a stable manner too. ;-)

Now, if you would like to pick on FAT32, I'm game. =)

EMBED VERSIONING!

[chill]

Searching is all wonderful and that, but not the direction I believe would provide the most benefit.

Embed versioning into the filesystem. I believe Reiser has talked about this. Imagine being able to right-click on a file, folder or even partition and choose "roll back" or "restore" from the context menu. It then presents you with a list of snap-shot points you can restore to, starting with "last change".

Who backs up their hard drives any more? Have you thought of the problems and time involved in backing up 40, 80 or even 200 Gb of data? I'd MUCH MUCH rather have this feature than some enhanced search.

Re:EMBED VERSIONING!

[pe1chl]

One could conclude that either:
- this feature was not considered important by users and thus the systems offering it were not surviving
or:
- the feature was considered important or nice to have, but decision which system to buy was not made based on important or desirable features.

I think it could be the latter. However, that means that introducing useful features will not sell your system... what a wonderful world.

I'm working on this problem today

[jmulvey]

I'm an architect for a large corporation that is today trying to find a replacement for NFS. Our key goals are:
- Integration with a Kerberos SSO strategy
- Fast performance
- Cross-platform compatibility with Windows
- Robust Access Control mechanisms, RBAC would be nice but DACL is probably reality.

In my opinion, these are the primary goals that companies are looking for. Not a "journaling" file system, or built-in encryption. Sure those are nice, but let's get the basics first. Unfortunately, CIFS is still in quite a state of beta (even on the 2.6 Kernel) and there don't seem to be any real other alternatives.

Apple does NOT have a new FS coming out.

[JohnsonWax]

Apple is simply adding functionality to HFS+. Everything you've read about Spotlight describes a services that sits above the file system. It takes advantage of HFS+, but there is NO database driven FS coming out from Apple.

Their solution is to build a service that can interact with individual files, including their native metadata (ID3 tags, pdf metadata, MS Office metadata, email headers, etc.) through metadata importers and to store the metadata indexes in a separate database. This is relatively similar to how iTunes does it's thing. The services will have lots of APIs open to apps to incorporate the functionality locally.

The obvious clue that HFS+ isn't going away is that Apple is finally pushing full HFS+ support back up to the command line utils like cp to support resource forks and whatnot in 10.4, so hopefully we can stop needing OS X specific tools like ditto.

They've been adding improvements steadily over the years, such as journaling and most recently case sensitivity. The more obvious question to me is why doesn't the Linux community just jump all over HFS+ and build off of Apple's work since they seem more than willing to give the HFS+ support back anyway?

Re:Apple does NOT have a new FS coming out.

[virtual_mps]

They've been adding improvements steadily over the years, such as journaling and most recently case sensitivity. The more obvious question to me is why doesn't the Linux community just jump all over HFS+

Because the features they're adding to HFS+ are already available in other filesystems? There's nothing in HFS+ that would make linux users want to use it, and some compelling reasons why they would not. (Performance, size limits, lack of an online resizer, etc.)

Laughable - at best. Likely just worthy of a groan

[CAIMLAS]

Does anyone else find the fact that someone is comparing WinFS to the likes of reiserfs (despite my dislike for it), XFS, and hell, even ext3?

Granted, the proposed featureset of WinFS is vastly 'superior' to that of the 3 main linux contenders, but it could be argued that WinFS is neither a filesystem itself, nor is it on par with any of the linux filesystems in terms of performance or stability (if NTFS5 is to be of any forboding).

I seem to recall reading about several projects that impliment WinFS-like features. I don't recall what they were, and I don't think they were kernel-space projects, but I recall thinking, "this looks nice".

Besides, let's be honest here. What practical functionality does WinFS provide that is above and beyond the combination of 'locate', and 'file' used in conjunction? WinFS seems to me to be merely a crude hack so as to make up for the fundamental shortcomings with MS's OS design.

OS400 is still different

[mabhatter654]

OS400 native FS is still the neatest model for DB-FS integration out there. It's unique because the File system is written to allow file-member-record-field access directly from a command line call... It's similar to what was posted above with Reiser4 and having plugins. The key to AS400 success is that the "file system driver" is pushed down into hardware controller roms so the DB like access is nearly fool-proof. Stuff like queries and SQL are just "plugins" on top of that model. The only thing I see preventing linux from adopting such a scheme is that it would require an entire dedicated HDD/partition to do properly because you need complete control of the disk structure to the FS driver...and where AS400 has very few native file types it deals in, linux would need to litteraly create a file model "plugin" for every type of Mime type..otherwise you still have a bunch of meaningless BLOBS to parse. OSS/Linux is uniquely qualified to write this because they have the "keys" to everything, but it would take enormous cooperation to implement it correctly!

The leader of the pack...

[joh]

There is a tendency to make the developers of Free Software responsible for hunting after every and any feature and idea MS and Apple care to implement to fish for users that is really annoying.

First, all Unix file systems since some decades have proved to just fit the bill quite fine. Searchable Metadata and other "features" is actually application-level stuff. 98% of the data on an average Desktop Unix system (and 99.5% of the data on a server) does not need that, because it rarely changes and is of no special interest for the user at all. And if it does, application-level data is better integrated, faster, and more flexible.

What is happening here (and in many other recent discussions) is dragging the Free Software community into an arms race it can not win. You can't make Linux/Gnome (or FreeBSD/KDE or whatever your favorite Open Source system may be) into a system that is just like Longhorn or Tiger but gratis and free. Never. What Free Software really needs, now more than ever, is to be picky about its users, its uses and its features. Better offer 10% of all users a system that is better than offer 90% of them a system that is a poor emulation of the OS they get for free with their PC anyway.

This is a point that the Free Software community has to (re)learn, better today than tomorrow.

Not really

[Synn]

IMHO the OSS community is underperforming in picking up new technology and good ideas.

Hardly. There are a lot of OSS projects that are leading the way with new technologies and in implementing good ideas.

But in quite a few areas it's not at all uncommon to see slow support for new tech. The community divides about how to implement the new ideas, which slows things down, but that division fosters competition and provides a base for testing out different ways of getting the new tech out the door.

Sometimes doing it well is better than doing it first.

EXT3 FS

[jonnystiph]

At work here, the previous admin installed a number of machines with EXT3 FS on the drives. These machines (RH 8.0 - EL3) crash sporadically often giving indications the FS was at fault.

While I personally believe Redhat is known to push "unstable" releases, I was suprized that from 8.0 - EL3 the EXT3 fs was still crashing and Redhat was still offering this as default on an install.

Anyone else had better expierences with EXT3? I am curious if anyone has more information on why this FS seems so damn unstable.

For test purposes we run "out of the box" installs, so there should be no kernel tweaking or any other "anamolous" things going on with the install's or the boxes.

My wish: Case insensitivity

[Lars Clausen]

I really don't see the point in a case-sensitive file system. Remembering case but ignoring it in comparisions makes life a lot easier. Can anyone point at some application or library or kernel part or anything that depends on the case of the filename alone to tell files apart? I can't think of any -- and besides, I'd consider it very bad behaviour.

-Lars

Some features I'd like to see in a new filesystem

[cdn-programmer]

The first feature I would like to see added would provide the capabilities of a Partition Data Set from the IBM TSO days. We would not copy what IBM did of course because they did some stupid things meant to sell more hardware. One thing was deleted files were not really deleted and you needed to "re-generate" the PDS when it filled up. Nice trick!

Suppose you had an infinit number of loop back devices and these were hidden and used internally by the file system and when you started an application you could "mount" what for many intents and purposes looks like a TARBALL and the application in question and ONLY the application in question got to see all the files in this TARBALL. Well, the files inside a "TARBALL" of this nature would probably not be compressed, but, they could be if desired... Well, that is the concept of a Partition Data Set.

In the case of a user logging in, when the shell is started a mount could take place against the user's private data set. By doing this on a shared machine, file security can be guaranteed. For export and import the system could mount a "shared" dataset.

This sort of secutiry is far superior to ACL's and anything present file systems offer for the very simple reason that normal people including systems administrators would not normally see any of the files inside one of these datasets. Consider the advantages of running an apache server where you KNOW all associated files needed by that release of apache are in a single dataset. There IS not easy way to lose a file or clobber it or accidently delete it and so forth. Next consider that when that copy of apache starts up it _could_ simply mount a set of files each of which contains the whole website for a given domain.

Upgrading to a new copy of apache would be as simple as copying in a new dataset and mounting it against the web datasets. If a glitch is found, simply restart the old copy.

Backing up a set of files becomes a simple copy operation. Replication can be accomodated as well.

Systems Administration in those old IBM mainframes was MUCH easier than with UNIX systems and this is in large part because of the way the system handled partition datasets.

------------

Now, with this we would want to be able to mark certain files as being external sort of like opening up a window, and through this window we could for instance access certain files which might be the executables and supporting scripts.

Of course people will point out we can accomplish some of this with a loop back mount. The problem with the loopback mount is that it populates the directory tree and this is what I really want to avoid. Frankly there really *IS* no reason for even a sysadmin to be able to see 90% of the files that say consitute a web server, or say PostFix, or PostgreSQL. We accomplish a lot if the executable which needs access to its supporting files has a "private" loopback and only this executable by default gets to see the mounted dataset.

--------------

Next idea is versioning the way Digital Equipment Corporation did it in the VAX. We simply append a version number and what the delete does is append a version number. With disk drive capacities heading into the stratoshere there is no reason to be conservative.

And this leads to the next idea which has been mentioned before... that is replication - across machines.

I can buy for $20 bux a machine (P1 200mHz) that can run a 20 GB hard drive and in fact I think they can run 80 GB hard drives as well. Rsync is useful, but a full replicating filesystem at the kernel level, or at least at the level of a deamon close to the kernel would mean that a machine can be backed up to another machine in perhaps another building automatically and with little effort.

Well, I'm sure other people have other things they might like to add. This is my wish list.

Re:New FS (Reiser4 has a compression plugin coming

[hansreiser]

Reiser4 has a compression plugin coming. We got gzip to work, but it consumes too much cpu, so now we are doing lzo which can compress at disk drive speed. The lzo plugin has a bug, maybe next week....

Hans

(You can email edward@namesys.com for details).

Didn't you know? Linux is a penis

[rd_syringe]

It's not about actually making good applications for computing, it's about a losing penis competition with Microsoft and Apple.

Microsoft is the big penis. It got big through buying lots of natural herbs, getting its dick sucked by lesser folk, and having an expensive personal trainer make them do cock pushups everyday.

Apple is the small penis, but it works hard to make itself look bigger by trimming the pubes around the base of its shaft. Commonly, the pubes are trimmed into artistic mutton chops, and you'll see cockrings and piercings.

Linux is the extremely tiny penis, desperately trying to extend itself with a very painful penis pump. It's also desperately downing lots of penis enlargment pills and furiously masturbating itself in any attempt whatsoever to compete with the other guys.

The other two penises accept their size and work with it. The Linux penis tries to be the other penises.

Be your own penis, Linux!

Re:Laughable - at best. Likely just worthy of a gr

[swissmonkey]

NTFS5 is miles ahead of most Linux filesystems (reparse points, encryption, compression,... all supported transparently), only ReiserFS 4 comes close, and it's not yet in a usable state.

I'm wondering if you even know what WinFS is, comparing it to file and locate it laughable at best.

Try finding all mp3 by Brian Adams or Withney Houston on a 200Gb disk filled with 250'000 files with file and locate, you'll get the answer 10 minutes later.

With WinFS, it will take you a whooping 2 seconds maximum.

That is without talking about the user-defined attributes, etc... that make WinFS more powerful than anything of that kind on Linux.

Ah, Cairo.

[flacco]

Anybody remember Cairo?

Do i remember Cairo? How could I forget it. It was the first time I'd ever laid eyes on you, and my thoughts have been consumed by that moment ever since.

It was hot - the sun in the sky, and the Egyptian counterintelligence on the ground. Very hot indeed. Dust filled the bazaar, crowded with people and the scent of spices. The cameras around my neck (for show, of course) seemed to grow heavier throughout the day, and the perspiration threatened to show through my white linen suit.

I stopped by a stall and pretended to peruse the knives for sale, but under the horn-rimmed sunglasses, my eyes scanned the bazaar. I was worried - you were supposed to be here an hour ago.

As I turned back into the crowd, I saw you. I will never forget that moment. Your face was mostly covered by the traditional dress of the region, but your eyes were glittering blue jewels that mesmerized me.

I stood unable to move for several seconds. Coming to my senses, I gave the pre-arranged signal: taking the hat from my head, I wiped my forehead and the back of my neck with a handkerchief, exposing the black widow spider design stitched into its corner.

Immediately, you relaxed, smiled with your eyes, turned, and walked into a dark, dusty doorway, and I followed after you.

And so I have followed you ever since. I tried to remain professional, but I was merely denying to myself what I already knew on that first day: I was doomed. It was a classic case of giving way to my heart, and in this business, that is the first step to giving away your life.

In the previous cell - the torture cell - there was nothing but blackness, and agony. Here, there is at least a crack in the ceiling. Light sometimes filters through the perpetual dust, and it falls across me during certain times of the day. In the dustbeams, I still see you. Even after everything, I see you every day, not knowing what thoughts hold court behind that beautiful, mysterious face.

The dysentery has left me quite weak, so I don't often move. There is no water here, and no food. But mostly it is the lack of water. That's what will get me in the end. I've been put here to die. In these final days, I don't think of the apple orchard in Indiana, or of my mother, or of the ice storm that trapped me in the mountains so many years ago. I think only of you, and the bazaar, and that moment I first met you. I think of... CAIRO.

Re:New FS (Reiser4 has a compression plugin coming

[mysticalreaper]

uhhhh... what?

if the filesystem does the compression, the apps (or you) can't see it happen. that's the POINT. your suggestion, above, is ridiculous. If you had a tar.gz file, you could extract it to the FS, but it would actually be equally compressed (cause it's a gzip compressed FS), and then you could play with the files to your heart's content, without worrying about the compression, cause it's transparent. You wouldn't need or want some kinda plugin or something...

Unless the FS wasn't compressed, and you wanted a transparent way to access tar.gz files. That idea would make sense.

Re:Laughable - at best. Likely just worthy of a gr

[nathanh]

Try finding all mp3 by Brian Adams or Withney Houston on a 200Gb disk filled with 250'000 files with file and locate, you'll get the answer 10 minutes later.

I assume you're finding those particular MP3s so you can delete them. Badoom tish.

Modern drives

[warrax_666]

automatically map bad blocks to good blocks, but you can monitor the current number of bad blocks through SMART. If your drive is causing errors you had better stop using it (production environment or no) as quickly as possible. Bad blocks are almost always a sign of impending failure.

Jebus, do you want to run a production system on a drive with known bad blocks? Whoever hired you must be a complete moron.