Slashdot Mirror
'Storage' to Replace Traditional Filesystems?
JigSaw writes "OSNews is reporting on Storage, an innovative project which aims to replace the traditional hierarchical filesystems with a new document store which is database-based (PostgreSQL). The current implementation, built under Gnome 2.x for now, offers natural language access, network transparency, and a number of other features. The project is currently in alpha (screenshots already available), and it is part of the next major generation of Gnome. It is currently developed by Seth Nickell, the person responsible for the enhanced Gnome usability on 2.x and its HIG, among other things."
it's better for programs to abstract data like that, the fs should only to provide access to the medium, nothing else.
I think Longhorn will be the first Windows with a database filesystem. It will probably be based on SQL Server
It's really a sad that there was a perfectly good implementation of database file system, but the company wasn't able to topple a monopoly and got squashed. MS really should have just bought BeOS and ported everything over to it. They could have just called it LongHorn and released it this year instead of waiting until 2006.
Hopefully they plan on extending this to the networked environment, allowing multiple domain/realm file permissions, authentication, and encryption.
Anything to replace NIS and its bastard stepchildren.
SELECT * FROM MY_FILES
Of course it's only wishful thinking. I'd be nervous to see exactly how this integrates into other "Legacy" applications. I can also see be performance penalties since you are now querying a database, rather than looking at a simple file structure...
Yeah, and as Longdong gets pushed back and delayed and delayed and pushed back and postponed and delayed, it'll be last to market but microsoft will still have been the first to announce it. I guess that's more innovative than they've been in the past when they'd simply wait for someone to do something interesting before buying them out.
It's not enough to say. One has to do. Microsoft has proved many times over that it often makes grand announcements only to provide something far more watered down by the time they get to market.
We'll see what they're DB-based file system really is when (and if) it gets here.
Outside of a work environment, I've rarely encounter anyone who keeps consistent, useful filenames, let alone metadata indexes; it seems to me that people will skimp on the metadata, and thus limit the usefulness to metadata that the computer can collect automatically ("All movies that last under 90 minutes"). It's like CD collections, or books; libraries have nicely catalogued and ordered collections. Private individuals don't; they have roughly ordered collections on the shelf, and don't bother keeping them in any better order. I suspect the same will happen with these metadata systems; people won't do the work needed to make them truely useful.
I appear to have a blog. Odd.
No, because doing away with the root filesystem, user stuff in /home, config files in /etc, and so forth would break a number of Unix standards Linux's big advantage of being able to run many Unix apps (if you compile from source) would disappear.
Storage will apparently be an interface to the existing real filesystem. Joe User won't know the difference.
GPG Key ID: 8C444E97 Fingerprint: E7BA D851 9714 8D97 C4F9 1777 8168 6913 8C44 4E97
I don't know how a database system can improve a file system's performance, especially with the unnecessary overhead associated with, the current state of the ext3 file system is doing quite well, and updatedb/locate works fine for me.
What can really interest me is something like updatedb/locate but with SQL syntax support, this could be awesome.
The IT section color scheme sucks.
> I know, I'm the first to look for screenshots, but antialiased filesystems are a bit too much, maybe.
Reminds me of an internal joke we have here. Our ClearCase file server was an SGI.
Why?
Because the filenames were rendered so much prettier than on a Linux or Sun box...
What then happens to SQL as a MS product? If its built in to every OS, why then would anyone buy it.
Remember how Windows XP Home and Pro editions can serve files only to less than a dozen simultaneous clients? This is to boost sales of the IIS bundled with Windows 2000 Server and now Windows Server 2003. Microsoft SQL Server Home Edition will probably be limited.
Will I retire or break 10K?
The filesystem on AS400 is actually a db2 database and it work quite well
To my understanding, the delay in Longhorn's release is a result of the TrustWorthy computing initive...
This, IMHO, is a good thing. The big difference between MS and Open Source on something like this... in Open Source land, you can often see progress from day one... no matter how unstable it is. With MS, you wont see anything until the whole product is done... Not saying one is better then the other, but...
It seems silly to tie the implementation to a single database, when gnome-db is fast approaching 1.0.
"What this world needs is a really big injection of orginal thought"
They are original ideas, they just don't make it into the PC world where MS dominates. MS come up with as many original ideas as McDonalds
and since all KDE & Gnome (and frankly most open source projects) are doing is playing catchup with MS then originality is never going to be
a prime concern.
There are also issues with gaining acceptance for the change in the way things work. This kind of thing has not really been done on a large scale in the wild before, on any OS, so whether people will be willing to accept the security and reliablity issues that may ensue is another matter. For example, what are the implications of a compromise in the database engine? MS is planning on using SQL, so if things go awry and it becomes possible to maliciously inject raw SQL to the filesystem interface... Oops. On the otherhand, the benefits for data retrival are *huge*. Imagine being able to find any audio files on your entire system by Justin Timberlake or Britney Spears and delete them all in one go by searching on the tag fields! ;)
(1) Technically, all filesystems are databases, it's just that current ones are a collection of flatfile database tables that can point to each other, generally in a heirarchial manner. When people say "database" in the same sentence as "filesystem" they usually mean "relational database". As an aside however, high end databases usually forgo the need for a file system and provide the ability to write their tables directly to disk on a dedicated partition.
UNIX? They're not even circumcised! Savages!
"Well, where do you go?"
"Stanford."
"No problemo, I'm heading that way later and I can grab it for you. What's your room?"
"Dorm 5, Room 109. It's the desk on the left."
( We didn't bother to state earth.us because we were already inside those directories)
Yes, yes we do think heirarchically. Most of the history of human thought has been fitting everything we can lay our filthy little brain cells on into heirarcheis, whether they wish to fit into them or not. It's intuitive.
As for natural language didn't we learn about that with COBOL? Natural language only speeds the learning process slightly ( the majority of the learning still lying in the realm of understanding the basic concepts involved), but then becomes a pain in the ass forever afterward.
Looking at the screenshots it's also ugly as all sin. The physicist in me can't help but feel that a model that ugly can't possibly be correct.
I think this makes just about as much sense as using a document preperation language (XML) as the basis of a database.
Which is to say, none.
KFG
Not true.
This isn't a replacement filesystem, just a document-storage system - you won't be able to access your documents easily from a shell.
It is a good starting point though, once working, the next step would be to compile it into the kernel, so that you can create Storage partitions, etc. and be able to do something like:
cd music by U2Now that would be cool
Am I the only one that isn't totally into the idea of "googling" data on my hard drive?
Granted, it's mostly pr0n on there, so it's almost the same thing, but still...
"It is seldom that liberty of any kind is lost all at once." -David Hume
SELECT * FROM videos WHERE name LIKE '%porn%'
Having SQL Server as the underlying filesystem technology doesn't mean that you're going to be running SQL Server directly. I mean, if you currently use NTFS, there isn't a NTFS daemon that the kernel connects to when it does filesystem transactions. Just like every other filesystem, the support will be built into the kernel. Instead of writing data as NTFS does, the structure will look a lot more like how SQL Server stores data -- with built in indexes, etc.
Many database servers already have some fairly optimized code when it comes to file access. This just implements it at the kernel level, rather than having it sit on top of a traditional fs.
listen here bud, we're just trying to whore some honest karma, don't go at it with your "facts" and shit.
thanks for understanding.
Actually, Be had two flavors of "filesystem as database" in widespread deployment. OK, not as widespread as Windows, but certainly thousands of users. The first version of Be's filesystem, by Benoit Schillings, was very database like, but performance was so-so. The second version of BeFS, by Dominic Giampaolo, was less general in implementation, but had the same metadata-driven capabilities. There's an interesting article on this at http://www.theregus.com/content/4/24485.html. Basically, Be did everything that this project is talking about, years ago. That's not to take anything away from the project -- it's cool if more mainstream operating systems catch up to the innovations of niche players, because more people benefit. Dominic is working at Apple, so there's hope that MacOS X's filesystem will start incorporating the rich-metadata, dynamic view model of the world. And while MS has (I think) pushed the "filesystem as database" out of the next version of Windows NT/XP/whatever, it's still planned for the next version after that, so perhaps in a deade or so we'll all be able to do what Be did back in '91. And of course, Palm owns the Be code, so perhaps PalmOS will lead the way?
Enable 3D printed prosthetics!
Everyone is everyon is looking at this from the Geek perspective. I teach computer technology to absolute beginners and the file system is the MOST confusing aspect. Most people have trouble with it and some people will never get it. If a database driven file system had a very simple interface (i.e. text searching that had fuzzy logic so that misspellings were okay) it would be GREAT for 90% of the population.
It was called IFS and Oracle did it like, almost four years ago.
Versioning and various other metadata existed. It could be exported via SMB, NFS, FTP, and as a regular "local" windows filesystem.
And, why is this such a great big deal? I don't see the same stink raised as the possibility of Longhorn having a DB for a filesystem.
The copper bosses killed you, Joe. 'I never died', said he.
1995 Signs to Cairo
1996 Unearthing Cairo
The so call Longhorn WinFS directory is just another rencarnation of the Cairo object orientated file system.September 1, 2003 Eweek 'Longhorn' Rollout Slips
Microsoft have been attempting this type of functionality since 1991, over a decade. Meanwhile, one open source GNOME developer, with help from the other core GNOME developers, provides most of the features within months.People don't seem to see how great this is. Maybe it's because most people don't have all that much data.
On my home systems, I have over 250GB online. That doesn't even count my music or videos/movies, which I keep on seperate, removable, optical storage.
I can tell you from experience, that managing that much data is a huge hassle. Let's say you've got your files organized well. You probably have hundreds of folders for each subject, and you have to broswe to each one with each new file you save. I have a folder (several actually, for various subjects) where I save thing that I've haven't taken a look at yet. Let's say it's a program that I haven't installed. Well once I do install it, I need to clean up all the temporary files, then browse around to another folder (takes a minute or two when you have hundreds of folders), where I save installed programs, and browse to the appropriate sub-folder, and save it. But then I end up doing the same thing with a video clip... Watching it, deciding to delete or save it, then browsing to a sub-sub-sub-sub folder to move it.
Of course, that's enough of a hassle, but things get complicated when I want to move things to another systems, which obviously isn't going to have the same filesystem. Merging each individual folder, into each different folder is seriously time-consuming, and teedious. Without fail, there always ends up being a couple folders in the wrong place, because they were a sub-folder of something else, that I did happen to see when I coppied the contents of the folder.
Then matters are even further complicated, because I may choose to delete older content months later or so, and locating everything is a huge mess.
Personally, I would like to save everything in one place, not having to change folder to folder for each file. When saving something, I could just enter a handful of keywords (eg. "picture penguin snow") which would be much less work than moving to directories or even typing in a long filename. From there, a simple database system would be be able to know what type of file it is, how large it is, and how old it is. That would make it incredibly easy to manage. Whenever I want a file, I type-in "images older than 1 years" or "programs marked as archived" and I get EVERYTHING I'm looking for in a fraction of the time. Not only that, but it makes pruning out old data as easy as it could possibly be. Just search for "linux" and delete older version, no worries about what folder it's in... If it's in a temporary folder and you haven't used it yet, or if it's archived and been in-use on your system forever. Obviously you'll be able to see that information, but unlike in our current systems, it won't stand in your way when you want to find things.
It's absoultely no work at all to transfer files, since the info should stay with them, and it will automatically integrate perfectly with your local file management/organization scheme. What's more, data like marking something as "archived" is great in that your system could automatically move it over the network where you archive your files. Since your filesystem would be a smart database, when you search for the file, it could still turn up in the search results, and be automatically moved back where you need it, when you need it.
Personally, I think this would not only save time and effort, but money as well, because so many people wouldn't be dealing with their file problems by just throwing more space in their systems, instead of spending time on figuring out where every file is, what they can get rid of, dupilcate files, and junk like that.
With this, I should be able to say "tar -xjf 'newest version of mplayer'" However, this will need to be in the actual filesystem to be useful, not just supported for GNOME applications.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
> SQL is slow compared to things like BerkeleyDB
:) But modern RDBMSes have integrity control facilities as well.
BerkeleyDB is a hierarchial database. SQL is godzillion times faster on complex searches.
> Your database becomes corrupt, you lose everything.
Your filesystem becomes corrupt, you lose everything.
And yeah, I know about journaling, so don't bother
Lisp is the Tengwar of programming languages.
What we need is to get some information storage/retreival experts to provide some guidance to the developers of these ideas.
Librarians have been working on these problems for centuries, why not start with what they know?
http://theregister.com/content/4/30670.html
"The oft-misunderstood Windows Future Storage (WinFS), which will include technology from the "Yukon" release of SQL Server, is not a file system," reports Thurrot. "Instead, WinFS is a service that runs on top of - and requires - NTFS."
Although this is an interesting idea, an all-relationsl filesystem would prove to be a usability nightmare.
The relational zealots are quick to point out that a relational system can model any sort of data. Indeed, it can do this. This does not, however, mean that it's always good at doing this. Sometimes it's the right tool for the job, and sometimes it's not. In this case, it is very much not a good tool for sole access to files on the system (though it can make an excellent tool for complementary methods of access).
The reason that hierarchical filesystems have survived for so long is due to one thing: navigability. It's relatively easy for any user to browse what's on the system and get a good idea of how it is organized.
You can't navigate a relational system, which will prove to be the downfall of any all-relational system which comes into being. You can, of course, do a SELECT * FROM volume if you really want to, but that does exactly that: it gives you all the data, with no particular organization. Examining the entire "sea of data" suddenly becomes cumbersome in the extreme. So while User A might be able to set up an all-relational filesystem completely according to his own tastes, User B will be totally lost on that same system. This is, to say the least, a nightmare for anyone working in a shared environment.
This is not to say that the relational model isn't necessarily a useful thing for filesystems. On the contrary, it can be very useful for certain kinds of searches. As time goes on, I believe we'll see more relational-style searching technology incorporated into file managers and search tools. However, there also needs to be a means of hierarchical navigation. Humans tend to think of things in terms of locus, and a means of providing that kind of reference point have to be maintained.
Luckily, this can actually still be emulated using relational-style tables, even though it's somewhat less efficient than classical storage techniques. Some filesystems already do something similar to this, and the results are promising. Look at Be's filesystem for an example of that.
The best way to go, moving forward, is something not unlike what BeOS did, with both hierarchical and relational methods of examining data. This allowed for the best of both worlds. The default method of getting at data is still the hierarchical paradigm, but relational searches can be applied to create what some have called "smart folders" (perhaps "boxes" might be a better term?) Systems like this "Storage" should be focusing on complementing traditional systems in this way, rather than replacing them.
Actually, I was just thinking about this problem, and you know what would make a *really* easy solution and is readily available already? P2P! Think about it; a new file arrives on the system by whatever means, so the file system has zero idea about it's nature beyond what's available from the file. We probably know the type of file from its header, extension or whatever other "file" command type trick was required. We also know its size, any tag type information that may be present, the filename, and we could maybe calculate a checksum too. So we fire off a P2P query with what we have and what we want to know, then wait for responses.
Sure, you will probably get responses that conflict, so some kind of progessive weighting and elimination system is required. If you search on Kazaa and look at the meta info returned, it's fairly easy to see what is correct and what is not; automating this analysis is the next step. There is also the probabilty of CDDB type services springing up to act as the "Supernodes" of such a system, or as dedicated standalone services.
Of course, you probably wouldn't want the OS doing this for you automatically. Imagine the fun and games that would ensue if you started getting Bill G. sending out P2P queries to fill in the meta tag blanks on a document about "increasing revenue through tweaking our licensing strategy again"! ;)
UNIX? They're not even circumcised! Savages!
Microsoft didn't really put in much investment in Cairo after it was pretty apparent that nobody really cared for it at the time. Most people really don't like novel ways of doing things. There is too much investment in the old ways atm. I guess if the world were different, we would all be using Microsoft Bob right now.
So, I think this GNOME thing will also sizzle out after a while.
Is it just me that sees this as a really bad idea? Nothing against the gnome project, you understand, but I see no earthly reason why a filesystem should require X Windows, let alone a full-blown desktop environment. Surely this kind of thing should be a kernel-level project which userspace tools can hook into as needed, whether from gnome or KDE or the CLI?
Anyway, I thought Reiser4 was doing exactly what this promises, but with the advantage of a proven track record on high-performance filesystems. Perhaps, if gnome wants this kind of functionality, they should base it on Reiser4 which will at least be widely-used and not locked into the gnome project.
"'I pass the test,' she said. 'I will diminish, and go into the West, and remain Galadriel.'"
- JRR Tolkien.
An OT side-note:
Just as each of us has our own organizational scheme for our own bookshelves, libraries tend to vary more than we think too.
Just about every school and community library you'll find uses Dewey Decimal, of course, but others have other schemes.
For instance: the Library of Congress, in order to conserve space on their shelves, orders their books by size. (No, I'm not kidding. Look it up.) The directory is computerized, of course, so aside from the inconvenience of having same-topic volumes wildly separated in space, it's not a big deal for them.
Allegedly real newspaper headline from 1998:
Man Struck by Lightning Faces Battery Charge
My guess is that they'll use MSDE, which is already freely available and "royalty free". I think it's basically just the core of SQL Server without any of the extra tools.
Alarm bells ringing at Redmond: now they are even copying our vaporware! Push back the release dates, hire some programmers, we have to actually implement this stuff!
No, this isn't what is needed. Hierarchical object-oriented persistent object trees is what is needed.
Let me explain.
Information in real life is organized in trees. It is obvious anywhere one can look. From the organizational chart of a company to the chair that you are sitting on, everything is a tree; a tree of information, where each little piece of information consists of other pieces of information.
If you check computer applications, almost all application contains some sort of tree. Take a Word document, for example: the master document, the contents, the heading 1 and subheading paragraphs, the pictures, the drawings. Everything is a tree, and the document can be browsed as a tree.
Take your favorite mail client. Information is organized in a tree: inbox, outbox, sent, trash; each of these contain an e-mail. An e-mail itself is composed of subject, body, attachments. The body consists of paragraphs; the attachment consists of files.
Take your favorite drawing program: the picture consists of layers; each layer consists of shapes; each shape may consist of other shapes.
Take your favorite 3d/cad program: a 3d object consists of other 3d objects.
Take the gui: a window consists of other windows.
Relational databases don't provide tree organization. I don't want a freaking flat table to store my documents. I want to organize them in trees. That's why a filesystem has subdirectories.
The biggest problem of the current filesystem technology, is that a 'file' is as dumb as it gets: it is just a collection of bytes, waiting to be manipulated by some other program. It is even untyped, for God's shake!!! one program may view it as a series of bytes, another program may view it as text, another program may view it as code!!! The file itself can't tell you anything about its properties, about its contents, about the way it is supposed to function....
If a file could tell the outside world how to be operated, then the world would be a much better place. If a file could tell me its properties, if it provided me with the tools to manipulate it, then I would make any type of app that processes the file as needed.
The above is essentially object orientation on the filesystem level. RDBMS don't offer such kind of functionality!!! at best, an RDMBS offers an index on a key for quick searching, and that's it!!! There is no notion of tree, nor each file exposes its properties/methods/functionality to its users!!!
So I say a big 'NO' to relational filesystems.We can immediately move to the upper level:
1) each node of information is AN OBJECT.
2) the object specification is defined at the filesystem level. Much like COM or .NET or CORBA.
3) each object can contain other objects, if it inherits and implements a specific interface.
4) each object is PERSISTENT. The filesystem takes care of persistence, according to attributes of the object's fields. Complex objects that are composed of other objects are also managed in the same way.
5) the parent object provides the storage implementation. The storage implementation would be object-oriented!!! An object could implement an RDBMS-like storage capability with indexes, keys, etc.
At each given time, the information model inside the computer could be:
1) splitted in multiple computers.
2) shared by multiple users.
3) checked for security in ONE place, inside the operating system.
4) provided as a framework to programming languages.
5) replicated across sites with minimum of code
6) a unified GUI could handle it
7) searching through it will become a breeze!!! (for example show me all MP3 with artist = Elvis and title = rock)
After all, 90% of the programming goes to load/store and display information. It is silly not to provide a unified mechanism for that. And a simple SQL-based RDBMS does not cut it.
www.namesys.com/whitepaper.html describes why the relational model is not the right one for large heterogeneous stores (filesystems), and describes the approach ReiserFS (a Linux filesystem used mostly in Europe) is taking instead.
Hans
I can see it now... 'penis enlargement guaranteed' popping up at random places in the database.
You'll have to type "I do not want a bigger penis" to remove them all.
Heh.
If I'm to have a natural language interface to find my files, I'd really like to make spoken requests instead of typing a long sentence. Do they have plans for that in GNome?
Of course, databases are very useful for organizing user data. People already keep PIM info, images, and lots of other stuff in databases. Lotus Notes is built entirely on databases.
But "replacing the traditional file system" carries with it the notion of ripping ext3 out of the kernel and putting a relational database there. That's a very bad idea. Databases don't belong into the kernel. They are far too inefficient to handle most storage needs, they are far too complex to go into the kernel, and they just don't need to be in the kernel. Operating system kernels need simple, fast storage systems. Something like ext3. ReiserFS is pushing the limits. PostgreSQL would be going too far.
As an aside, this is an idea that just about every nerd has when they learn about databases and retrieval. It's been tried various times since the 1960's. There are probably good reasons why interfaces don't use them. Perhaps most importantly, keep in mind that the vast majority of files on your system are not user files, they are bits and pieces of the operating system. And for the files that actually are used by users (mail, PIM info, images, text, etc.), they usually already have special-purpose database interfaces available to them as part of the applications that users use to access them.
It's innovative because it's an idea implemented on Linux, whereas when it's to be implemented on Windows it's, a lousy idea (well, lousy because of 3rd party compatibility nightmares).
What time is it/will be over there? Check with my iPhone app!
Where does this metadata come from? I assume I have to enter it myself. This means the more files I have, the more detailed and specific my data entry becomes. And that much more tedious.
Even worse is the uncertainty that would arise. Is my search for "horn solos" not returning results because there are no such files, or because the filesystem does not have meta data describing the files I want as such?
At this point, hierarchial organization once again becomes much more appealing again.
1. I know precisely where one more more files are located and I want to see their contents as fast as possible. I want to move around 100s of such files easily from directory to directory at the potentially optimal speed of the disk subsystem. I might also want to do batch edits or renames of these files. Speed is the only truly important characteristic.
2. I vaguely have an idea of what type of file I am looking for. I want to find one or more files satisfying a particular metadata (or full text) criteria and manipulate one or more such files. I want all versions of my files to be maintained and there to be full auditing of interactions with these files.
Many times 1. and 2. are mutually incompatible. The typical way these days to address having both is to have an automatic spider that maintains an indexed view of the files and the file's metadata and hope that the spider is not too far behind the actual changes being made to the contents of the files or the locations of the files. If you want to have a transactionally guaranteed implementation of 2., then you have pretty much eliminated batch manipulation of files as a reasonably performing option. Database tracked file systems do not do well when you unzip a large collection of files and then start batch copying those files to different locations.
Now, I know almost nothing about the current implementation of the new "database" file system being discussed. But, I would very much want to allow a user to designate which directories or file types would be put into relational tables and which ones would not. I might also want to be able to choose whether the relational tables were interacted with using a transactional guarantee or whether a "spider" was used. If the end user had control over when the "heavier" management of the files occurred and how much of it should be applied, then it might have utility. Part of the user's file system would then be a document management library and other parts would be a normal file system. However, I would find creating a user interface that tried to make such a solution comprehensible to an end user somewhat of a nightmare.
Programming can be fun again. Film at 11.
This was one of those revolutionary products that never really took off for some reason. Text -based word processor , but this was pre-Windows (perhaps that was the problem).
From what I remember, you could create a document, and then save it based with keywords. It was really aimed at writers and was a great way to organize a bunch of documents, create outlines etc. Sort of like having an electronic file card system. Very very cool - the Windows Explorer is almost primitive by comparison. Could've easily extended it to support any type of file.
It's a great idea nice to see GNOME pick it up.
main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}
I suppose it is probably too late to inject comments and have them moderated to the point of visibility as the madness has largely subsided... but here's to futile acts ;-) I was not really intending Storage to make a big splash right now, I wanted to keep it low-key, but I guess the damage is done so I might as well comment. I'm sorry that I didn't have time to put up a more technically-oriented exposition of Storage. *shrug*
Some technical notes... that site is sparse on technical information so I'll fill in some for the curious.
If anyone would like a nice read on filesystem implementation and/or Dominic's approach to the fs redesign, check out:
Practical File System Design with the Be File System
by Dominic Giampaolo, Be Inc, Dimonic Giampaolo
1. Nice overview of various filesystem's in use.
2. Quick and to the point.
3. Enough detail to go about rolling one up yourself.
4. Being written by Dominic it provides nice BeFS insights.
Not really a comment on "storage" but just a comment on something that has constantly bugged me when someone says "let's put it in a database!"
A filesystem is a special case of a database. So it is perfectly acceptable to store your data into a filesystem. Some people seem to think everything has to be put into a relational database or that is it somehow cool to do so. I have seen people store loads of graphics as BLOBS in databases. Someone once suggested storing a ton of MP3's in a database. Most recently someone said (and this isn't the first time) that we should store all of the emails in a database. It's just another unnecessary layer of complication, especially when you are going to be referencing the email/graphic/mp3 by name all the time anyway (fs's like reiserfs index on name so it's blazingly fast) and not by a bunch of other pieces of meta-data. And if you are going to need to do lookups by various bits of meta-data then store the meta-data in a db and also store a record pointing to the actual file on disk. I have done that lots of times and it works great.