Database File System

ozy writes "With all the fuss about searching and Spotlight and WinFS, check out the Database File System a completely different interface for your files, implemented in KDE. There is actually a request for developers to join a project to implement this under GNOME and leave how we use the desktop today behind."

Comments

"Implementing in GNOME"

[kosmosik]

Such thing should be implemented at kernel level to be transparent for *any* aplication. Without this it will just lead to a mess (like 4 different implementations) and some apps working with it and most not. As f.e. you can browse SMB network with Nautilus but when you actually try to open a file (from SMB via Nautilus) in OpenOffice.org you will get a info that viewer does not support this method... It must be a standard system routine not another level between system and GUI.

stubborn

[mn3m05yn3]

Article doesn't address whether or not we can turn DBFS off and use the more traditional hierarchical method of file placement. Will we be dragged into this kicking and screaming?

Disadvantages

[BHearsum]

How much permforance overhead will this cause? The 'Desktop Environments' already eat a lot of RAM and CPU.
How much disk space will you lose over this? All the metadata has to be stored somewhere, and just glancing over the link I read something about a versioning system, which will definently take up quite a bit of space. Will a 20gb hard drive become 15gb with DBFS?

Re:Disadvantages

[aodl]

While performance is something that should always be kept in mind, we are a long way away from the days of the original Macintosh where a desk accessory had to weigh in at 600 bytes in order to make the cut and fit into both memory and on a floppy disk. As current desktop machines outperform the high end servers of a few years ago, it would be nice to put a lot of that muscle to use in improving the user experience. I'm not excusing bloated and slow code here, but we don't really need to be counting bytes.

In any case, database based operating systems have been around for decades, from OS/400 to the BeOS. Many BeOS users claimed it was hands down faster than any other shipping OS at the time, and it featured a journaling, database-styled file system. One of the primary developers of that file system is now working at Apple on Mac OS X 10.4's spotlight functionality.

The thing is - as our desktop storage continues to grow at the pace that it does, and as we curiously find ways to fill it up, new ways of looking at and finding the information we store are going to be needed.

DBFS, Gnome Storage, Apple's Spotlight, and WinFS, all take different routes to get there. It's worth looking at all and what they offer and where they differ. WinFS, is a new storage layer that combines file system resources with more structured data in a Relational/XML hybrid system, with the aim (from what I gather) of turning the file system into a global "soup" of data. That sort of soup can be seen in office suites or PDA style applications, and in older Operating Systems like the Newton OS, where everything is a shared and available resource that is stored and available through common structures. Spotlight, on the other hand, combines file system searches and indexes (think 'locate') with full content indexes and a metadata index, which uses 'importers' to parse out other file formats. Spotlight is not a new file system, but an indexing system that acts on files in the file system. From what I remember of Gnome Storage, it is similar, using the VFS layer and Postgres triggers and callbacks, along with plug-ins, to parse and extract relevant metadata and contents out of files. DBFS looks to be like WinFS in that it purely wants to be a new kind of information store. I don't know which style will win out. My theory is that technologies like Spotlight will eventually evolve into a new kind of storage system, while remaining familiar and file based for todays users and developers. But this is an idea whose time has more than come. It's something that's been promised for the desktop for at least a decade, and has been shown to work, albeit in targeted OS's (the Newton) or ones that never achieved mass market penetration (BeOS).

So I think that performance concerns aren't that big of a concern, so long as (like all development) there are good people working on the solution.

Re:Performance?

[Anonymous Coward]

Depends on what you are using your computer for of course.

You can say the same thing for a GUI, and its correct for certain applications of computers, but wrong in others.

Re:Performance?

[psavo]

Isn't this thing with DB's getting a little excessive? You're adding another layer and step to storing data which will in all likely hinder performance. I'm not sure the benefit out weight the cost.

Well, if it's only a name-translation thingy, then it shouldn't affect performance of file reading (when operating on sufficiently big files), only file opening/stat:ing.

This is not a file system

[MobyDisk]

Maybe we could call it a "filing" system since it indexes files that are on another file system. Really, a file system IS a database, not an add-on that indexes files. Still, perhaps this is a better approach than trying to redo all the file-system internals. Although to be truly useful, this needs to be an API that is GUI-independent, with GUI-bindings as needed.

Re:Performance?

[BenjyD]

Why not just run in console mode? All this GUI stuff is just getting in the way of absolute performance.

If it adds 0.5 seconds to every time you save a file, but saves you 20 seconds of filesystem navigating every time you open the file, that's a worthwhile tradeoff. Add to that the fact that copmuters don't get tired or bored, while humans do, and it makes even more sense to shift as much of the burden of working onto the computer as is practical.

Re:Performance?

[jgardn]

Not necessarily. Consider the performance of finding a document you wrote two years ago. How long does it take you to walk through the directory hierarchy browsing file names? How fast is the file search tool? Wouldn't it be faster if you could say "Show me the documents I wrote two years ago" and the refine the search or browse the results?

Storing data in a relational database is natural because it is more like the way we store data in our minds than the hierchical structures of traditional file systems.

Also, we allow a complete abstraction of the underlying database in relational systems. The database can store the data however it sees fit, and can arrange the data on disk without the users noticing a change.

I look forward to experimenting with a relational filesystem. I think it would be a wonderful thing to try out and see if it actually has the advantages I outlined above. I'd also like to see the actual disadvantages.

Any innovation is good

[Stevyn]

People can offer their opinions for or against this, but I think that any innovation benefits linux. I've read about WinFS and it sounds like a good idea, but who knows when it will be ready. If people working in their spare time can get something like this working in linux before Microsoft can get it out, I think that would just be another reason to trust the open source model of developing code and squash Ballmer's FUD.

I don't have too much trouble using a hierarchy file system. I keep my stuff pretty organized, but computers are supposed to save time, not create more problems. If this database can do a good job, I'll give it a shot.

Re:Backups, and being organized in a general way?

[Ignominious Cow Herd]

Think of a directory structure as just one instance of a relational structure - a heirarchical or location-based one. Directories (or locations, or relationship to other files), even nested ones, are just another type of metadata. Once you have that concept in mind other things become obvious. For example you may group files by type (Image, Text) or by project, or by author. With a directory structure you either have to make multiple directories and symlink everything, or you're stuck with one view of your files.

In short it gives you multiple, simultaneous groupings of your data.

Jeez, when will these people learn?

[Grendel Drago]

Joel on Software said it best:

For example, WinFS, advertised as a way to make searching work by making the file system be a relational database, ignores the fact that the real way to make searching work is by making searching work. Don't make me type metadata for all my files that I can search using a query language. Just do me a favor and search the damned hard drive, quickly, for the string I typed, using full-text indexes and other technologies that were boring in 1973.