Database File System
ozy writes "With all the fuss about searching and Spotlight and WinFS, check out the Database File System a completely different interface for your files, implemented in KDE. There is actually a request for developers to join a project to implement this under GNOME and leave how we use the desktop today behind."
...seems to have something more interesting: storage
...I can kind of see this would make it easy to search and locate documents. What about backups though? How would a user be able to group (manually) related files together, so that the whole bunch can be backed up later, without having to search for all seeminly related (or unrelated) keywords to trace all hitherto-unrelated documents?
Secondly, with this mass of files being spread over several disks, surely, this is in a way forcing the user to "search" for everything. Or isnt it? Will the underlying FS layer still be accessible in the general way that it is?
http://efil.blogspot.com/
Such thing should be implemented at kernel level to be transparent for *any* aplication. Without this it will just lead to a mess (like 4 different implementations) and some apps working with it and most not. As f.e. you can browse SMB network with Nautilus but when you actually try to open a file (from SMB via Nautilus) in OpenOffice.org you will get a info that viewer does not support this method... It must be a standard system routine not another level between system and GUI.
Article doesn't address whether or not we can turn DBFS off and use the more traditional hierarchical method of file placement. Will we be dragged into this kicking and screaming?
I have always thought that version control (file histories, branching and atomic changes) would be nice to have at the file system level. Instead of storing myessay-firstdraft.doc, myessay-seconddraft.doc, myessay-final.doc, the file system should do the work. Then if I want to make a bunch of changes (perhaps I want to try a new page layout), I should be able to commit them as one atomic change (or throw them all away if I change my mind). Then, when I want to make a set of documents with US spelling, I should be able to branch the whole lot (using no disk space) and make the small changes from UK spelling while still being able to integrate other changes I have made.
How much permforance overhead will this cause? The 'Desktop Environments' already eat a lot of RAM and CPU.
How much disk space will you lose over this? All the metadata has to be stored somewhere, and just glancing over the link I read something about a versioning system, which will definently take up quite a bit of space. Will a 20gb hard drive become 15gb with DBFS?
Perhaps I am more organized than most but I already categorize my files and such in the hierachal file structure.
Isn't Rieserfs planning to do this on the kernel level?
Where does that leave other fs choices and storage and other idea dbfs?
I see more and more people saying look what neat things you can do with these tools.
But really why do you personally want something like this?
Curious to see the response is all.
ACK
The author is asking for help to move the project to Gnome.
Quote:There is of-course the hard choice of platform. I choose KDE? because I am familiar with QT a bit, and because it is inherently object-oriented, being C++ and all. But in my mind GNOME? is much closer to how I would like a desktop system to function. So I would like to go for the GNOME option. I leave KDE developers to do what they want with this, and I am offering them support. But I would like to focus my efforts on GNOME and implementing the above in GNOME.
Any volunteers?
In the first place we will need developers. Would you like to join, send me an email (o.gorter@student.utwente.nl) with DBFS and JOIN somewhere in the subject. If you are not a developer, but still would like to help, please revisit this page in a few weeks. There will probably be a community website by then somewhere.
Depends on what you are using your computer for of course.
You can say the same thing for a GUI, and its correct for certain applications of computers, but wrong in others.
Isn't this thing with DB's getting a little excessive? You're adding another layer and step to storing data which will in all likely hinder performance. I'm not sure the benefit out weight the cost.
Well, if it's only a name-translation thingy, then it shouldn't affect performance of file reading (when operating on sufficiently big files), only file opening/stat:ing.
fucktard is a tenderhearted description
(In the course of the heated discussion about Reiser4.)
Maybe we could call it a "filing" system since it indexes files that are on another file system. Really, a file system IS a database, not an add-on that indexes files. Still, perhaps this is a better approach than trying to redo all the file-system internals. Although to be truly useful, this needs to be an API that is GUI-independent, with GUI-bindings as needed.
Normally I file things in a hierarchial method by year and month and by project name (2004file/9sep/) or (2004file/workfile/projectname), but still I lose track now and then and need keywords. Change the "slash" slant to fit your OS.
Nobody is sugesting to use such database FS for entire system. Only for specific data (f.e. user documents) - not entire system (binaries, libraries etc.) where such performance matters. Well in fact it will improve performance since right now applications that need such indexing (best examples are apps for organizing music (like iTunes) or digital pictures colections (like Adobe Photo Album or Google Picasa)) do it themselves which probably is not the fastest way and is not unified across the system. Now for *some* applications such view on files that lets you query for specific files/objects operating on query results rather as directories of files have much benefit. But it is only for organizing data, and in limited scope (as I've said - digital music, photography, probably some other fields). I don't really belive that this would seed up searching for office documents over LAN or smth. - when somebodys documents are in mess DB-FS won't change anything as the documents probably lack metadata, proper naming anyway.
Adding a database layer to Gnome sounds like using another 300 megabytes of storage on my hard drive. I simply do not need the database.
If the FSF/GNU folks really want to do something revolutionary, they should fork Linux+Gnome into 2 distinct paths: minimalist and maximalist. The maximalist is what we have now. The minimalist is a minimally featured Linus+Gnome distribution. It is the bare minimum in functionality that we need to have a decent operating system and desktop.
Into this minimalist installation, I will then add the applications (e.g. MatLab) that I use daily.
Keeping the traditional file system is only logical as this database sort of file access is in fact a higher level of abstraction.
.....
Considering there are numerious project of such higher level of file access abstraction going on, it does become a secondary choice for the user if they want to use one of these higher abstraction level file access systems.
To remove the traditional file system altogether would be a mistake, as then it could become a system of babel or keywords.... "what was I thinking when I created that keyword and lets not even get into what crazy joe was thining when he came up with his keywords...
But hey, given how MS based developers would create some obscure dll name and place it in some obscure location in order to copy protect
higher level abstractions are useful only to the point that you can, if need be, drop down in abstraction level to get your bearings as to where you are. If you cannot touch physical reality then how do you know you are not floating around aimlessly?
being out of touch with physical reality can evenm be very dangerious and hard to correct.
Why not just run in console mode? All this GUI stuff is just getting in the way of absolute performance.
If it adds 0.5 seconds to every time you save a file, but saves you 20 seconds of filesystem navigating every time you open the file, that's a worthwhile tradeoff. Add to that the fact that copmuters don't get tired or bored, while humans do, and it makes even more sense to shift as much of the burden of working onto the computer as is practical.
standard filesystems only have ONE index - a hierarchical one that contains a certain amount of real-time-updated indexing (such as the timestamps on a directory)
but it is NOT a relational database: you CANNOT easily create or use an alternative index to your files.
that's what all the fuss is about.
some people mentioned here that they already organise their files. great. fantastic.
HOW LONG DID IT TAKE YOU?
and how long would it take to reorganise?
with a relational database, all your indexes are updated AUTOMATICALLY.
therefore, doing searches on a relational database filesystem (find me all music files with dates between last week and last month: SELECT * from files WHERE files.type = "music" and files.date NOW() - 7days
you _can't_ do that sort of thing on a traditional filesystem.
sure, you can emulate it by creating symbolic links all over the place, but what happens when a file is deleted or moved? you need to manage / relocate the symbolic links...
nah.
databases.
fantastic idea.
now can we have them as a kernel module, pleeease?
Not necessarily. Consider the performance of finding a document you wrote two years ago. How long does it take you to walk through the directory hierarchy browsing file names? How fast is the file search tool? Wouldn't it be faster if you could say "Show me the documents I wrote two years ago" and the refine the search or browse the results?
Storing data in a relational database is natural because it is more like the way we store data in our minds than the hierchical structures of traditional file systems.
Also, we allow a complete abstraction of the underlying database in relational systems. The database can store the data however it sees fit, and can arrange the data on disk without the users noticing a change.
I look forward to experimenting with a relational filesystem. I think it would be a wonderful thing to try out and see if it actually has the advantages I outlined above. I'd also like to see the actual disadvantages.
The radical sect of Islam would either see you dead or "reverted" to Islam.
At a deeper technical level, nany of the questions asked here have historical answers or clues in The Design and Implementation of the Inversions File System. The abstract reads:
Note that this paper was published in early 1993. Many of the issues it addresses are relevant to DBFS, and many of DBFS's advantages are foreseen by that paper. IMHO DBFS has chosen a direction that should have better performance than inversion, not to mention lower risk and easier failure recovery.Inversion was built on POSTGRES, which makes one wonder what happened to the source.
For those of you that have not yet looked at the Mac OS X (Tiger) preview and WWDC web cast, the new spotlight technology built into the next version of Mac OS X looks very much like a fully integrated database file system. And it's incredibly fast. Go check it out!. Note: QuickTime required. Mplayer may work for us Linux heads but I haven't tried it.
-- DuckWing
People can offer their opinions for or against this, but I think that any innovation benefits linux. I've read about WinFS and it sounds like a good idea, but who knows when it will be ready. If people working in their spare time can get something like this working in linux before Microsoft can get it out, I think that would just be another reason to trust the open source model of developing code and squash Ballmer's FUD.
I don't have too much trouble using a hierarchy file system. I keep my stuff pretty organized, but computers are supposed to save time, not create more problems. If this database can do a good job, I'll give it a shot.
Whether or not you look at system files every day probably depends on what you are doing with your machine and what you consider "system files" to be. Moreover, this idea would seem to go entirely against the whole UNIX "everything is a file" philosophy which is supposed to be one of the great strengths of UNIX.
flossie
Write now. Defend liberty
I'm using Subversion for a project and the idea of Atomic Commits seems like an obvious direction for file systems. If that other slashdot story is correct, storage becomes less of an issue and it would be possible to roll back the system to any point in time or to only roll back one file if need be. Now throw an intuitive way to navigate files on top of that and you've got a sure winner.
In the grand scheme of things, only a very small handful of us on earth are aware of Linux or even know what an Operating System is for that matter. File systems seem to be the big stumbling block for new users. Anything that can make computers and therefore access to information easier for the coming waves of new computer users (maybe billions of people?) will be a good thing. Even if the "bloat" slows down the system by 10%.
I hate to preach but that old quote comes to mind "With great power comes great responsibility". I don't think most of the people working on the OS that will soon dominate in developing nations (that's Linux) are aware of the harm they can do by slowing down Linux development with petty personal disputes. Like it or not, Linux is no longer just an edgy hacker tool. It has the potential to change the lives of Billions of people.
What if Digg added local news and a Slashdot inspired comment karma system? ---
http://houndwire.com
Search for: Documents
Modified since: last backup
I don't know about the other implementations but Searchlight and WinFS are implemented atop the existing filesystem. (The FS in WinFS supposedly stands for "Future Storage" and not "File System") Sort of like how Google is implemented "on top" of the regular hypertext-linked internet.
Take a look at an AS/400 (iSeries). They've been around for more than 10 years. And before that you had the System 38 and System 36... and so on..... Why is this some big revelation now?
-Cnik
Joel on Software said it best:
For example, WinFS, advertised as a way to make searching work by making the file system be a relational database, ignores the fact that the real way to make searching work is by making searching work. Don't make me type metadata for all my files that I can search using a query language. Just do me a favor and search the damned hard drive, quickly, for the string I typed, using full-text indexes and other technologies that were boring in 1973.
Laws do not persuade just because they threaten. --Seneca