Slashdot Mirror
How To Implement A Database Oriented File System
ALundi writes "A really great read from Andrew Orlowski over at The Register on how Benoit Schillings and Dominic Giampaolo created the 64-bit journaled and attribute based Be File System. Schillings and Giampaolo discuss a variety of design and implementation issues, including data integrity and file system performance. " Interesting in the context of MSFTs plans to
implement a DB filesystem
in future versions of MS Windows.
If I remember correctly, Be originally used a "true" database backend as the filesystem, but ran into performance issues compared to the R5 fs implementation. I can't help but wonder how many of these issues were largely due to the speed of the technology used at the time.
I suppose it depends on your application, but I know a lot of web-based platforms already use true db backends (Oracle, PostgreSQL) to handle all data storage, representing a filesystem as a hierarchial set of rows in numerous tables. I've written several applications this way, and am currently working on a content management platform which also uses this model. Need to make a change to the filesystem structure (adding attributes, changing the security model)? Just modify the DB structure and you're done, especially with databases like PostgreSQL where you can use the database engine itself for a *lot* of functions (via triggers, stored procedures, security settings, etc).
As more and more functionality is brought into web-based application environments, I can see the importance of "old style" filesystems starting to fade somewhat for a lot of apps. Yes, they'll still be necessary (the database itself has to reside somewhere, obvisouly), but not in the same way they used to be. Just a few thoughts
Dominic's book, Practical File System Design with the Be File System, is wonderful. I'd never delved into the innards of a file system before. Reading his book was enjoyable and interesting. I learned quite a lot.
Transcript show: self sigs atRandom.
MS is have trouble getting major clients to switch to Active Directory. I think there is little chance a DB base FS will be accepted in any reasonable time frame.
No, it's called the Classic MacOS. The Mac has used a database to store files and their attendant metadata since HFS was introduced.
It uses B-Trees which if you RTFA is thought not to be a very good solution, but it was revolutionary for its day.
The idea of having a resource fork to all files, a little name badge for every file that tells file type and creator type, is tremendously helpful too. However, the resource fork on Classic MacOS files tends to corrupt at the drop of a hat, rendering this little innovation, also introduced with HFS, fairly useless.
Oddly enough, however, I find good ol' MacOS fairly robust...so much so that there is no need to reformat and reinstall on a regular basis. Its memory management sucks, but aside from that glaring problem it's a beautiful operating system. I have PCs running Windows and PCs running Linux here at Catseye Labs, but it's my good ol' G3 that I do most of my work on.
Knowledge is power. Knowledge shared is power multiplied.
Bruno G. Albuquerque and Axel Dorfler are working to make a 100% compatiable clone of the file system that shipped with the later versions of BeOS as part of the OpenBeOS project.
:
They have a fantastic amount of the work done already such as
o Read-only BFS
o Kernel Interface
o Full Attribute Support
o Indexing
o Symlink Traversal
o Queries, full UTF-8 Support, and support for non-indexed attributes.
I believe they are also fixing some problems that were in the original FS.
I am sure they would be glad for some more file system engineers. Come to think of it, the rest of this open source project is going really well too, but as always it needs more programmers....
A great book on the BeOS file system, its design and the issues involved, is Practical File System Design with the Be File System by Dominic Giampaolo. This was the first book I read about file systems and is a great discussion on the types of decisions the BeOS folks made to address the specific users and OS functionality they were looking for. It's not too heavy and is really a casual read in comparison to most other Computer Science books.
A transactional database is very good at surviving system crashes because it is completely journalled. Accessing the data out of the database would probably be no faster, but searching on it would be considerably faster.
well lets see......Ciro (code name for Win 2k) was begun in 1995, they wanted to have a DB file system in that, that was thier plan and that is when it began.
BeOS began development in 1991. the first iteration of a DB backed file system came out with BeOS 1.0 and BFS replaced in around 1994ish.\
soooooooo.......MS again is a day late and a dollor short.
same with MATURE Office productivity products, since workperfect was mature long befor word 1.0 was released, as was lotus spread sheet.
perhaps they did good work on the OLE crap, but that stuff sucks (imho) and Adobe has much better products to crate desktop publishing work that OLE was intended to help with....not to mention that word still can not do what TEX can do.
I am the Alpha and the Omega-3
D3 and other Pick-like systems have been using filesystems based around a database for years.
Check out this link for more information on the D3 database filesystem. ;)
fast enough is fast enough
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
It's really not Microsoft's innovation.
IBM's AS/400 (a midrange computer system targeted for commercial use/accounting/warehouse/etc...) is based on an object-oriented database filesystem which is implemented at the firmware level (SLIC) rather than at the OS-level - and this system has been around for about 20 years and IIRC it always had quite good performance.
-arch----
A few words about its architecture, if you're interested...
The operating system (OS/400) itself runs on top of this object-oriented low-level "OS" by calling its APIs - as a result, most parts of OS/400 are platform-independent. If you'd manage to get the SLIC running on another hardware platform, you could probably install a nearly unmodified version of OS/400, and it would do its work.
Actually, I'd call the SLIC code the 'real' operating system kernel rather than OS/400, because OS/400 itself would not work without an apropriate SLIC layer.
Everything on the system is an object, so you'll always have to use the object's methods to perform some operation.
For some applications that may be an advantage, because security is enforced on each object at the firmware level. For other applications it might also be a disadvantage, because you'll always have to use a limited set of APIs for modifying data. That blocks many methods commonly used for writing highly optimized code.
-end arch----
One of the benefits of having a database-filesystem is probably the fact that you do not need to run a database product on top of the OS.
Every object on the system can be backed up and restored in a very simple way. Logical files (multiple logical views of one physical file) can help to keep data management simple and consistent.
On the other hand, you will have to update the entire OS (including the kernel) when you need to install a new release of the database - which means, that you'll have to reboot the machine.
And - last but not least - the more code you have in the OS kernel, the higher is the probability of having dangerous bugs somewhere in the kernel.
It should not be necessary to mention, that bugs in the OS kernel may compromise all system security.
There are certainly many advantages and disadvantages regarding the database-filesystem issue, so I think it all depends on what you want to do with your computer.
-----
kind regards from Austria,
octogen
PS: i hope my english isn't too poor..
And - by the way - even Microsoft uses AS/400 boxes for running its business, so what do you think, where did they get their inspiration from...?
No, you don't.
Be used the following method of determining which app would open a file:
Notice that nowhere in here does it take into account what app created a file.
Even better, you can always right click on any file (or any list of files of the same type) and get a list of:
- The preferred app for that file
- The preferred app for that filetype
- Any apps that can open that filetype
- Any apps that can open the supertype
- Any apps that can open any filetype
And the list shows up pretty much instantaneously. Do that in any other system, anywhere, no matter how many hacks you add. And yes, you'd use it every single day of your life if it were available to you.So no, I'd say that you don't understand the problem.
Oracle has a database based filesystem Internet File System/IFS that is pretty sweet.
Supports protocols HTTP, WebDAV, SMB, FTP, IMAP, and SMTP gives the ability to store, manage, and search documents, presentations, multimedia, Web pages, and XML files.
Also do Check-in/check-out, version control, and event notification features. Unlike most of the examples i have seen developers can extend these base features to build custom content management applications.
members are seeing something, your seeing an ad