Slashdot Mirror
'Storage' to Replace Traditional Filesystems?
JigSaw writes "OSNews is reporting on Storage, an innovative project which aims to replace the traditional hierarchical filesystems with a new document store which is database-based (PostgreSQL). The current implementation, built under Gnome 2.x for now, offers natural language access, network transparency, and a number of other features. The project is currently in alpha (screenshots already available), and it is part of the next major generation of Gnome. It is currently developed by Seth Nickell, the person responsible for the enhanced Gnome usability on 2.x and its HIG, among other things."
It's really a sad that there was a perfectly good implementation of database file system, but the company wasn't able to topple a monopoly and got squashed. MS really should have just bought BeOS and ported everything over to it. They could have just called it LongHorn and released it this year instead of waiting until 2006.
Hopefully they plan on extending this to the networked environment, allowing multiple domain/realm file permissions, authentication, and encryption.
Anything to replace NIS and its bastard stepchildren.
Of course it's only wishful thinking. I'd be nervous to see exactly how this integrates into other "Legacy" applications. I can also see be performance penalties since you are now querying a database, rather than looking at a simple file structure...
Who owns the patent on this type of filesystem implementation - there must be one? Microsoft, IBM, Seth...SCO?
The idea was probably stolen from Xerox Parc in the first place, of course.
UNIX? They're not even circumcised! Savages!
I disagree, strongly. Files are an artifact of a bunch of bad implementation decisions when stripping Multics down to produce UNIX. What programmers want to be able to do is manipulate data structures and store them persistently. What files force you to do is waste tons of time writing code to take your data structures and write them out as sequences of bytes and read them back in.
One OS that solved this nicely was NewtonOS. If you wanted to manipulate persistently stored data you opened a "soup" that contained objects. So if you wanted to, say, set up an appointment with someone for lunch, you could find the person in the address book "soup" and then create an entry in the databook "soup" recording the appointment, which would immediately appear in all other apps that dealt with appointments (because app's accessed the same data structures, and were notified of changes so that they could update). So your data was not trapped in a particular application's proprietary format, and users weren't forced to learn the artificial concept of a "file" but instead could think about "my appointments" or "my address book".
If you haven't tried it, don't knock it. As a developer, and as a user, it was wonderful -- much more straightforward than "files" and "directories".
Enable 3D printed prosthetics!
1995 Signs to Cairo
1996 Unearthing Cairo
The so call Longhorn WinFS directory is just another rencarnation of the Cairo object orientated file system.September 1, 2003 Eweek 'Longhorn' Rollout Slips
Microsoft have been attempting this type of functionality since 1991, over a decade. Meanwhile, one open source GNOME developer, with help from the other core GNOME developers, provides most of the features within months.People don't seem to see how great this is. Maybe it's because most people don't have all that much data.
On my home systems, I have over 250GB online. That doesn't even count my music or videos/movies, which I keep on seperate, removable, optical storage.
I can tell you from experience, that managing that much data is a huge hassle. Let's say you've got your files organized well. You probably have hundreds of folders for each subject, and you have to broswe to each one with each new file you save. I have a folder (several actually, for various subjects) where I save thing that I've haven't taken a look at yet. Let's say it's a program that I haven't installed. Well once I do install it, I need to clean up all the temporary files, then browse around to another folder (takes a minute or two when you have hundreds of folders), where I save installed programs, and browse to the appropriate sub-folder, and save it. But then I end up doing the same thing with a video clip... Watching it, deciding to delete or save it, then browsing to a sub-sub-sub-sub folder to move it.
Of course, that's enough of a hassle, but things get complicated when I want to move things to another systems, which obviously isn't going to have the same filesystem. Merging each individual folder, into each different folder is seriously time-consuming, and teedious. Without fail, there always ends up being a couple folders in the wrong place, because they were a sub-folder of something else, that I did happen to see when I coppied the contents of the folder.
Then matters are even further complicated, because I may choose to delete older content months later or so, and locating everything is a huge mess.
Personally, I would like to save everything in one place, not having to change folder to folder for each file. When saving something, I could just enter a handful of keywords (eg. "picture penguin snow") which would be much less work than moving to directories or even typing in a long filename. From there, a simple database system would be be able to know what type of file it is, how large it is, and how old it is. That would make it incredibly easy to manage. Whenever I want a file, I type-in "images older than 1 years" or "programs marked as archived" and I get EVERYTHING I'm looking for in a fraction of the time. Not only that, but it makes pruning out old data as easy as it could possibly be. Just search for "linux" and delete older version, no worries about what folder it's in... If it's in a temporary folder and you haven't used it yet, or if it's archived and been in-use on your system forever. Obviously you'll be able to see that information, but unlike in our current systems, it won't stand in your way when you want to find things.
It's absoultely no work at all to transfer files, since the info should stay with them, and it will automatically integrate perfectly with your local file management/organization scheme. What's more, data like marking something as "archived" is great in that your system could automatically move it over the network where you archive your files. Since your filesystem would be a smart database, when you search for the file, it could still turn up in the search results, and be automatically moved back where you need it, when you need it.
Personally, I think this would not only save time and effort, but money as well, because so many people wouldn't be dealing with their file problems by just throwing more space in their systems, instead of spending time on figuring out where every file is, what they can get rid of, dupilcate files, and junk like that.
With this, I should be able to say "tar -xjf 'newest version of mplayer'" However, this will need to be in the actual filesystem to be useful, not just supported for GNOME applications.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
I think the examples he shows are pretty good. In my mp3 collection, I would like to see "All bluegrass songs" or "all remixes of Parliament Funkadelic stuff". How do you propose to do this in a hierarchical filesystem? Most of my bluegrass artists are under 'bluegrass', but then there are some bluegrass songs that were in non-bluegrass artists and albums folders.
In my workplace we are having the same problems. On our shared folders, we have shipping documents in each clients' folder. But then, what if we what to see all shipping documents from a particular vendor? Currently, we would have to go into each customers' folder (which are also broken down by year archives) and grab all documents which *might* be from said supplier, and then open each one, and look to see, because the supplier name isn't in the filename. It's horribly broken, which is why we are moving to a database storage system for such documents.
Computers are useless. They can only give you answers.
-- Pablo Picasso
Although this is an interesting idea, an all-relationsl filesystem would prove to be a usability nightmare.
The relational zealots are quick to point out that a relational system can model any sort of data. Indeed, it can do this. This does not, however, mean that it's always good at doing this. Sometimes it's the right tool for the job, and sometimes it's not. In this case, it is very much not a good tool for sole access to files on the system (though it can make an excellent tool for complementary methods of access).
The reason that hierarchical filesystems have survived for so long is due to one thing: navigability. It's relatively easy for any user to browse what's on the system and get a good idea of how it is organized.
You can't navigate a relational system, which will prove to be the downfall of any all-relational system which comes into being. You can, of course, do a SELECT * FROM volume if you really want to, but that does exactly that: it gives you all the data, with no particular organization. Examining the entire "sea of data" suddenly becomes cumbersome in the extreme. So while User A might be able to set up an all-relational filesystem completely according to his own tastes, User B will be totally lost on that same system. This is, to say the least, a nightmare for anyone working in a shared environment.
This is not to say that the relational model isn't necessarily a useful thing for filesystems. On the contrary, it can be very useful for certain kinds of searches. As time goes on, I believe we'll see more relational-style searching technology incorporated into file managers and search tools. However, there also needs to be a means of hierarchical navigation. Humans tend to think of things in terms of locus, and a means of providing that kind of reference point have to be maintained.
Luckily, this can actually still be emulated using relational-style tables, even though it's somewhat less efficient than classical storage techniques. Some filesystems already do something similar to this, and the results are promising. Look at Be's filesystem for an example of that.
The best way to go, moving forward, is something not unlike what BeOS did, with both hierarchical and relational methods of examining data. This allowed for the best of both worlds. The default method of getting at data is still the hierarchical paradigm, but relational searches can be applied to create what some have called "smart folders" (perhaps "boxes" might be a better term?) Systems like this "Storage" should be focusing on complementing traditional systems in this way, rather than replacing them.
Just wait till you see the way "Pivots" work in the new Longhorn shell. The canonical example is sorting thousands of mp3s by artist, but it'll be A-FUCKIN-MAZIN.
Face it: databases rock. You never know how many interesting questions you didn't ask because you couldn't think in sets until you do it, and then it's FAST as all get out.
"Files" are not a bad idea. It is nice to have an interface of commands that is limited in size and easily serialized (ie open/read/write/seek). If Unix had instead mmap'd files in it's original design there would probably not be transparent access to network file systems or many of the other things we take for granted today. So the design of files was actually a huge win.
1. The primary problem is implementation. Filesystems today are designed to store small numbers of very large files (ie more than 1K in size). Anybody who wants to store "objects" that are smaller than about 1K in size (like if you are implementing a "registry", for instance) is forced to write or use a database program, with needless complexity, to force all this data into a single file, so that it can be stored efficiently. What we need is a design where tiny files (like 4 bytes) can be stored efficiently.
Supposedly ReiserFS addresses this, but it is not clear if it does the necessary level of compression: ideally if you had 100 files with the same 50-bytes name and 1 byte stored in them, all those names would be in the same 50 bytes on the disk.
Sadly NOBODY seems to be trying this, and keep spouting "attributes" and "registry" and "config file". Those are all work-arounds for poor file systems.
All files must have the capability of being a "folder" and having subfiles. Any time anybody says "attributes" this should mean this sort of subfile.
2. The other problem is the blinders so people believe the "filename" is some sort of user-friendly data. This leads to brain-dead ideas like "case independence" and "wide characters" and the fact that certain bytes like "/" and zero are disallowed. This requires programs to cook data in strange ways to use it as indexes into the filesystem. This used to be true of the *data* in old systems, and we know now how horrid that was (only a rudimentary piece of that old stupidity remains in Windows text/binary distinction but I hope newer Windows systems will move that out of the kernel).
The filesystem should identify files with a counted length of bytes, just like the data in the file. In fact "name" should be a subfile of any file, and you rename it by writing a new "name". I don't think this can be solved without fixing existing filesystems.
(for "user friendly" names some form of quoting is going to be necessary. Since Windows has made "\" useless I would use that for quoting. "\0" is a null, "\\" is a backslash, "\/" is a forward slash. Just "/" itself indicates a break between hierarchy levels. For semi-Windows compatability you can also make just "\" followed by an unassigned code also mean a break between hierarchy levels.
3. The other thing that is needed (but could be done atop existing implementations) is to change the model of files. They should be "atomic" in that when you open a file for writing, you get an empty file, and this is invisible to any other program. The file only appears at the moment you close it, and only to programs that then open it for reading (programs with the same name already open continue to see the old file). Current files where you can replace a block in the middle are a special case that only a few programs use, and support can be operating-system dependent (and while you are at it, try making it so you can insert or delete data and not just overwrite).
4. As for "database" this can all be done with symbolic links (which can be implemented atop any file system which efficiently stores identical small blocks of data).
Where does this metadata come from? I assume I have to enter it myself. This means the more files I have, the more detailed and specific my data entry becomes. And that much more tedious.
Even worse is the uncertainty that would arise. Is my search for "horn solos" not returning results because there are no such files, or because the filesystem does not have meta data describing the files I want as such?
At this point, hierarchial organization once again becomes much more appealing again.
main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}