Slashdot Mirror
Shuttleworth On Redefining File Systems
moteyalpha writes "Mark Shuttleworth described the beginnings of what could a great step forward in making file systems more usable. I've personally had the experience of trying to find a file for a customer who had just finished editing a critical report, saved it, and then couldn't locate it to deliver to their client. Quoting: 'My biggest concern on this front is that it be done in a way that every desktop environment can embrace. We need a consistent experience across GNOME, KDE, OpenOffice and Firefox so that content can flow from app to app in a seamless fashion and the user's expectations can be met no matter which app or environment they happen to use. If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn't have to work with two completely different mental models of content storage.'"
Was it a Word file? Locate all .docs, run them through antiword, grep for words from that critical report, and report back the matches. Less than a minute of Bash scripting.
Merging the efforts of Nepomuk and new file systems like brtfs are the way to go with this. Handling of file events can be done better than what we have now with inotify. File systems should allows plugins to update indexes on files within the file system structure and file systems should allow queries and query monitors directly.
DBPedia shows the power of SPARQL and implementing an efficient storage for it into a file system is the first step forward. Then user interfaces in GNOME and KDE can take advantage of the queries that are currently very expensive to do.
Ubuntu is in a good position to help out on the Nepomuk effort. Mandriva is already sponsoring this work. More support for this desktop-independent project would be a boon for achieving the file system Mark is looking for.
DNA is the ultimate spaghetti code.
Unless you store it at very cold temperatures. In that case users expecting to be able to sip away at their water may be a little disoriented when someone hands them a block of ice.
Yeah, that's going to work for someone who's trying to make money selling Linux.
Hail Eris, full of mischief...
E pluribus sanguinem
Version control, searching, and all of the other advances since the first directory tree are good things to add, but they must be supported down to the application level.
VAX/VMS had a wonderful system of versioning baked right into things, if you worked on a file, it kept versions for you as you saved them....
login.com;1
.. etc.
login.com;2
login.com;3
The default was the last version, unless you explicitly chose a different one. This is an incredibly useful tool, and I still miss it to this day, 20 years after I last used it.
If you can't express an idea explicitly, your power of expression is radically limited. If we can get consensus and support a bigger set of expressions, we can do a whole bunch of cool new stuff. As long as we follow the leader, we'll never do anything this innovative, and we'll always be playing catch up.
It won't be easy!
To do even this simple thing with Linux, all of our applications would have to be re-written to enable a new file specification syntax, hopefully one reasonably compatible with the past. We're talking about a shitload of work, so it's important to agree on a set of goals first, to avoid having to re-do it later.
--Mike--
... just finished editing a critical report, saved it, and then couldn't locate it to deliver to their client.
Well, there are some people who can't find Pacific Ocean on the map. I dont see map makers running around in panic, thinking how to make their maps more accessible to the general population...
Or Akonadi/Nepomuk in KDE4. There's a lot of metadata that you can attach to your files and documents to make it easier to search in the future.
Comment removed based on user account deletion
KDE has had this feature for awhile and unless the window's search has improved radically since Windows 2000 it's still just as bad.
So...it seems that someone on Linux has realized the benefit of UI guidelines.
Linux: 30 years behind Apple on the desktop - but they're catching up!
So the user decides not to pay attention to where the file was saved (I mean, you do get to choose where it goes, it does not just happen) and later has difficulty locating it. Yes, that is unpleasant, but is additional complexity in the file system really the best solution? I am honestly not sure how I feel about that. At the same time I agree that there are "user error" type of problems that better technology can either prevent or mitigate, I also feel like some of the proposed solutions I have heard are borderline ridiculous, that at some point there needs to be a minimum expectation of competence on the part of the user.
Is it really too much to ask of a user that they understand that it is a machine, an inanimate object, and it generally does only what they tell it to do (insert Windows jokes here), and that if they tell it to do something by mistake (like saving a file in an unintended location), the mistake is theirs and not the machine's? If that is too much to ask, then what is a more reasonable standard? How far should we go to accommodate users who, to put it bluntly, refuse to take responsibility for their actions?
It's like that Unix saying, (paraphrase) "Unix doesn't try to stop you from doing something stupid, because that would also stop you from doing something clever". I like that, not because I think it's witty but because in my opinion, it reveals a design philosophy that assumes that maybe this is new to you and you don't understand everything right now, but one day you do wish to understand how the system works and you do wish to achieve a degree of mastery over it. I really believe that just about anyone who really wants to understand something can do so, that gradually getting better and better at something over time is the most natural thing in the world unless you keep telling yourself that it's too hard. That's why I really don't understand these "permanent newbies", the people who can use a system for five years without grasping the basics. They claim that they are not interested in understanding, but it seems like they are strongly interested in not understanding. Is there something to be gained by accommodating this?
It is a miracle that curiosity survives formal education. - Einstein
Seriously, there are efforts in place for more advanced filesystems - but it's all to no avail when the linux kernel will neither merge these into its tree, nor provide a stable API for them to be maintained outside it. It's kernel politics that's the biggest thing holding back linux filesystem development.
I am trolling
...had the experience of trying to find a file for a customer who had just finished editing a critical report, saved it, and then couldn't locate it...
None of this would be an issue if folks were competent and created directories themselves, and Word (or whatever) asked where to save stuff, as opposed to just assuming (or insisting on) some default system provided directory.
Am I the only person who hates those "My Documents" folders? Or on a Mac iTunes insisting on putting music in a certain weird place? I want to create my own folders, and maintain why own directory structure, and know exactly where stuff is because I put it there---not because Microsoft/Apple/Ubuntu think that's where I should keep stuff.
For the most part, maintaining my own folders for stuff works out just fine (easy backup, easy moving among environments, etc.), except when some program assumes it knows better, and saves a file "somewhere"; really hate it when that happens.
ie: The problem is caused by Microsoft/Apple (and Linux following) to cater to stupid users who just want to create a document and not care where it is saved. Those same users probably wouldn't be able to locate the file (for copy/backup, etc) unless they use the same program they used to create the file.
"If anything can go wrong, it will." - Murphy
to crappy applications isn't to tack it onto the kernel. Ken Thompsons paper, written 30 odd years ago, reprinted in linuxjournal.com/article/2792, provides a better framework for understanding why this approach is so brain damaged.
On the other hand, every program within Gnome [ or whatever the desktop bling of the day is ] should co-ordinate ideas like 'recently referenced/saved/etc.... files', so that after clicking save in 'on app', I can find it immediately in the next app. That doesn't need any kernel work [ or even use of inotify like hacks ] to achieve.
how?
./
Alright, the good (!?) folks at Redmond have a nifty --if limited-- solution to this problem, but we're talking about Linux here.
Also, that 'pirated music' bit wasn't a rant, it was a use case. It would be good if read more carefully TFS before trolling on teh
Meddle not in the ways of filesystems. Lest you succumb to the curse of Reiser.
Evil people are out to get you.
We need a consistent experience across GNOME, KDE, OpenOffice and Firefox so that content can flow from app to app in a seamless fashion and the user's expectations can be met no matter which app or environment they happen to use.
In Windows we have survived several such attempts. e.g. "Recent File" or "Documents" in Start menu; or the useless location buttons in open/save file dialogs.
Let's just hope I would be still able to open a file at random location. Because the statement makes me feel that eventually I would be able to find all possible files - system thinks I may need to find, but not the files I actually need.
I've personally had the experience of trying to find a file for a customer who had just finished editing a critical report, saved it, and then couldn't locate it to deliver to their client.
Orienting future development on full idiots worked well in past... Or not? Well, GNOME full of it already and another drop of inusability into the mix will not hurt much its rabid fans.
As they say, give man a fish...
P.S. rfc1925, ch 11.
All hope abandon ye who enter here.
The best system that I've seen that provides a unified name spaces, database, and file system is the very notorious Reiser File System. You can find out more at: http://web.archive.org/web/20071023172417/www.namesys.com/.
What I like about it is the ability to search, find things, and the way that it can handle tiny chunks of data in their own tiny files as well as massive data files and directories with massive amounts of data. Who needs a database when one has a system like ReiserFS?
Bringing the advanced file system name space and search capabilities upwards into applications and presenting it to the user ensures that one uniform model is presented from the ground up through all layers of the illusion to the user.
Maybe the current implementation has problems but it can be improved...
While I'm not using the actual implementation in my system to end all systems I'm making use of a number of the key ideas in the ReiserFS system. Read the papers on the archived web site linked above.
If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn't have to work with two completely different mental models of content storage.
And you wouldn't have to, if every app would just show the frigging directory tree as it exists, instead of trying to fool the user with a random bunch of stupid fake roots in every GUI.
If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn't have to work with two completely different mental models of content storage.
Yet another abstraction layer.
The higher the technology, the sharper that two-edged sword.
I should be able to tag a file with "friends, birthdays, gifts" and be able to find that file again using any of the terms.
Hierarchical filesystems are ok for systems management, but they are crap for assigning meaning to user data. A hierarchy implies that any particular file can only have a single aspect when in fact it may have dozens of aspects.
Deleted
Is a new file system really necessary for doing that? How about just restricting the user to what directories he's saving files to. ie:
.imafuckingstupididiotwhoiswastingyouroxygen go to the imafuckingstupididiotwhoiswastingyouroxygen folder
.doc files go to the "doc" folder
.txt files go to the "txt" folder
.jpg files go to the "jpg" folder
Think of it as a car problem. The user is having a hard time remembering how to keep the car on the road. Instead of redesigning how the road is constructed, install guard rails. If the driver still manages to drive the car off the road, the driver is an idiot and the gene pool is better off chlorinated.
greed@All_Evils:~#
When I misplace a file I simply search on its extension for all files modified today. I never realized that this was such a big deal.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
Or you could just go back to the recently used file list and resave it in the proper location.
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
Remembering where files that you last accessed or other forms of this type of organization shouldn't be part of the filesystem at all. It will only make things go slower. If the main problem here is that people don't remember where they saved their files (a la PEBKAC) then there should be something that should be mandatory for open file dialogs (in gnome(gtk2) or kde(qt3/4) - a recently accessed file tab. Hell, if you want, you can even have it remember all files that were last accessed in a small db if you want to make it desktop environment-wide for all programs to access instead of only finding the files accessed by that specific app, but i think that would be overkill.
Anyways. Shittleworth should remember what the Linux kernel is, and that it shouldn't be designed around stupid peoples preferences. (yes, i've been using Linux since the mid 90s so I'm no anti-gpl troll). Nepomuk is an interesting project to check out. As for making a filesystem that does all this stuff by itself - not worth it.
What does any of it have to do with a better file system?
Sounds like he wants Office.
Seriously, what the hell was his problem in finding the latest document someone "misplaced" in their PC.
Recent Documents?
find/locate?
I can think of more ways to find a recent document, but so can any other reader here.
But, this idiot thinks it's a new idea to have... What do they call it, a DOCUMENT STANDARD?
Can't even call this one fake advertising.
--Toll_Free
My mother in law in upstate NY had a Windows box that she used for exactly two purposes: email, and playing online Scrabble. Her Windows machine got full of malware, to the point where it wouldn't even boot. While she was visiting us in California this summer, I set up a machine with Ubuntu for her to use, and she got fairly comfortable with GNOME and Firefox. I sent an Ubuntu install CD home with her on the plane, and she went ahead and installed it with virtually no problems. I only had to talk her through a couple of issues on the phone, the main one being non-Linux-related: her BIOS wasn't set to boot from a CD.
She got going with email, and then it was time to get her set up for scrabble. The one she plays isn't the famous facebook one, it's a java program that accesses a club's server in Romania. Well, I think I spent about an hour with her on the phone, and we still don't have it working. One thing that took us a heck of a long time was that when she downloaded the jar file for the scrabble app, neither of us could figure out where the file had gone. Probably if I'd been in the same room with her it would have only taken me thirty seconds to locate the file, but over the phone, it was more like I was experiencing it from her point of view, and it was completely confusing. She was clicking around in the Firefox download manager, in the GNOME file manager, all with no luck. It seriously took her about 20 minutes, *with my help*, to find the file. It probably didn't help that I use fluxbox myself, and am not familiar with GNOME or its file manager. (Now we're almost there, except that apparently she's got a completely dysfunctional version of the java runtime installed. You click on the widgets in the program's UI, and it doesn't respond.)
Anyway, what kind of indictment is it of Firefox/GNOME's usability when it's easier to install Linux than to find the file you just downloaded?
Of course now I have to slap a steel helmet on my head to withstand the inevitable onslaught of know-it-all slashdotters telling me what an idiot I am, and how I could have easily found the file. Of course that's always how it is with usability. To the person who already knows how to use the software, it seems painfully obvious.
Find free books.
"throw out the old "files and folders" metaphor and leap to something new" - sounds like WinFS?
"They import a picture into F-spot and then have no idea how to attach it to an email." - picasa?
"single-file version control system" - Shadow Copy
"We need a consistent experience" - Windows, OS X
"it would be phenomenal if free software were leading the way"? - I don't see any new ideas here that hasn't been implemented before.
Why not in OSX a combination of OS and filesystem data allows me to move applications while they are in use. I can uninstall an application or change it's directory location and all files know how it gets opened again and where it moved to.
Literally you can move an application to a thumb drive, and the next time you open a file that launches said app it will try to mount the thumb drive if it isn't already done so. I get so frustrated in locating files under windows. Why can't the OS get out of my way so i can work? if it takes a file system, and OS to track them properly the so be it.
As for the missing file in the description. most applications include a recent file list as does windows(since 95) and OS X. if you can't remember where you put something, try checking those speed lists first. It is like people only want windows and then refuse to read the dialog box that pops up every time they click on the start button. It is as bad as bill gates with outlook open goes to the task bar to find a calendar.
i thought once I was found, but it was only a dream.
From the headline I thought this was going to be about lost/cross-linked clusters not idiotic users forgetting where something is saved. You can improve the problem in this story by implementing better and more intuitive search features through to overhalling the traditional file-browser window implimented on all OS's that I see.
These changes are independent on the underlying file system which could be anything from fat16 through to reiser ... heck it could even be PICK or a relational DB. Of course - some FS's naturally lend themselves to a particular style of search/browsing - but that simply makes it easer/harder for the interface developer. They are still very seperate things.
Changing the file system will NOT solve this problem
Redundant but why not:
If you can't find the file, there's a user problem here.
Physics is like sex: sure, it may give some practical results, but that's not why we do it.
Bring back the filesystem as a database and allow extensible columns attached to any file.
select * from pr0n where hair = 'red';
Err, I mean, uh...
select * from reports where type = 'TPS';
It seems to me that we are trapped in an old way of thinking without even realizing it. Why do we follow a spacial model for storing things when we don't need to? Have we asked ourselves the question: "Is it beneficial that we have to remember a location for each file?" Perhaps it is. As humans we have become very good at remembering where things are for later recall. However, for storing media, it makes little sense. A traditional file system with nested directories allows you to organize files, but no matter how clever your directory structure, some files will defy your organizational model. With iTunes-like media interfaces we are moving away from location-based recall to CONTENT based recall. Unfortunatly ID3 tags are overly limiting, but it feels like a step in the right direction (at least for media files). Could this be a the right direction for the rest of our file system? Content based recall instead of an imposed location based recall?
- Captbaritone
You know, I've never understood why we have one big filesystem with everything located on it anyway. Why does a user need to be able to store his files with the system binaries, or hidden in the logging directory, or in the directory for shared resources?
I think filesystems should have "slices" where different types of files live in different slices. All user files will always be in the user slice (and more specifically that user's slice) so they can never get mixed up. Within that slice, sure, they should be able to organize to their heart's content, but there's no reason to allow mixing.
Breaking the filesystem up into slices would allow for greater security as well. Think of it like chroot jails, on steroids, combined with the power of things like nosuid per-partition.
If a a user process had read only access to the executable slice, than no amount of buffer overflow would ever write code into that partition.
Downloadable executables? Put them into a sandbox slice which allows you to see what they do. No chance of some screensaver in email rifing through your bank data, etc...
The problem is that application dialogs often open on unexpected places. Some don't attempt to remember where you last saved something; some do, and end up in even worse places. For example, if you went out of your way to save something in an obscure directory last time, the next time you try to do a save it will also land in that obscure directory.
The latter at least is easy to discover, but most users never think of the solution: just "save as.." again, and see where it takes you.
If all applications agreed on a standard API to discover the "last used" directory, neither would be a problem. Let's say you accidentally save a file in %USERPROFILE%\Application Data\Mozilla\Firefox\ or something stupid like that, well, when you open your email client to email the attachment to someone, it will ALSO open up that directory, and you'll see your file.
In other words, this is easy to solve at the application layer, we just need a tiny bit more cooperation from the applications. The "last opened" directory should be a trivial API available on every platform, and should be a basic service, not part of something high-level like Gnome VFS.
It's rare that you're presented with a knob whose only two positions are Make History and Flee Your Glorious Destiny.
This is largely the idea behind GoboLinux I think. As a matter of fact, a lot of what's going to be said here has probably already been said here.
By what name do you wish to be mourned?
yeah, the UI could really embrace the tree structure and help the users maintain it!
It has options for dealing with times amongst other things.
There is software for Windows in the wild that will search across all connected drives for a word or phrase. It's been years since I used any Windows product but I wish a global for of grep existed for Linux users. Obviously the search might take an hour or two but at the very least it will find what you ask it to find. In the ideal case it would be able to search various forms of zip files also. That would leave only encrypted files as unsearchable.
I wish, with humonguous hard drives we have today, that the file systems would natively support and administrate tags. They'd be attached to the file but seperate from the content.
Every application or device (camera) could add tags to the files it works on (a jpeg file: image, gimp, etc) and users could also add tags (vaction, dec.2008, skiing, gps.coordinates....).
I'm really tired of folders, that force you to shoehorn into one category something that can fit in multiple categories (no, I don't want to mess with symbolic links) and also people trying to stuff the entire contents of a file into the folder name:
billclintonfunnydemotivatorobamacampaign2008.jpg
It doesn't even work most of the time. The alternatives is to get people to tag their images individually, no matter if it's downloaded or their own. This just requires a duplication of effort on internet downloaded ones and is also stupid since it needs to specific apps and those apps are likely not cross compatible (+ probably extra files that contain the tagging database). In short, anything but file system tagging is doomed to failure.
How about:
% find $HOME -newermt "now - 5 minutes"
for finding those pesky files you just wronte under a weird name or path...
This is what I do lately when those stupid annoying GTK programs do their thing and illogically refuse to save files by default relative to the CWD where I started the program. I wish they would stop de-unixing unix.
It sounds like you are talking about something like Filehand Search for Windows. By default it chooses all the places that folks normally dump files,and is easy to add any drives/directories you desire. Low resource usage,oh and free is always good too. And I agree it would be nice to find a product that worked as easily in Linux.
ACs don't waste your time replying, your posts are never seen by me.
Perhaps someone needs to take a step back and think about the principles involved. Then they might see that there isn't actually a problem at all, other than that their desktop is not really doing much.
The "can't find my document" issue is just a proper consequence of data classification and access control, and what he sees is evidence that it is doing its job well.
If every document were stored in the same directory, you could never fail to find a file if you knew its name. That strategy would create other problems instead though, like collisions and selection overload, which is why we use a hierarchical storage space to segregate the files. In effect, this gives each plain file a hierarchical classification prefix that is directly supported by the operating system.
The fact that you can no longer access a file if you know only its name but not its "classification prefix" (in other words its parent directory pathname) is not a problem, it's an essential part of managing the data space and avoiding a file glut nightmare. Without it, our esteemed Mr Shuttleworth would instead be complaining about collisions and data overload.
Is there a solution? Of course there is: multiple views. (And one could improve good old-fashioned searching as well.)
In Unix, Linux, *BSD and similar systems, the name of a file is just a link to its actual contents, and you can have any number of names linked to the same file content. This can be used very easily to provide quick file access for forgetful desktop users through auxiliary views.
Just create a new "view" directory (say "LAST_USED_FILES_VIEW") in a standard place, and use links to populate it with names derived from the filenames used by applications. The desktop just needs to ensure that newly created or opened files get an additional link to them placed there, which can be done automatically. A desktop application could then file away its documents in sensibly-chosen hierarchical storage space, yet its application files would also be immediately accessible within "LAST_USED_FILES_VIEW" (and it wouldn't be a desktop fiction, but the real thing at O/S level). This is quite easy to implement with the help of the inotify(7) mechanism to keep the desktop informed of changes in the filestore, for example.
Not being able to find a file doesn't mean that there is a problem with applications nor that operating systems are inadequate. It merely indicates that your desktop hasn't done enough to make items easy to find.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
They require a considerable effort to tag consistently.
No. With tags you don't have to carefully choose keywords or categories. You just describe what the file is. The words become the tags. Remove the "fors, its, the ands and the thes" and finding any file becomes simply a case of describing it.
Deleted
VAX/VMS had a wonderful system of versioning baked right into things, if you worked on a file, it kept versions for you as you saved them....
login.com;1 login.com;2 login.com;3 .. etc.
The default was the last version, unless you explicitly chose a different one. This is an incredibly useful tool, and I still miss it to this day, 20 years after I last used it.
Indeed. Files-11 originated on RSX (PDP-11) even before VMS (VAX). I recall using it almost 30 years ago. I also recall being appalled on exposure to the file systems of personal computers (PET, Apple-II, IBM PC), and discovering that they lacked file versions. This is probably less of an issue nowadays, since big cheap disks are common. It would cause problems (and give benefits) for those who do a lot of image/media editing and use a significant amount of disk space for images or other media.
There is a substantial disk space overhead in versioning file systems, which was probably one argument against them in early personal computer days. The number of versions of a file increases on each modification, and each version occupies disk space.
However, there is also an administrative overhead in versioning file systems. To prevent disk space exhaustion, it is necessary to limit the number of versions kept or to purge older versions. This can be automatic (losing some of the benefit of versioning), or on request (requiring thought and action from the user: bad idea). I recall having various command files in RSX to purge different UICs in various ways and regularly issuing commands such as
PIP DL:[10,33]*.*/PU:4
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
That's fantastic! I mean, if all you have on your computer are office files, I guess. What about the other 90% of the world's data? See, in the real world, people use their computers for lots of different kinds of things. A musician, for instance, would not find your solution to be terribly applicable.
Hey, I finally got my first freak! Took you long enough!
The hierachrical system is an outdated concept based on traditional methods of meat-space organization.
I'm surprised no one else has mentioned this yet, but a metadata filesystem would save so much time. Tracks would be autotagged with timestamp, size, and filetype if possible, but allowing the user to set their own custom tags would bring out the real benefits.
It would work like Google, where every file is in a single folder. You would an ultra-fast filter window to narrow down to any criteria. Tags could be reused easily (the filesystem would save recently used tags in a dropdown menu on opening or saving), and a thesaurus would help detect near misses if need be. Here's a little more info:
http://www.skytopia.com/project/articles/filesystem.html
In any case, for all the hardened folder adherents, there's no reason why they two systems can't coexist.
Why OpalCalc is the best Windows calc
But it's not about preventing the user doing something stupid, because nothing can really do it, it's about *error recovery*.
An OS which provide system wide file versioning and a journal of events to allow easy recovery of misplaced file doesn't prevent the user from making mistakes, it allow them to recover from them which is much better!
Same thing for system wide undelete, crash-only computing (here it's about allowing users recovering from developers mistake)..
Unix is really lacking here compared to VMS as someone else said (system wide file versioning), this allowed a simpler implementation, but as the cost of lost user-friendliness..
"To do even this simple thing with Linux, all of our applications would have to be re-written to enable a new file specification syntax, hopefully one reasonably compatible with the past. We're talking about a shitload of work, so it's important to agree on a set of goals first, to avoid having to re-do it later."
So basically the timeline comes to the file system. Good thing there are standards.
Shai Schticks:"You don't make peace with friends, you make peace with enemies"
I am a very experienced software developer and have been using command-line and UI based systems since I was 12. I cut my teeth on TRS-80 Model I, II, III; Commodore 64; Atari 800/1600, Apple II/IIc/IIe, Aquarius, etc, etc. Moved to developing in DOS, then OS/2, and then Linux (back in the Slackware, calculate your video card mode-line days). I've developed software from small desktop systems to enterprise-wide heterogenous systems. I've supported nationwide networks, client-bases, and software deployments. So, I think I'd qualify as an *expert*. That being said, I think all OSes currently have A LOT of cognitive disonance in the handling of files by various applications. IDE's, Editors, Office Software, Photo Software, Graphics Software, Music Software, Web Browsers, etc, etc, etc, etc. They consitently handle things in subtly different ways and present subtly different models. If not for the fact that I'm such a command-line jockey I would not be able to even remotely keep the various models straight in my head. It is only that I can consistently map each individual applications file/storage model to the underlying hierarchical filesystem, that I can understand what is going on. I think this type of mental gymnastics is WAY WAY WAY too much to ask of the average computer user who wishes to use it as a tool to get a job done. And, as a paying contributor to the GNU/FSF, and a developer who is making an effort to become more involved and more contributory to the GNU/Open Source/Linux ecosystem (like so many others), this is, along with the leadership provided by all the pioneers of the open soure/free software world, exactly why Linux WILL AND IS GETTING THIS RIGHT!
Over-the-top Response Guy! Giving "Over-the-Top Responses" since 1970.
Could someone explain to me why they think I'm being modded flamebait? I didn't aim for that, that's for sure...
Tracker and Beagle already do this for Linux. They are very fast too, since they scan new/modified files and build an index of their content and metadata.
http://www.mhall119.com
Heaps of metadata is just going to slow things down and every application is going to be interested in different metadata. As was said above changing the filesystem isn't really going to help.
This is largely the idea behind GoboLinux I think.
Well, it is not.
GP is talking about moving a file around and having the system keep track of it none the less.
Gobolinux redefines the standard file system hierarchy and moves away from traditionnal packaging system by allowing you to use multiple versions of a same package/program and using the filesystem as a database.
Maybe it's not the interface you want, but you can get this very, very easily in Linux (or in fact, any Unix-like, even Cygwin) without any third party software.
find /media/disk1 /media/disk2 /home -type f -exec grep -Hn "monkey" {} \;
You can wrap that in a convenient shell script or even a GUI if you want. You can narrow down to just .txt files by adding -iname "*.txt" as well, and so on. find is ridiculously useful and it's a shame so many people never harness it.
Sam ty sig.
People in real life very often do not attach that kind of information to documents and objects, yet they don't completely suck at finding them later (even if they're far from perfect at it). Some folks are better at organizing stuff, and some are worse, but typically the creation of new paper documents and their classification and storage for later retrieval are separate activities, often performed by different people.
I really think the current UI paradigm for files is just wrong. Here's my (not thought through) ideas about it (some copied from various sources, others inferred but with no claim to originality):
Yes, under this system, some people will spend too little time filing, and thus make a huge sprawling mess of their desktop. That's not great, but it's better than the current paradigm, because they will at least know where their recent document is (in the desktop), and at least one method for finding it (looking at the documents on their desktop). The point is to make the system easy to use, not to fix the character flaws of the user.
Are you adequate?
"Why can't the OS get out of my way so i can work?"
But what you're talking about is the exact oposite of that... you're talking about an OS that tracks changes you make, so when you try to use something that you've moved, it will automatically compensate and use the new location. If the OS got "out of your way", you'd have to look after this yourself.
Incidentally, Windows has the 'distributed link tracking' which (by sounds of it) does something similar (if enabled).
The revolution will not be televised... but it will have a page on Wikipedia
It is not the fault of the filesystem that applications save things in different places. Sure, a monoculture of apps will minimize this, but at no point is it a _file system_ problem. That's like saying its an operating system fault when your desktop background image is ugly.
Fix the apps, fix the users. Leave the filesystems alone.
I want to delete my account but Slashdot doesn't allow it.
I agree that trees have outlived their usefulness. When complexity and quantity exceeds a threshold, it seems hierarchies are not longer sufficient.
I've talked this over with others, and it seems they would rather enhance the file system rather than outright replace trees. Paths could still serve as a starting point. However, adding a multiple key-word (category) system on top of it could be very helpful. One would add existing keywords/phrases to any given file plus custom ones not in the list if needed[1]. Then one could search either by selecting keywords/phrases from checkboxes or pulldown-lists (depending on size), and/or by typing them in Google-like queries.
The creation date and change date range could also be used as part of the query so that the example customer could find his recent file.
[1] If you give keywords/phrases by typing them in instead of predefined lists, it could tell you which ones are not in the list. One would be encouraged to use the existing keywords when possible to avoid duplication. However, a synonym system may also be of help.
Table-ized A.I.
It would be trivial to have a pseudo directory tree (Linux already uses them for /proc and /sys) which allows you to see any regular filesystem via whatever view you felt like having. A series of meta-tags can be represented by a graph (not a tree, since there are multiple paths to the same point) of directories, one directory per tag, of those tags you specifically want to look for, provided there is a complete graph of all tags stored on the filesystem. Instead of using user-provided tags, have a self-organizing network or an expert system shell generate the tags for you. Or have some combination thereof.
To extend this concept to work with networked filesystems such as NFS and Lustre, you want something that caches the way SQUID does and CODA was supposed to, with the capability of remote systems flagging a page in a file as dirty in much the same way ccNUMA does. To get scalability, just use NORM or one of the other scalable reliable multicast protocols. In terms of web content, you're talking nothing more complex than the 80s idea of PUSH. The ideas are already there, but have never been brought together in the way required or to the level required. But they could.
I posted a slightly longer version of this on Mark Shuttleworth's blog, but it was flagged as spam by some nannybot he bought that itself is clearly in need of an urgent re-write. Yeesh. The ugliness of the software currently out there horrifies me. The fact that I'm currently employed to write work-arounds for botched-up business software (and the fact that it's cheaper TO have someone fulltime to de-munge the output commercial software than buy something else) just illustrates to me how bad things currently are. There's an old saying that if builders built buildings the way programmers wrote programs, the first woodpecker that came along would destroy civilization. I regret to say that this saying is wrong. Based on my current experience, the first air movement from said woodpecker's wings would have destroyed civilization long before the woodpecker got there.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
What one may want to do is adding tracking data at some other layer that uses change-notification mechanism for indexing. However for the sake of everything that is, was, will or might be holy, stop trying to resurrect "everything is a database, and files are collections of records" model of early IBM and Digital operating systems. It was hated and replaced with a modern "named chunks of bytes in a tree" for a reason.
Contrary to the popular belief, there indeed is no God.
I realise that this is jumping off on a tangent from your comment, but there was (still is?) a bug whereby if you use Ubuntu with Compiz Fusion then Java Swing will mis-calculate the position of clicks by an amount approximately the size of the window decorations. I vaugely remember that there was some way you could fix this in Java itself, but I just disabled Compiz Fusion and I've been happy ever since.
Also, you should make sure that you have the sun-java6-runtime package installed and selected as the default JVM.
hook the file system api and update a subdirectory in home with a global list of soft-links to recent used files grouped/tagged by application and operation... make that subdir a link on the file open/save common dialogs
From the headline I thought this was going to be about lost/cross-linked clusters not idiotic users forgetting where something is saved. ... These changes are independent of the underlying file system ...
Alas, Mark Shuttleworth chose to use the term "file systems" in referring to the method(s) that people use to organize data on their computers. Before 1972 or so, few people would have been confused. However, I'm intrigued by efforts -- such as Jef Raskins' Archy project -- that appear to deliberately conflate the two notions in service to the end-user.
That said, I'm not keen to use a system that forces me to use somebody else's scheme for document storage and retrieval. In 25 years of using UNIX, I've developed (and continue to refine) my own filing schemes. I'm more than comfortable with the "files and folders" model and my understanding of the underlying (hierarchical) storage scheme serves me well.
Beware the new dogma, especially when it's based on the notion that "one size can fit all" -- apparently the implicit guiding principle behind Windows and MacOS. Not always, but often, there's too much dumbing down in the process of presenting the system through a GUI. As an expert user, I hate to be slowed down for routine or repetitive tasks by moving a mouse, highlighting icons, and selecting from menus; give me the command line interface. For tasks that I don't perform very often, such as configuring a network interface, invoking a calculator program, or (shudder) launching a word processor, I appreciate a simple, easy-to-use GUI with consistent behavior across the O.S. (There's the rub.)
If Mark's goal is a one-size-for-all file storage model, then I will likely never agree with it. Not because I think I'm better or smarter, but because people don't think alike or work alike. Give us systems that support diverse filing schemes -- not just by content/topic or file name, but by date, tagging (user-supplied metadata), origin, file type, whatever.
Understanding the low-level model of a UNIX file system plus a few commands (including the byzantine "find") allows an experienced user to do some of this, but that initial learning curve can be a bitch. So find ways to make that expressive power accessible to beginners and occasional users. And do so in ways that help them develop a useful mental model of what's going on, and facilitate the transition to power user status (for those who want it).
*** Rant off
I'd settle for simple consistency in the menus for common operations such as opening and closing files from applications with GUI front ends.
oh, like "grep -r /"?
Yeah, that would take a while.
i forget
We need a consistent experience across GNOME, KDE, OpenOffice and Firefox so that content can flow from app to app in a seamless fashion and the user's expectations can be met no matter which app or environment they happen to use.
Like iLife?
Seriously, what apple has done in iLife is very good. Each iLife app has access to the music, movies, photos from the other iApps, including play lists, photo albums etc...
If the Linux bods can emulate and build upon this functionality it would be a huge step forward for usability.
{} \; forks and starts a grep for every file found. /media/disk1 /media/disk2 /home too. Or using rgrep, that saves you one more character to type.
{} + is rather what you might want to have - so it starts a grep with several files as an argument.
Although with GNU grep you can as well simply grep -rli "monkey"
HTH
"First they ignore you, then they laugh at you, then they attack you, then you win." -- Mahatma Gandhi
Just one word - beagled. :-)
Mark! While you 're reading this, send me my 2%
My 2 cents about content management: have an application that keeps track and remembers the last 20 files or so opened with any application. The program works like this: it's split up in to three parts:
- A daemon that keeps track of what application wrote data to what file (this should be possible, right?), and stores this info in a database.
- A graphical application with a list of "applications that accessed files" on the left hand, and an empty pane on the right hand. Click any application in the list on the left and the right pane will show the files (+ document titles, possibly) it accessed (sorted by last date first).
- An applet to quickly access the application and have some options/settings available quickly.
And then some useful extras in the graphical app like setting the importance for a file and putting files into a seperate paragraph; think like this: there's some small bold paragraph text labeled "Last accessed files" with the bunch of last accessed files below it, and then below that another paragraph "important" or "some custom label"... And one global area to put any type of file in any paragraph.
I think it'll definitely help those people who are a mess when it comes to file storage... At least, I'd be able to find my latest/most important things back more quickly, even though I'm very tidy when it comes to file/folder structures.
I am not devoid of humor.
The problem is that the user couldn't find a file. The solution is a better INTERFACE - this has absolutly nothing to do with the file system!
I agree with you. To make file systems work for average users, what you want is this: full text searching on the contents (plus name, tags and other metadata) and an index that updates as soon as the file is saved. Possibly with some smarts on parsing the query, though Google demonstrates that you don't need that much there. This then needs to be integrated into the standard open dialog, and apps need to use the standard dialog rather than rolling their own (I suppose we ought to have a few open dialogs, e.g. one for "text" files and another for images). The vision then is that users can search for it using what they remember about it: this will work better than anything else since it doesn't assume the operator has a tidy mind.
BTW, the "update the index immediately on saving" part is important; too often I've seen users close a document and then immediately think "ooh, I didn't mean to do that". This means that you can't put off the update until a cron job, and you hence need incremental update of the search database. (It's this that Spotlight on OSX gets wrong too.)
As you say, none of this has anything to do with actual filesystems; you could even do it with FAT12 (though that's a horrible horrible FS!) This is all about applications and how they present the FS to the user. (I'd not present the actual directory structure by default; it doesn't help the untidy minded and the tidy can click/shortcut to reveal the structure.)
"Little does he know, but there is no 'I' in 'Idiot'!"
No link tracking only works for shortcuts, something apple has had since i think system 8. No I can move(aka drag and drop) the preference and data files for one particular application to my encrypted drive. Now the next time that application loads it goes to look for it's preference files at the new location which then tries to mount the encrypted drive. That asks for a password to decrypt it, and then mount it. The application then finishes loading.
Try moving firefox's bookmarks some times and see what happens? On pure OS X apps nothing changes as far as the end user knows, In windows or linux it thin s there aren't any bookmarks.
i thought once I was found, but it was only a dream.
So an application that wanted to use this facitily would just create a shortcut to its configuration data (or whatever)... when you move that data, the shortcut gets updated, the app knows where to look. Personally I don't like the operating system doing things for me, if I want an app to look elsewhere for files, I'll tell it, I don't want application settings (including search paths) changed during file operations. Just different way of using your system I guess.
The revolution will not be televised... but it will have a page on Wikipedia
I know most here would find the XO journal very limiting but in my experience it's a nice way of working for not savvy users. basically it stores entries like which application, which filename at which time. For example Yesterday you used application "Write" to write such and such file. You can filter it and it's very straight forward. The save and open dialogs are wired into it and it's fairly easy to find stuff. Or we could have something like BeOS/Haiku BFS full of metadata and a nice tracker.
-- Por mais que eu ande no vale das trevas e da morte, meu PowerMac G4 Não Travará!!!
Or just use spotlight, Mac users have been able to do find files quickly for years.
UNIX systems have had far more powerful text search engines for more than a decade. You could also alway buy far better add-ons for all major platforms.
Spotlight doesn't cover many file types, its ranking suck, and its user interface makes searching through large numbers of files painful. Spotlight is typical Apple: putting lipstick on a dog.
My experience with naive users is that they have only a hazy understanding of the existence of a file system. They think in terms of apps. If I ask them "where did you save that file?" they say "in Open Office". If I ask "where do you save your MP3s?", they say "they're in Amarok". Using a file manager like Konqueror, Dolphin or Nautilus is unattractive to them; they don't see any benefit to it because they don't really understand that there is a file system underlying their usage of apps. I'm not sure what the answer to this is. Having applications rigidly throw everything into /home is probably the best short-term solution.
That is the most I have ever learned from a single Slashdot post. Thank you very much :)
Sam ty sig.
I've used database filesystem. They are hard to learn, give people bad habits and get unmanagable when they get bigger. Another problem is that metadata get lost when you communicate with other systems
The best, and simplest, filesystem I have ever used was Sintran III. It was based on descriptive filenames, where parts of the name was separated with a divis, that could be shortened every time the file was used after it's creation. Much easier to learn and almost as powerful as a unix shell. It took about twenty minutes to learn to a total computer illiterate, no I'm not kidding. I believe SINTRAN actually had one level of subdirectories (it had user and group areas), but nobody used them because they didn't make things simpler. You just need a good name and some simple metadata like creation and modification date. When you got files that came from other users or groups, there where mechansims that added their name at the end of the file name. Ie: you get a dokument from user John Doe, it was renamed so that it ends with -FROM-JOHN-DOE, then you only have to add -FR-J-D to the filename when you wanted to use his version (unless, of course, you had documents from a Jane Doe, then you had to add -FR-JO-D to find his document). Version numbering, add -VERSION-2-2 to the filename, find all version 2 documents with FILE-NAME-VER-2- (or even F-N--VER-2--FR-J-D-). Want to edit the latest dokument with a filename that starts with BIG-SCARY-DOCUMENT-, type W-P B-S-D- at the command line (like all REALLY user friendly computer systems, SINTRAN was mostly command line based, it only took a few bad ones (UNIX, DOS, CP/M ...) for users to become scared of the command line).
Apparently I was unclear in my critique of Mark Shuttleworth's proposal to replace the "filing system" model implicit in GNU/Linux. Understanding the hierarchical model of UNIX file systems works for me, but not for a user like my mother. (I helped introduce her to computing at age 72 or so, though she opted for Windows over the MacOS approach that my spouse and I originally recommended. I watched how files piled up on her computer over time. It's clear that she doesn't grok files and directories, but she doesn't need to understand that model for what she does.) By the same token, her "filing scheme" would never suffice for the thousands of files that I use in my own projects.
Perhaps the easiest problem to tackle is that there appears to be no consistent set of rules for saving files by default. Application A stores files in one place, and application B in another. Your 99.9% of users probably expect that data is data, and therefore that it should be "all in the same place" unless they say otherwise. Turns out that their expectations are more sophisticated than the "system design", which is little more than aggregating applications from disparate sources.
Mark could make a notable contribution to usability if he could get the developers of widely used applications -- Open Office, Gnumeric, Acrobat Reader, Firefox, the GIMP, etc. -- to save files in a consistent fashion -- consistent across applications. Then develop some kind of Spotlight-type wrapper for "find" for those users who don't use an explicit filing system to organize/retrieve their work. Such a scheme would make it easier to form an accurate and useful mental model of how the system works in this regard. It's not revolutionary technology, but would make life easier for a lot of users (myself included), and just might be simple enough (to implement) that developers would buy into the idea. A more radical approach is likely to die on the vine.
Try MacOS X. Pretty consistent... oops, that's not Linux;^)