Slashdot Mirror


Shuttleworth On Redefining File Systems

moteyalpha writes "Mark Shuttleworth described the beginnings of what could a great step forward in making file systems more usable. I've personally had the experience of trying to find a file for a customer who had just finished editing a critical report, saved it, and then couldn't locate it to deliver to their client. Quoting: 'My biggest concern on this front is that it be done in a way that every desktop environment can embrace. We need a consistent experience across GNOME, KDE, OpenOffice and Firefox so that content can flow from app to app in a seamless fashion and the user's expectations can be met no matter which app or environment they happen to use. If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn't have to work with two completely different mental models of content storage.'"

29 of 414 comments (clear)

  1. This would be easy by CRCulver · · Score: 4, Informative

    I've personally had the experience of trying to find a file for a customer who had just finished editing a critical report, saved it, and then couldn't locate it to deliver to their client

    Was it a Word file? Locate all .docs, run them through antiword, grep for words from that critical report, and report back the matches. Less than a minute of Bash scripting.

    1. Re:This would be easy by larry+bagina · · Score: 5, Insightful

      or search by last modified time.

      --
      Do you even lift?

      These aren't the 'roids you're looking for.

    2. Re:This would be easy by antifoidulus · · Score: 5, Insightful

      Or just use spotlight, Mac users have been able to do find files quickly for years.

    3. Re:This would be easy by haeger · · Score: 5, Funny

      Less than a minute of Bash scripting.
      Obviously you're not a consultant.
      "If you're not part of the solution there's good money to be made in prolonging the problem."

      Do you honestly expect ANY customer to pay you if you solve their problem in less than a minute?
      Back to school young grasshopper, you're obviously not ready for the real world.

       

      .haeger

      --
      You are not entitled to your opinion. You are entitled to your informed opinion. -- Harlan Ellison
    4. Re:This would be easy by MrCoke · · Score: 5, Insightful

      You don't get it.

      My grandmother could use spotlight. She won't be able to use find, locate and grep.

      And that is the target audience of Shuttleworth's point: the Computer Illiterates.

    5. Re:This would be easy by tolan-b · · Score: 4, Informative

      So like Tracker that comes installed by default on Gnome based distributions then?

      Or Beagle, that was released somewhat before Spotlight.

    6. Re:This would be easy by samkass · · Score: 4, Insightful

      Next I suppose you're going to say that rsync is the same thing as Time Machine, when in reality while they may be in vaguely the same arena of functionality, they are orders of magnitude different in utility. Instant searches of both local and remotely accessible drives tied to various easy filtering and categorization functions makes Spotlight a game-changer. Just like always on, incremental, and back-through-time searches and intra-file record retrieval (ie. 1 address book entry, photo, song, etc) make Time Machine a game-changer.

      --
      E pluribus unum
    7. Re:This would be easy by Facegarden · · Score: 4, Insightful

      You don't get it.

      My grandmother could use spotlight. She won't be able to use find, locate and grep.

      And that is the target audience of Shuttleworth's point: the Computer Illiterates.

      I agree, everyone on here acts like linux is way better because it's had this stuff for ages but i STILL can't use most of it because it requires spending hours online searching for answers (and when you're trying to get the internet working in linux on a dual boot machine, it's hell... you have to reboot to something else, search for answers, reboot to linux, try it, forget what you had to do, reboot...)

      I don't really need to try hard to make the argument because you guys either already know what i mean or you pretend like it's easy ("duh just type ~rf - m" or something something, because yeah, a menu to do that would kill someone).

      Anyway, yeah, spotlight is probably nice. Google desktop is also awesome. I especially like being able to just double tap control to bring up the search, type what i want, it's right there.

      Anyway, now that i've pissed off everyone...
      *hides*
      -Taylor

      --
      Worldwide Military budgets: $2100 billion. Worldwide Space Exploration budgets: $38 billion. Really, world? Really?
    8. Re:This would be easy by chromatic · · Score: 4, Funny

      We don't press enter.

      That'll free up space on the new Macbook!

    9. Re:This would be easy by jonaskoelker · · Score: 4, Informative

      "duh just type ~rf - m" or something something, because yeah, a menu to do that would kill someone

      For no one thing would a menu item to do that thing be particularly bad. But you can't put _every_ task in a menu, because there are infinitely many tasks.

      If you find people often tell you to type in commands you don't understand, it's probably because it's the most efficient way to do something once you do master it. See for instance http://www.linuxquestions.org/questions/linux-newbie-8/modifying-functions-678643/. I've built a 1650-byte podcatcher in #!sh [and that's including proper error-checking and all].

      It's also dense communication-wise; compare "sudo ifconfig eth0 192.168.0.1" with "System -> Administration -> Network; unlock, wired connection, properties, enable, static ip, 192.168.0.1".

      That being said, though, deskenvs should support the most common and important tasks in an easy-to-use way.

    10. Re:This would be easy by Daimanta · · Score: 4, Funny

      "Or just be a frickin file nazi."

      Yeah, but exterminating .doc files makes me sad :(

      --
      Knowledge is power. Knowledge shared is power lost.
    11. Re:This would be easy by Risen888 · · Score: 5, Funny

      Yeah, but exterminating .doc files makes me sad :(

      Not me. They're a plague, it's time for a final solution.

      --
      Hey, I finally got my first freak! Took you long enough!
    12. Re:This would be easy by fatphil · · Score: 5, Insightful

      Who cares? Are people really so insecure about their OS that they want the reinforcment of knowing that even senile grandmothers can and do use it? Personally I would prefer it that the idiots _weren't_ using the same OS as me.

      If you (not *you* you, the generic "other" you) don't care where you are storing your files, then I don't care if you find it hard to find them. Removing the usefulness for ad hoc organisation, and improving search functionality is tantamount to just doing away with the hierarchical file-system altogether. Welcome to the 60s - enjoy your stay. This "it shouldn't require me to think" attitude is the attitude which gets people driving cars off river embankments because of their reliance on their GPS system.

      Yes, this patronising and pure, unadulterated, snobbery; I won't pretend otherwise. No need to flame me for it; I already know.

      --
      Also FatPhil on SoylentNews, id 863
    13. Re:This would be easy by zapakh · · Score: 5, Funny

      Honestly, "I just saved a file and now I don't know where I put it" is more indicative of the human operating the computer, than it is of the computer apparently lacking facilities to find the files.

      And this statement perfectly demonstrates why Linux is not now, and will not be for a very long time, a true Desktop OS.

      I couldn't find my keys this morning. It's a real shame, too, since I used to think my car was roadworthy. But it turns out that it expects me to know where I put my keys, and blames the user when they go missing.

      I just want to drive to work! I don't want to manage a bunch of keys...

  2. Semantic desktop by oever · · Score: 5, Interesting

    Merging the efforts of Nepomuk and new file systems like brtfs are the way to go with this. Handling of file events can be done better than what we have now with inotify. File systems should allows plugins to update indexes on files within the file system structure and file systems should allow queries and query monitors directly.

    DBPedia shows the power of SPARQL and implementing an efficient storage for it into a file system is the first step forward. Then user interfaces in GNOME and KDE can take advantage of the queries that are currently very expensive to do.

    Ubuntu is in a good position to help out on the Nepomuk effort. Mandriva is already sponsoring this work. More support for this desktop-independent project would be a boon for achieving the file system Mark is looking for.

    --
    DNA is the ultimate spaghetti code.
    1. Re:Semantic desktop by harry666t · · Score: 4, Interesting

      Why not the other way around?

      Get rid of the file system at the OS level.

      Treat every document (text file, source code, song, album, web page, picture, movie, contact, email message, etc) as an object, with associated tags: class, interfaces, metadata, etc, and store it in a database.

      Locate the objects by querying the database, not by going through a hierarchical tree (eg. 'find-object type:program, name:"python" version:2.5' instead of "/usr/bin/python2.5" - yeah, more typing, but consider this: when the object is not found, the system already has ALL the info necessary to download and install it, and all of its dependencies - just catch the exception thrown when "no matches found in local storage", invoke apt-get like magic on it, retry).

      Remove the distinction between "regular" memory and cache, and just make whole RAM a big cache for accessing on-disk objects. Ensure object persistence across reboots - no more shutting down, hibernation the only way. No swap file or partition needed - the whole disk is a swap area. The anonymous memory (the malloc()/new one) is no longer anonymous - it belongs to the process object. Versioning - use the free space to hold older revisions of every object (unless explicitly marked not to do so - in case of /var like stuff or highly confidential data), and rm old backups as more space is needed for "current" (or more recent) objects.

      Permissions - get rid of Unix permissions or ACLs and use capabilities, eventually use different namespaces for each running process (like Plan9 does it). Get rid of all-powerful root.

      Exchange of objects over the network? Serialization (for example, turn the "image" object into a regular jpg or png, store other metadata in associated xml file, and pack everything into a zip or gzipped tarball).

      Legacy apps? Implement the traditional file system API in a library, some LD_PRELOAD tricks or whatever. For example every object with a "type: program" tag would be accessible from /bin/, /sbin, /usr/bin...

      The only problem?

      $ apt-cache search ".*"|wc -l

      25221

      The transition would be ***painful***.

      Anyway, if I'd be doing an OS from scratch (I tried some time ago - definitely not a task for a team of one human), that's how I'd do it.

  3. Expansive syntax, and the work required.... by ka9dgx · · Score: 4, Informative

    Version control, searching, and all of the other advances since the first directory tree are good things to add, but they must be supported down to the application level.

    VAX/VMS had a wonderful system of versioning baked right into things, if you worked on a file, it kept versions for you as you saved them....

    login.com;1
    login.com;2
    login.com;3
    .. etc.

    The default was the last version, unless you explicitly chose a different one. This is an incredibly useful tool, and I still miss it to this day, 20 years after I last used it.

    If you can't express an idea explicitly, your power of expression is radically limited. If we can get consensus and support a bigger set of expressions, we can do a whole bunch of cool new stuff. As long as we follow the leader, we'll never do anything this innovative, and we'll always be playing catch up.

    It won't be easy!

    To do even this simple thing with Linux, all of our applications would have to be re-written to enable a new file specification syntax, hopefully one reasonably compatible with the past. We're talking about a shitload of work, so it's important to agree on a set of goals first, to avoid having to re-do it later.

    --Mike--

    1. Re:Expansive syntax, and the work required.... by jonaskoelker · · Score: 5, Insightful

      To do even this simple thing with Linux, all of our applications would have to be re-written to enable a new file specification syntax,

      Why? Hans Reiser demonstrated that files can be directories too, without breaking the VFS layer.

      Say we bake versioning into the file system. You want the old versions of /home/user/shopping-list.txt; you go look in /home/user/shopping-list.txt/old/1. If you want the one from yesterday you go for /home/user/shopping-list.txt/old/bytime/2008-10-24.00:00:00, and the file system figures out which of the old versions was present at that time.

      Same old syntax. The name resolution is handled differently, but that's all in the file system. You could probably even write a fuse file system that adds a layer of versioning on top of another file system. No need to ever touch the apps.

      If you want the duct tape solution: write a shell script that checks whether anything changed every n minutes, then commit your home directory to subversion/git/....

      Do you have any numbers on how much space was used on extra versions for a "typical" distribution of files and usage patterns? TANSTAAFL and all that ;)

  4. Why? by wumpus188 · · Score: 5, Insightful

    ... just finished editing a critical report, saved it, and then couldn't locate it to deliver to their client.

    Well, there are some people who can't find Pacific Ocean on the map. I dont see map makers running around in panic, thinking how to make their maps more accessible to the general population...

    1. Re:Why? by Sentry21 · · Score: 4, Insightful

      Yes, but most users don't create the pacific ocean, save it in the default location, and then have no idea whatsoever where that default location is, because it's not a fixed location across all apps, it never says it anywhere except the Save dialog, and it's not a sensible, obvious default.

      That's the problem we're trying to solve here - not 'clueless users lose things', but 'regular users get confused when every program saves somewhere different by default'.

  5. What's wrong with directories? by Prof.Phreak · · Score: 4, Insightful

    ...had the experience of trying to find a file for a customer who had just finished editing a critical report, saved it, and then couldn't locate it...

    None of this would be an issue if folks were competent and created directories themselves, and Word (or whatever) asked where to save stuff, as opposed to just assuming (or insisting on) some default system provided directory.

    Am I the only person who hates those "My Documents" folders? Or on a Mac iTunes insisting on putting music in a certain weird place? I want to create my own folders, and maintain why own directory structure, and know exactly where stuff is because I put it there---not because Microsoft/Apple/Ubuntu think that's where I should keep stuff.

    For the most part, maintaining my own folders for stuff works out just fine (easy backup, easy moving among environments, etc.), except when some program assumes it knows better, and saves a file "somewhere"; really hate it when that happens.

    ie: The problem is caused by Microsoft/Apple (and Linux following) to cater to stupid users who just want to create a document and not care where it is saved. Those same users probably wouldn't be able to locate the file (for copy/backup, etc) unless they use the same program they used to create the file.

    --

    "If anything can go wrong, it will." - Murphy

    1. Re:What's wrong with directories? by Mwongozi · · Score: 4, Insightful

      Or on a Mac iTunes insisting on putting music in a certain weird place? I want to create my own folders, and maintain why own directory structure, and know exactly where stuff is because I put it there---not because Microsoft/Apple/Ubuntu think that's where I should keep stuff.

      I know that you're not really looking for a solution - but that behaviour in iTunes is optional. You can turn it off, and then iTunes will use whatever folder structure you've already got.

      The thing is that 99% of iTunes users don't know, and don't want to know, exactly where in the filesystem their music is stored, they just want to click on iTunes and see it. So that option is on by default.

  6. Simple solution by Waffle+Iron · · Score: 5, Insightful

    If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn't have to work with two completely different mental models of content storage.

    And you wouldn't have to, if every app would just show the frigging directory tree as it exists, instead of trying to fool the user with a random bunch of stupid fake roots in every GUI.

  7. recent experience with a new Linux user by bcrowell · · Score: 5, Insightful

    My mother in law in upstate NY had a Windows box that she used for exactly two purposes: email, and playing online Scrabble. Her Windows machine got full of malware, to the point where it wouldn't even boot. While she was visiting us in California this summer, I set up a machine with Ubuntu for her to use, and she got fairly comfortable with GNOME and Firefox. I sent an Ubuntu install CD home with her on the plane, and she went ahead and installed it with virtually no problems. I only had to talk her through a couple of issues on the phone, the main one being non-Linux-related: her BIOS wasn't set to boot from a CD.

    She got going with email, and then it was time to get her set up for scrabble. The one she plays isn't the famous facebook one, it's a java program that accesses a club's server in Romania. Well, I think I spent about an hour with her on the phone, and we still don't have it working. One thing that took us a heck of a long time was that when she downloaded the jar file for the scrabble app, neither of us could figure out where the file had gone. Probably if I'd been in the same room with her it would have only taken me thirty seconds to locate the file, but over the phone, it was more like I was experiencing it from her point of view, and it was completely confusing. She was clicking around in the Firefox download manager, in the GNOME file manager, all with no luck. It seriously took her about 20 minutes, *with my help*, to find the file. It probably didn't help that I use fluxbox myself, and am not familiar with GNOME or its file manager. (Now we're almost there, except that apparently she's got a completely dysfunctional version of the java runtime installed. You click on the widgets in the program's UI, and it doesn't respond.)

    Anyway, what kind of indictment is it of Firefox/GNOME's usability when it's easier to install Linux than to find the file you just downloaded?

    Of course now I have to slap a steel helmet on my head to withstand the inevitable onslaught of know-it-all slashdotters telling me what an idiot I am, and how I could have easily found the file. Of course that's always how it is with usability. To the person who already knows how to use the software, it seems painfully obvious.

  8. Re:We need a tag based filesystem by vadim_t · · Score: 5, Insightful

    You think it's going to be any better for people who can't find things they saved?

    They can't find it because they didn't care at the time of saving to attach enough information to the file to be able to find it later. Instead, they saved it under a name like "letter5", or even worse, "asdf", and possibly left it in a random directory as well.

    Tags won't be just as bad, they'll be worse. They require a considerable effort to tag consistently. You also have to think of all the possible tags that could be related to the file. Is it "friends", "acquaintances", "buddies", etc? Is it singular or plural? Will "birthdays" be enough, or you also have to file it under "parties", "celebrations" and "events" in case you remember the file you need was related to some sort of celebration but you can't remember which?

    What happens with categories that are diffuse, change meaning, or their contents? For instance, take emails from Alice, that initially get tagged with "acquaintances", then progresses to "friends", then "significant other", then "ex". If you search for something that was mentioned in a friend's email, are Alice's emails tagged as they were initially (in which case after the upgrade from acquaintances to friends her previous mail needs an extra keyword to find), or have they all been updated to "ex", in which case the search might fail since she was a friend back then?

    Coming up with a good keywords system is something that only geeks and secretaries are going to do. Your average person will at best pick a couple keywords, then complain they can't find stuff because they didn't use the right keywords, or that every single document comes up because all the mail is tagged as "email" and nothing else.

  9. Re:In other news, by peragrin · · Score: 5, Interesting

    Why not in OSX a combination of OS and filesystem data allows me to move applications while they are in use. I can uninstall an application or change it's directory location and all files know how it gets opened again and where it moved to.

    Literally you can move an application to a thumb drive, and the next time you open a file that launches said app it will try to mount the thumb drive if it isn't already done so. I get so frustrated in locating files under windows. Why can't the OS get out of my way so i can work? if it takes a file system, and OS to track them properly the so be it.

    As for the missing file in the description. most applications include a recent file list as does windows(since 95) and OS X. if you can't remember where you put something, try checking those speed lists first. It is like people only want windows and then refuse to read the dialog box that pops up every time they click on the start button. It is as bad as bill gates with outlook open goes to the task bar to find a calendar.

    --
    i thought once I was found, but it was only a dream.
  10. Re:I think he failed to identify the problem by m.ducharme · · Score: 5, Interesting

    So the user decides not to pay attention to where the file was saved (I mean, you do get to choose where it goes, it does not just happen)

    This isn't strictly true, as each program has a default action on saving a file. Some default to desktop, or to "My Documents" or ask for a directory by default. Others may do "smart" checking and try to sort the files depending on what file type gets detected. It's this wide variation that's the problem. Even for someone who is knowledgeable enough to handle the situation without help, it's still a pain in the ass (and waste of time) to dig out the setting and replace it with your preferred setting.

    Yes, that is unpleasant, but is additional complexity in the file system really the best solution?

    Probably not the best solution. Better would be for the OS developers to set a standard directory scheme and enforce other developers to comply with it. If you want to change the scheme you can dig in and do it yourself. Of course, this may not be practical, as it may prove more complex to enforce compliance (and still keep things customizable) than it would be to bolt on the same functionality in the file system.

    that at some point there needs to be a minimum expectation of competence on the part of the user.

    But who gets to decide what that expectation is? Don't forget, computers are not "for" hobbyists, developers, experts and geeks, they are for normal people to make their other tasks easier. The more time these people spend training and raising their minimum level of computer knowledge, the less time they spend doing their real work/play, the less value the computer has for them.

    Is it really too much to ask of a user that they understand that it is a machine, an inanimate object, and it generally does only what they tell it to do (insert Windows jokes here), and that if they tell it to do something by mistake (like saving a file in an unintended location), the mistake is theirs and not the machine's?

    Yes it is too much to ask, because it's not true. Computers as machines do what developers tell them to do. The amount of instructions given by the programmer vis-a-vis object code vastly outnumbers the number of instructions a typical user is going to issue to the machine. This is necessarily true since each instruction given by a user is a blind proxy for at least one instruction (and probably more like hundreds or thousands of instructions) given by a developer.

    Users can't be expected to predict a) all the myriad directions that a programmer has given/is giving the computer and b) the effects of the user's instructions when interpreted in the context of the programmer's instructions and assumptions. Most users are bad at using computers effectively because they don't understand how developers think. You wanna try teaching them how you think?

    If that is too much to ask, then what is a more reasonable standard? How far should we go to accommodate users who, to put it bluntly, refuse to take responsibility for their actions?

    Well, in my opinion, I think we're pretty close to the reasonable standard right now. People can decide for themselves how much they want to learn, how much pain they want to suffer for their ignorance versus how much time they want to spend reducing their ignorance, and vote with their wallet. You're basically saying that you want users to learn more, to make developers' lives easier, but in fact developers get paid to make users' lives easier. Maybe the developers should do their jobs better.

    It's like that Unix saying, (paraphrase) "Unix doesn't try to stop you from doing something stupid, because that would also stop you from doing something clever". I like that, not because I think it's witty but because in my opinion, it reveals a design philosophy that assumes that maybe this is new to you and you don't und

    --
    Rule of Slashdot #0: You and people like you are not representative of the larger population. - A.C.
  11. Re:I think he failed to identify the problem by TuringTest · · Score: 4, Insightful

    So the user decides not to pay attention to where the file was saved

    Of course the user paid attention to where the file was saved: it was saved in Word, inside the Save As... dialog. But when she tried to recover the file using the Open..., the file was no longer there. That's what Shuttleworth means when he says there are two completely different mental models of content storage: one is tied to functionality in applications, the other is a tree of folders and files.

    Users understand perfectly that it is a machine that should do only what they tell it to do. They get upset when the machine *doesn't* do what they told them, because the machine changed their data to a different level of abstraction that they don't know about. To someone without a complete mental model of the inner workings of a computer, those different abstraction levels are a source of utter confusion.

    You geeks only see the last one, and typical users only see the first one - and when they are required to jump the gap between the two completely unrelated abstractions, they are lost. At least the "My Documents" kind of folders tries to simplify the model so that users don't have to learn the two models.

    So don't blame the users of something that is fault of the software designer because of their insufficient research about the human API. Throwing layers upon layers of abstraction is a good way to tell programmers how the machine works, but it's not good for everybody else. If you designed a machine that only required one abstraction layer to be used efficiently, users would love to learn it to the highest proficiency.

    --
    Singularity: a belief in the "God" idea with the "demiurge" relation inverted.
  12. Re:In other news, by woot+account · · Score: 5, Informative

    This is largely the idea behind GoboLinux I think. As a matter of fact, a lot of what's going to be said here has probably already been said here.