Slashdot Mirror
How Do You Organize Your Data?
kpellegr asks: "After returning from a well deserved holiday, I was faced with an exploding inbox. While organizing and deleting my mail, I realised I was having trouble classifying each mail into one specific folder. I had the feeling I should be able to link to one email from several folders (e.g. product information should be linked to from the 'vendor' folder, as well as from a specific project folder where this product is used). The more I thought about this, the more I realised that trees (such as the Windows filesystems) are not really ideally suited for organizing data. On UNIX-like filesystems, symbolic links allow the creation of simple graphs for organising data, but I have the feeling data could be organized more efficiently. How does the Slashdot crowd organize their data? How do you manage files, email, contacts, meetings and all the relationships that might exist between them?"
This is exactly the concept behind virtual folders. The idea is that folders, whether they be in the context of an email program or a filesystem, are actively updated searches. For example, all of your emails could be in one pool, invisible to you. Then each folder would be associated with a rule similar to email filter rules we use now. If an email matches, it shows up, maybe in multiple folders. Bayesian rules allow for even better classifications, if an email is similar enough to several catagories, it can show up in all of them.
Spencer Ogden
A dash of arbitrary directory trees and a pinch of grep.
But seriously, this subject is kind of lacking. The problem I have with organized storage is keeping it organized. I don't have the time nor the will. I need some sort of automagic organization.
A friend of mine used to use what he termed an archaeological filing system.
It was based on the simple principal that the older something was the further down in the pile it would be.
Your all-in-one-folder technique and "ls -t" would work equally well.
Boffoonery - downloadable Comedy Benefit for Bletchley Park
All mail are kept into one place (say, a MySQL database). You, however, setup filters (that is, SQL queries) that show your e-mails in virtual folders.
That is, messages can be in as many folders as they meet the selection criterion of.
In addition to the obvious "from", "date", "subject", you could assign an arbitrary number of categories which could constitute more selection criteria.
David Gelertner, the comp sci professor author and unabomber victim, has created software he calls Scopeware. It basically organizes information in a series of related chains. These can be date based or otherwise. I haven't used it, but I've read that he is responding to some of the same concerns you mention.
On a less lofty, but free, note, Evolution has "virtual folders" in which you can place anything a filter expression can select. I use them to sort my email by sender address. I still have my main inbox, and all the categorized subfolders, but the virtual folders select particular people out of the massive mail database. So I can recall that Joe said something three weeks ago that relates to a current problem, and look in the "Joe" virtual folder to find it. There's still no easy way to add arbitrary messages to a virtual folder, other than adding a filter rule that selects just that one message. At least I haven't found a way. But it seems to address part of your concern, for email at least.
"Even if you are on the right track, you'll get run over if you just sit there" - Will Rogers
Or is that KAOS (as in "Get Smart") ?
I'm currently playing around with putting all my mail messages, bookmarks, web pages loaded, file accesses (on a day to day basis) into a database. Maybe not all the actual data, but the stuff that might help me find it when I need it. I'm hoping to eventually scan everything that changes on my computer or that I do for keywords and so on and then organize them so I can browse them by some kind of visual graph/map metaphor on any of several axes (type of file, date/time, keywords, directory ....).
I want to be able to go in with a query like "sometime in july I did something having to do with a picnic and watermelon" and get a list of possibilities, then be able to rate those in the hopes of finding the exact info I'm looking for.
OK, so far I only have some pieces of it. But I'm getting closer to a database schema for the information and that will help me figure out better what info I need to collect.
I know other people have mentioned Evolution's vFolders, but here a little more.
My goal is to never have an email that has value to me land in my inbox. Every time I get an email of "value" which stays in Evolution's inbox, I right click, and "Create Filter from Message". (I'm paraphrasing.)
Every good message should have at least one filter putting it into at least one folder. Some emails have more than one rule, but the whole right click -> create filter thing makes this quick and easy.
-Pete
Soccer Goal Plans
I think MIT has a project called Haystack designed just for this
Ugh... hate to say it... Outlook client using exchange.
There I said it. Ok, to be fair, I use it because that is what is available and that is what everyone is use, all 800 or so of us... and that is in our org, which is a child org to a much larger org... so a total userbase of about 6000 users...
Here's why it works. I use partially Bayesian based InBoxer to kill spam. Our exchange server also runs Norton anti-virus (which has saved us from SoBig all that crap)... and then the exchange also has a spam filter which adds "spam:" to the subject of all incoming know spam e-mails (which does me not much good).
Ok, that takes care of spam. All list-serves I belong to get put into their own folders... Emails for friends get put into a specific folder. This leaves my inbox. My inbox is shared with all my 'trusted' co workers. When I am gone, they check it on a regular basis for me while I am gone. If I am expecting a high priority e-mail from a certain person, I set it up so an alert e-mail is sent to the right person then that comes in.
For my tasks, that is also shared. When I am gone, I forward my tasks that are due during that period to the right person.
My calander is also shared. On my calender, I mark when I will be gone, and then setup a special list of those who should be alert when they send me an e-mail or task during that period (this stops an e-mail alert being sent to those list-serves I am on when I am gone).
As for files: I manage the share on our central server that we all use. We just went through a major undertaking to get it up to par. ALL files are saved on the server. Everyone has a private drive, and then each 'task' or 'subject' or 'project' has its own folder on the server. Some folders are public, or 'all on our domain'... a majority are 'departmental access' (every one in our small org)... the rest are specific, generally with 3-4 people.
It takes work. But I have access to the files I need and so do the other people in my org. It takes a lot of user education, training, and hand holding.
Couple all this with decent VPN (cisco based) and most users get what they need when they need it.
Oh, and this is at a college. Most departments are as well off as we are. And, yes, slammer has been a bitch to deal with as students move in... but many dedicated staff have solved that problem (not to mention some ingenious network guys... hats off!).
I think this is a step in the right direction. I have been using it for a while now - check it out.
"The goal here is to do for email (starting with your personal mailbox) what Google did for the web... The Google principle: It doesn't matter where information is because I can get to it with a keystroke. So what is Zoe? Think about it as a sort of librarian, tirelessly, continuously, processing, slicing, indexing, organizing, your messages. The end result is this intertwingled web of information. Messages put in context. Your very own knowledge base accessible at your fingertip. No more "attending to" your messages. The messages organization is done automatically for you so as to not have the need to "manage" your email. Because once information is available at a keystroke, it doesn't matter in which folder you happened to file it two years ago. There is no folder. The information is always there. Accessible when you need it. In context." ZOE
I know other users have already pointed out how well Evolution works for sorting mail, but I just wanted to attest to how well it works even for large amounts of email.
I used to create new folders for specific types of email, but I found it very difficult to manage and search all the folders after a while, so I ended up moving all of my email to a single folder, Inbox. I currently have 24,949 messages in my Inbox and Evolution is still extremely fast when it comes to sorting and searching through them all.
I also make use of the excellent VFolders feature of Evolution, to save frequent searches into their own folders. I've been using Evolution now for several years, and it just keeps getting better and better.
--It's Pimptastic!--
For file systems I use symbolic links in a column viewed filesytem. I really like what a company formerly known as NeXT has done with some of their products. Their software for pictures and music both have a "Library". From there you can drag songs or pictures into "Playlists" (music) or "Albums" (photos).
Very cool.
As for software, I use OmniGraffle and OmniOutliner from OmniGroup. OmniOutliner is especially simple, yet unique. I wonder why no one else has an idea organizer that is so incredible? I couldn't do my job without it. Well, I could, but I'd use a lot of paper or spend a lot of time in OpenOffice messing around with things.