Haystack: A More Compelling View Of Your Data

Peristaltic writes "MIT's Haystack project has released the source for it's "Universal Information Client", Haystack. In their words: 'Haystack looks into the use of artificial intelligence techniques for analyzing unstructured information and providing more accurate retrieval.' Unlike some attempts I've seen in the past to pull it all together on my desktop, Haystack shows some promise -- One of it's more useful features allows you to take the information you've been wallowing through, and have Haystack continually refine a 'dynamic hierarchy' until you get what you need. Haystack also performs some neat tricks such as combining Email, IM, web pages, etc. into a single inbox."

Comments

That's great and all, but..

[notque]

Haystack also performs some neat tricks such as combining Email, IM, web pages, etc. into a single inbox

It may just be me, but this is a feature I never want.

I do not want 1 large program to run all of my applications. I do not want to get my email, from where I get my web pages, and my IM. I don't want any of this.

I am quite happy with seperate programs which I can use at my pleasure. I'm happy with the lack of bulk, and the fact I can change an email client without changing a web choice. (although I only use pine anyway.)

Is this just me? Do all of you want your programs shoved together in one large application?

I didn't get any options on my cell phone (like text messaging) because I purchased a cell phone. I wanted a cell phone. To make calls. Nothing else.

Re:That's great and all, but..

[rdeadman]

I think what Haystack is trying to solve is the data management issue. For thirty years we have been living with application-centric computers. So much so that we think in terms of best-of-bread point-tools. Do we know where Mozilla stores our email folders? No, its hidden by the application. (Okay, I do, but that is because I'm a bit geeky and share my Mozilla email folders from a File Server across my intranet...) How about Outlook, Netscape, Eclipse, etc.

In my inbox I have folders for home, each client project I am working on, future leads, charitable organizations I am involved in. A similar parallel hierarchy is repeated in my file system for documents. My IM tools have their own way of tracking contacts that is unrelated to my email or projects. I store my Eclipse projects in yet another place. Mozilla organizes my bookmarks in yet another hierarchy. It's all a real mess and makes working on a project a job of mentally mapping all the pieces together.

Now, what would be real nice would be if Haystack could define a plugin API (a la Eclipse) so that my email client could be wrapped and plugged in to Haystack. Same for IM clients, web browsers, etc. The point tool then only has to worry about its job and hands off data persistence to haystack. Then I can choose the best app and let Haystack worry about tying the data together. As someone else mentioned, this sounds more like a replacement for the file system. But it could be more, if each plugin could define how it interacts with other plugins and defines its own responsibilities.

I'm sure there is a lot of refinement needed, but it is an interesting new paradigm. Activity-centred desktop insteaed of a tool-centred desktop.

I just want a relational filesystem...

[Xerithane]

Not too much to ask, it doesn't even need to be truly a filesystem. Just overload all the file access commands (At this point, probably easier to just write a new filesystem)...

Group data by category, content, whatever. "Symlink" to the inodes, and you're off. We don't need AI for that and I think it would be a more complete solution. I don't see an AI engine that can correctly categorize my mp3's, I don't think I'd trust it for all of my data yet. Let's start small and get usable systems.

Spiffy program though, wish it weren't in Java... wish it weren't 42MB... wish it ran smoothly under Linux. I'll stop complaining now.

On a side note, Did anybody else find that scrolling image annoying and mentally confusing. Er, I'll really stop complaining now.

Agents...

[orn]

Haystack is an interesting idea, but I have a hard time distinguishing what it does from what, say, Lotus Notes does. And Lotus is _terrible_.

I like the idea of bringing all my information together in one place. I don't like the idea of only having it in that one place. What I would like would be an application that can watch how I use the computer, then bring those applications together to make it more seemless.

For example, I have about four different calendars in my life: the work calendar, the one on the cell phone that I use for stuff that I can't miss, the calendar that schedules airplane rentals, and (of coursE) my girlfriend's calendar. So how do I bring those all together, and yet still be making entries in them separately?

The same is true for information. I have a primitive blogging system (really just a bunch of text files that are date coded), I have work documents that I use regularly, I have web pages that I monitor (sometimes a little too often) and I have textbooks that I'm reading (instrument flying at the moment). So how do I get all these forms of information - or at least an index into them - together in one place? But again, without changing the current organization scheme.

This is the tool that will make the computer a lot more useful - an actual organizational tool.

Rudy

Re:Agents...

[gobbo]

Wholehearted Agreement with the parent. Lotus Notes is shoved into our laps at work, and it's been a struggle to part out its functionality into the proper parts: Mail.app, Safari/Camino, Address Book (waiting for propr LDAP support, grr), iCal, and other 'business' tools, on my machine. [Not that I'm an Apple Software Fanatic, but they work and fit into the budget.]

L.Notes had a whole wing on the now-MIA Interface Hall of Shame. It reinvents the conventions found on other platforms (it tries to be a platform unto itself) and does so badly; it's buggy, slow, and designed for administration [decent encrypted document database scheme].

Plus, it centralizes, for better or worse, all my information on servers controlled by I.T..

Now I'd love to have a central app that takes feeds from my favourite info management apps, sorts/ranks/prioritizes/interrelates the items for me according to my usage and prefs, and lets me 'zoom in' to a task by switching to the preferred stand-alone app at will. Haystack has only part of the picture, the model is still gather-control, rather than sift-sort-go.

One item I've found intriguing is StickyBrain, a sticky-on-steriods app, by Chronos LC, which takes info in many categories and allows for quick index searching, plus offers system-wide info-archiving services and some alarm and word-processing features. I had the same kind of thing running with BBEdit, a notes directory, and grep, but it was like hammering nails with a wrench.

I want all my info hotlinked to lists of related items, dynamically: make every significant word a keyword, realtime. After all, what are multi-GHz and piles'o'RAM for, anyway, when not rendering?