Microsoft's Search Engine Plans

prostoalex writes "Andy Beal from SearchEngineGuide.com interviews Robert Scoble from Microsoft. Scoble tells the audience what current search technologies Microsoft is working on as part of its Longhorn/WinFS development as well as in the field of Internet. Scoble also discusses current problems with local drive and Internet searching, such as absence of metadata for a lot of files out there: "When I take pictures off of my Nikon, they have some metadata (for instance, inside the file is the date it was taken, along with the exposure information) but that metadata isn't useful for most human searches. For instance, how about if I wanted to search for "my wedding photos?" Neither X1, nor Windows XP's built in search would find your wedding photos. Why? Because they have useless names like DSC0001.jpg and there's no metadata that says they are wedding photos.""

Comments

Search by date

[Saven Marek]

I can get around searching for "wedding photos" because I remember the date. 3 special days, and hundreds of wedding photos appear.

It's part of being human that we don't necessarily remember the phrase "wedding photos" but we may remember many other tiny pieces of data about a shoot that are unique to us, and the time and date are one of those. I can be certain the post 9pm photos done on those days are pretty embarassing.

Just concentrating on "Wedding Photos" is useful if someone else is searching my picture archive, but that's not useful to me

nude geekgrrls

Re:Search by date

[higgins]

David Gelernter built a system called Lifestreams that basically claimed that time-ordered series plus some simple search and organization operators was everything you needed. It always seemed like a pretty good idea to me.

That said, if you do have metadata available, you can do a lot with it.

Re:Search by date

[alangmead]

What if a system's search could draw information from all the applications within the system. For example, if your electronic datebook had a day long entry for "wedding" and the photo manager has photos taken on that date, then a search for "wedding photos" would be able to find out when the wedding was, match it up with the date the photos were taken, and come up with the appropriate set.

To some extent Apple tried this with "Newton Intelligence" on the MessagePad. If you wrote "Thursday, Lunch with Bob at Redbones" It would (after you fixed all of its handwriting recognition mistakes) look up Bob and Redbones in the address book, look in the calendar for the next occurrence of a Thursday, and schedule a noon time appointment.

Newton Intelligence really only amounted to a small set of interapplication tricks, but it was assumed that as the popularity of the units increased, the functionality would be extended. (which pretty much tells you what happened to it.)

Re:I'm not buying it

[Saven Marek]

Even easier than putting into directories is using a portfolio type application, like Picasa (the original version of Apple's iPhoto btw) which allows simple drag and drop library creation. You can have pictures in multiple libraries, it just takes a small few moments to drop photos into their correct places and they are sorted as need be. If you want wedding photos, look in there if you want photos of janine, kate or benson look in their respective folders.

It doesnt need to be a morass of embedded folder after folder either, as humans have mental acuity unlike a computer. You may have uncle bob who is photographed a lot and auntie beryl who isn't, but all the photos of beryl you may know will contain bob. We can store a surprising amount of information, and perhaps 5 to 10 libraries is all you will need for most peoples collections.

Special occasions get their own. It just takes moments after downloading the photos.

nude geekgrrls

Check out Phil Greenspun's similar idea

[Speed Racer]

Phil Greenspun has a similar idea and is looking for help on how to accomplish this on a personal level with existing the Windows XP filesystem. Check out his blog post for details. There's already an intersting discussion taking place in the comments for that post.

Re:Ouch

[Otter]

Yeah, I'm sure being a computer science visionary is harder than it looks. But from the outside, all they seem to ever do is to announce that computer use is difficult because software developers aren't as smart as them, and that what we really need is some way for everything to magically sort itself out. Details of implementation to be left to those of less rarified brilliance.

The closest thing to a workable scheme is Gelerntner's Lifestream stuff -- where your system knows that you got married on a certain date (even if you have trouble remembering it) and that documents (JPEGs, Word files, GNUCash transactions from that time probably pertain to it.

Adding metadata is not the way

[Flyboy Connor]

Scoble's idea is that you will add metadata to your files. Can you imagine? You have literally tens of thousands of files you created (photos, documents, etc.) on your hard drive and you are going to add metadata to all of them? Does he really think people are going to do that? If they would be willing to do that, they would just rename those photo files from "DSC00001.JPG" to "MyWedding00001.JPG".

Judiging from the interview, the "innovative" Longhorn seems to allow you to add metadata in a slightly user-friendly way. But virtually nobody will use it, except maybe to mark a few important files which you have stored in a special place anyway.

So what would be a better solution then? My idea is that metadata should be added automatically. For instance, a human will recognize most wedding photos for what they are. Getting a computer to recognize this is not trivial, but lots of research is currently invested in this. Already computers can easily recognize general categories ("groups of people", "nature", "animal", "portrait"). My guess is that it is already possible to implement a system that you can train to let the computer recognize your particular brand of photos.

I don't expect Microsoft to try to go into this way of innovation. They will probably wait until an entrepeneur develops it and then copy it or buy them out.

More feature creep

[krygny]

How many people have trouble finding files on their hard drive using the most basic search criteria. People who are so unorganized as to lose files on their hard drive are probably not sophisticated enough to use advanced search methods successfully.

MS already does this and nobody uses it

[Pointy_Hair]

Ever look at the properties page of an MS Office file? There's enough metadata tags in there to keep you busy for hours.

Does anyone really fill those in? Rarely.

Is there a method to search on them? Never looked.

Sometimes it's interesting to browse the properties page to see who really created a spreadsheet or document. For example, people who shamelessly "borrow" templates from former employers and either aren't smart enough or too lazy to do just a little clean up. But that's about it.

Define 'metadata'

[inkswamp]

Because they have useless names like DSC0001.jpg and there's no metadata that says they are wedding photos.""

Doesn't storing your photos in hierarchical folders labeled appropriately count as metadata? I know it's not very flexible or powerful, but it's metadata of a sort. Store your wedding photos in a wedding folder in a photos folder.

Now, if you're talking about a database of metadata about files, then that's something else.

Apple's Solution

[JohnsonWax]

Apple has a solution to this, which has trade-offs, but seems pretty functional.

Essentially, each of their iLife apps is a replacement for the Finder. Do we really need music search integrated with file search? Or is it sufficient to build independent metadata (ID3) and filestructure (playlists) just for music. That's really the brilliance of iTunes in that it never takes you back to your HD filestructure. You can even ask it to maintain the HD filestructure to reflect the metadata structure, so it'll keep everything in an artist/album/song structure, naming things as needed.

iPhoto is set up the same way, but it's pretty apparent that the iPhoto guys are the 'B' team, since they haven't gotten it nearly as slick as iTunes yet, but it also has the equivalent of content metadata, playlists, and smart playlists. So, yes, I can easily find my wedding photos. The trade-off is that you can't search for 'Wedding' in the Finder and get wedding photos, wedding songs, etc. Maybe that's upcoming, but I'm not totally convinced of the value.

The iTunes organizational structure does carry into iPhoto, so if you want to select a song for a slideshow in iPhoto, you can see your iTunes playlists, and filter against metadata. It also carries into iMovie, etc.

Other posters have clearly identified the problems with metadata. File organization is generallly only useful if you are willing to symlink across all of your metadata, otherwise your photos of you mom and your wedding photos are disjoint, since some should be in both places. The single biggest problem with metadata is putting it in to begin with. iPhoto now allows you to do that during photo import - using a slide-show type UI.

I think MSs tendency to do everything in one place is interesting, but tends to not come off so well. Having everything in SQL could eliminate one of the shortcomings in Apple's implementation which is that they need to maintain an XML intermediate structure for music files, photos, etc. While somewhat handy, it's main function is to join file metadata and the FS, which means that it is somewhat fragile.

iPhoto - The Application Paradigm

[Slur]

I think it's cool that Microsoft is taking cues from the iApps - interesting that they want to integrate it so much into the operating system. Whereas so far Apple is stressing an application-centered solution on top of a more general-purpose filesystem, Microsoft is getting deeper into the integration game, getting into file metadata a la BeOS, and tracking files according to thematic relevance a la relational databases.

If the "smart desktop" idea catches on it will be interesting to see the response from developers on Mac OS X and Linux, as far as offering intelligent activity tracking. Somehow I see a twisty maze of documents and activities, all alike.

Should operating systems do all the work of organizing users files for them, concealing the filesystem behind a database veneer, or behind a purely task-oriented veneer? Should this kind of thing be left to application developers, like the maker of Path Finder?

Wouldn't Windows be more useful if it was a truly modular system that could be configured simply by stripping away unwanted components? Isn't that what makes Darwin so healthy in the enterprise market today?