Slashdot Mirror
Examining Mac OS X 10.4's Spotlight
Ton writes "Apple has published a discussion of Spotlight, the radical systemwide search technology that will be part of Mac OS X 10.4 'Tiger'. The really interesting part is that metadata will be playing a big role in Spotlight while just a few years ago people were afraid metadata in Mac OS X was going the way of the dodo."
Can someone please explain a little more as to how Spotlight using metadata is a "radical" new thing?
I haven't seen any mainstream implementations (WinFS?) of it, but I didn't know it was a brand new concept.
Error 407 - No creative sig found
Mac OS X thread, not Mac OS 9 thread, silly.
You must have a different version of locate to me. I can't get mine to index my emails, it has no idea about the metadata entries in common document types and can't tell the difference between an image and a movie file.
Could you send me the source for the version you have installed that does that?
My windows XP search (at work) is very odd. It will not find text in assembly files (*.S) that I know is there. I've played around with turning the indexing thing on and off to no avail. That and other strange behaviour led me to find Visual Grep which is well worth whatever I paid for it (50 USD?). Still something like that should work in a real OS.
Nothing in the world is more dangerous than sincere ignorance and conscientious stupidity.
Anyone who has used the instantly updated searches in Mail.app or iTunes will have a feel for how useful a system-wide approach could be. However I too am concerned about resource usage. I think I'll wait and see how big the metadata index tends to get and how big the CPU/memory hit is.
I believe though that the indexing is done during saves, so you'll not notice a general system slow down. What you will notice is a slow down on file saves.
People who have used it report no performance degredation. And no, its nothing like Windows search, which Mac OS has also had since System 8 or earlier.
For one, it doesn't take half an hour, it shows you the results as you type, instantaneously.
Secondly, via plugins it can understand *any* file, such as an image metadata importer that uses OCR so you can search for words, or a Flesh-tone detector so you can search for all your porn that way.
Shitram Brown, PhD
Professor of Mathematics
From reading the article, I think Hans Reiser has been right about the need for reiser4 on mainstream linux.
He saw all this stuff comming from way back. If you read the LKML, you will remember that he warned us.
Its a pity no one listens to him.
makewhatis.cron can be a pain on Linux as well, if it is on a workstation which is mostly switched off.
Unfortunately for windows boxes, they do tend to be left shut down a lot of the time, so more of their runtime is spent rebuilding the search database when the machine is being used for something, rather than in the middle of the night, which is the preferred way
http://michaelsmith.id.au
The post links to the Apple Spotlight page that has been there for months. Is THIS the "discussion" that is being referred to in the post?
>>> "Apple has published a discussion of Spotlight, the radical systemwide search technology that will be part of Mac OS X 10.4 'Tiger'.
. html
What's really funny is that there's no link to the actual published discussion... but anyway...
http://developer.apple.com/macosx/tiger/spotlight
I read about beagle for linux it seems to be very similar in functionality. http://www.gnome.org/projects/beagle/
-- My site
Already the differences in Fat32/NTFS versus HFS+ (the mac filesystem) yield significantly faster searches before spotlight is introduced. Sit down on an OSX apple and notice that an entire search of the HD is actually a fast operation, not the waiting many-minute exercise that it is on windows.
Now since spotlight is built into the core of the system, and isn't just a tack-on service like the windows indexer is, there are significant speed advantages, updating the SQL database when files are modified, added, etc is incredibly light on the CPU, and is equivalent to doing something like changing the file name.
What spotlight isn't, and this might be where you are getting confused, spotlight isn't a spider that crawls from folder to folder cataloguing information about each file, which is what the windows indexer was doing, hence why it was resource intensive, as it was busy checking files and folders that you have possibly not made any changes to.
As a counter to the 'Filesystem metadata is great, but "instantly" updated search indexes sounds like a solution to a problem that doesn't really exist.' Microsoft, google and apple would disagree. Having an up-to-date catalogue without the CPU strain is a must have, go figure MS have been trying to implement it since NT4.0.
Apple are well known for optimising their software to be significantly faster with each pre-release build. Having had the opportunity to test the developer tester of 10.4 with spotlight on a 12" powerbook (which was bogged down with various applications at the time) I can assure you that spotlight remained snappy, and definitely true to the 'instant' claim (I've noticed apple are quite careful on not over advertising their products, as it cause more problems than sales and a bad image). After using microsoft products we become very used to how slow a process can be. Apple's advantage is clear, they know their target hardware, like video-card driver writers they can optimise any part of their OS to fit their hardware for optimum speed. Additionally the g4/g5 chipsets have some quite useful registers for performing these sorts of searches (think sort of like MMX for x86, except with developers actually utilising them outside of games)
Try to pretend that you're managing 2 or 3 or more major projects that can change or be passed along to someone else every few months with mails, im's, files, reports you don't look at, media submitted by other people in different countries, to-do lists and other project management data...
Now imagine someone asks you, the project manager (or just the last person still around) on a project from 3 years ago, what the initial proposal from that guy in japan who did the Flash files was versus what we paid him and what the VP's said about that....
People *will* have copies of these files still floating around *somewhere* in e-mail or im history, at least. You may not, I may not, but that's where this will come in handy.
A few years ago, hd space was not large enough to think that you'd keep all that data around, but gmail's new 1Gb e-mail storage just showcases the lack of a need to dump all that crap off your media if you can just organize it well, and who needs that when you can keyword search, anyway?
I'm waiting for Tiger so that I can try out Automator. This promises to be a point-n-click version of scripting. Hopefully this will be easy enough to use even my parents and maybe even my boss will be able to use it.
The first thing I'll do is try making an Automator to create thumbnails. Currently I'm using a bash script I wrote on my Linux box to do this. This will be the first time I've paid for an OS upgrade since Win98, so I hope it's worth it.
Vote for global prefs bug
Just a small info. The brain behind Spotlight is Dominic Giampaolo, the same guru that wrote the fantastic BeFS for BeOS.
uhm. No. It is not continually indexing the data, if you read the article you'll see it only updates the meta-data for items when they're saved - you can write custom plug-ins for new data types, or just go with the bundles ones for standard file types like images, text etc.
:
:
:
Filesystem metadata is great, but "instantly" updated search indexes sounds like a solution to a problem that doesn't really exist.
On the contrary, this is a *better* solution to a very basic problem that has plagued computers since they were invented.
The problem
How do I organise and access the data I use every day (emails, letters, images, music etc)?
The old solution
You can put your files in folders (one per file). You can name the files with a short description, ending with a cryptic 3 letter code to denote the file type. Files *must* be in one category/folder only at a time. Limited meta-data (date modified, file-type etc) may be stored.
The new solution
You add meta-data to files (often automatically) saying who created them, what project it's under, whether it's 'to do' or 'unfinished' or whatever. You'd do this in a save dialog for the application, as you saved the file. All other applications which use searchlight will update their view of this stuff for free, in real time.
When you want to work on a project, you click on the live project folder, and immediately you see all the files, emails, images etc for that project, no more, no less, regardless of where they are on the disk and what other projects they're shared with.
Want to see all the stuff to do with John, 5 months ago? On this project? Containing the word gizmo? That sort of query will be easy to make.
If you have an image editing application, it can show you all the images taken in Paris in 2002, without having to build a database application into it. This makes adding this kind of feature to applications trivial.
Ideally adding meta-data tags like 'project-1', and 'To do' should be as easy as choosing them in the save dialog or applying them like a label in the Finder. It's not quite at that stage yet, but that should come later. Some of these ideas are quite old (Be), but they are long overdue in a desktop operating system.
The Both links say quite a bit. I guess the kernel gurus know better, but i think the sql plugin for a FS would be a cool thing to show off with at the very least.
n .net/Articles/100148/
http://kerneltrap.org/node/view/3727
http://lw
Coming from a WindowsXP background, some things Ive noticed so far:
- Clicking the 'X' doesnt actually close the application. This annoyed me to start with, but ive slowly gotton used to it.
- Having to select the application window before I can quit it using the application menu. Or I have to right click on the dock icon to quit. Annoying still.
- Love the dock. Its just
..... right.
- Most of the file system is hidden from you, which I like. Put my data where I want it and ignore the rest.
- The ability to access the underlying BSD OS easily. Love it.
- Everything looks and feels 'polished'. THats what I always hated about KDE/Gnome when I tried them, the features were there, but noone had taken the time to step back and polish the entire thing off so it all looks and feels together.
- Every time I boot the Mac, my TFT display is 'wavey' until i have the monitor do an autoadjust. Dont really know whoes fault this is, tho its fine under windows and linux.
So, final conclusion? I love it, so much that I have already placed an order for a G5 Imac. And in the meantime, Ive purchased a G4 upgrade for this little baby, just to help it alongI'm a PC (Win/Lin) user, and I'm thinking about changing over to Mac.... lol, I'm not that cliche. But I might consider learning more about them. They are nice powerful beasts within. They'd be nice to have on a Folding Farm. :D
The reason Windows XP does not do full text search correctly is because it uses a specific registry handler entry for each type of file (*.txt, *.rtf etc). It uses a different handler for different types of files.
; EN-US;Q309173
.TXT flat text handler is identified by using a registry key:
{ 5e941d80-bf96-11cd-b579-08002b30bfeb}"
.ASP place
{ 5e941d80-bf96-11cd-b579-08002b30bfeb}"
However it only comes with a few configured filetypes settings, and no way to set a default "When no searchFilter available, treat as plain text" setting.
I stressed and strained about this when XP came out initially. The only way I found to do it so I got expected results was to build myself a scanner.
It searched through a drive, and identifies EVERY file extension.
It then looks through the registry to see which Extensions have linked Handlers.
It generates a reg file containing stub links for every unmatched filetype.
Its a bit shotgun, but allowed me to continue using the Text search for XP.
Microsoft have released their own shotgun registry pack, for more info see here:
http://support.microsoft.com/default.aspx?scid=kb
(I have since moved myself into using my own full search tool, but at least the XP search doesn't miss files which are clearly within visible range).
[Now for the science part..]
Take a file, something like "PunchTheMonkey.asp".
Make sure you have it open in notepad, and make sure there is a certain text string - for instance "spyware".
Open the windows XP search in that folder, tell it to search *.ASP, and give it the phrase "spyware".
Windows XP will NOT find this file.
-----
The Windows
[HKEY_CLASSES_ROOT\.txt\PersistentHandler]
@="
Adding an entry like the one above for each required filetype will restore the full text search functionality.
So, I add the following entry into the correct
[HKEY_CLASSES_ROOT\.ASP\PersistentHandler]
@="
After I have logged off/rebooted, I try the same again, and XP will now identify the file.
liqbase
Probably very similar to Search Kit which currently does the same thing, but has to be manually set up. You can choose the type of index it creates, inverted, vector or both together.
What if I want to find files from male colleagues?
Unless you used BeOS in the past!
This really is a big deal, much bigger than Microsoft's feeble attempts at full text search, or Google's desktop search. In many way's this much, much more useful than full-text search, especially for developers.
At home I have about 6,000 MP3s, a 1000 photos, 500 scientific articles in PDF format and hundreds of words files that I need to juggle. Each one has its own metadata database, and none of them are updated in real time.
Databases:
MP3 - WinAmp & AudioTron
Photos - Photoshop
PDFs - Acrobat Indexer
Word files - MS Indexer
That doesn't include any of the other data that is stored completely databases and would have been easier to store in the file system - like email, guitar tab files and god knows what else.
A properly implemented global meta-data store (that works at the filesystem level, not as an iterative service) profoundly changes how one uses the system, making sorting and finding data actually almost pleasurable.
+--------------------- You idiot! I told you we were facing the wrong way!
This has already been done to some extent in Quicksilver.
/.
http://quicksilver.blacktree.com/
It's an app that indexes parts of your file system and supports plugins to to index application data. The best part is that it is keyboard based. For example. type command-space "slash" enter and it fires off Safari opening
I'm not sure how Apple will improve on this.
The difference between Canada and the USA is that in Canada healthcare is a right and gun ownership is a privilege.
What's up with apple and German tanks? First the Panther (http://www.achtungpanzer.com/pz4.htm#panther) and now the Tiger (http://www.achtungpanzer.com/tigerp.htm). What's next, the Leopard? When apple releases Mac OS 1x.x Leopard II, then I'm buying a Macintosh!
The brain behind Spotlight is Dominic Giampaolo, the same guru that wrote the fantastic BeFS for BeOS.
Which explains why it's tied to the filesystem rather than using a general hook at the vnode layer to allow the same functionality to be implemented regardless of the filesystem in use. Having the filesystem support it would make it more efficient on HFS+ but it should be possible on UFS, ISO 9660 CDs, or even over NFS or SMB.
In fact, the way it's described... with one metadata store per filesystem rather than per file, and user-level metadata provided by applications... this is something that FreeBSD or Linux could implement right now, over any file system: all they would need would be a mechanism for the vnode layer to send messages to a usermode daemon that tracked inode operations (eg, creation, deletion, maybe mode changes or date changes, and renames) in a name-inode database (any database, including Postgres or MySQL) and updated any associated metadata in the background.
This could be done with negligable slowdown for file operations: the index can be updated asynchronously, because it can always be recreated in the background after a crash, so the vnode operation won't ever have to wait for the daemon to respond... and changes to the metadata are all in userspace.
Devon
What hasn't been mentioned is the smart folders will always keep you directories uptodate. No more drag and droping files after I download them.
The question is will I be able to make smart folders based on permissions I give on my files so that I can share them on my network.
weo
#=-weo-=#
Quicksilver is a pretty nifty program, and I used to use it as a free alternative to Launchbar (which used to perform roughly the same tasks). Both programs learn what you want certain shortcuts to mean, and both use Command-Space to activate them. For me, entering 'FX' is Firefox, 'PS' is photoshop, and so on.
However Launchbar has since updated to 4.0 beta release, and in doing so has pre-empted spotlight, as it does (right now, in 10.3) index system-wide metadata. So now you can cue up songs by entering MP3 names, open any kind of files by entering keywords for filename or type, open websites, perform google searches,Google image searches and so on.
It's worth trying out as an alternative to Quicksilver.
stat file.jpg
in linux. Would be nice in linux to beef up on metadata too.
I hope that spotlight will work also, if you have a linux partition exported to the Mac via NFS. Will file information of NFS mounted systems also stored in the database?
Having linux and OS X working together is already now not without issues. If you have a file Test.jpg and test.jpg in your Linux partition and you copy both to the same place in OSX, the finder (on the mac) complains, because the two files are considered the same.
Check out Mor Naaman at Stanford who is working on adding GPS metadata to photographs. Once he has the GPS coordinates he uses that to get information such at time of day, lighting, weather, elevation, temperature, etc... This allows you to create metadata searches for "All early morning images in clear weather in Las Vegas, etc..."
YOu can try the system out here with a collection of almost 4k images.
Apple has had this type of search engine before, they called it V Twin and it was a basic part of Copland. This is what Sherlock used in Classic and why it was so fast. The idea is even older, it's from a conceptual computer interface Apple dubbed the Knowledge Navigator. All this appears to be is V Twin running on SQLite instead of a proprietary method.
The interesting part to me is the focus on metadata. I loved this feature in BFS that metadata was king. This is going to lead the way to better file management. Hopefully the Finder will integrate it.
Not since Marie-Antoinette played milkmaid has looking simple and honest been so fake and complicated.
it handles pdfs! Yippee!
-- I speak only for myself
Documents are indexed as files are saved. The performance hit is during document saving. There is no need for "background indexing".
Apps need to be made "Spotlight-aware" in order to invoke the Spotlight indexing on save.
I've tried Spotlight and suggest that when it comes out, every time you step away from your computer make sure to lock your screen. All someone has to do is type 'porn' into the little search toolbar and within seconds it's all nicely listed.
Perhaps Apple needs to add a feature to turn off indexing for certain directories.
And without a proper search tool, how is it, exactly, that we're supposed to keep track of our fellow insurgents, plans for sneaky attacks, plots to undermine the "powers that be" and means of crippling the status quo?
;)
A disorganized revolution is just a waste of time.
Keeping one's data organized is a priority, bucko.
#SickNotWeak
I suggest the value of search depends on what you use your computer for and maybe how good your memory is. To me it is tremendously important. Most of my job is spent managing information. Searching for things and adding things to my knowledge base. I love Google's Desktop search for this reason. I also use blinkx, but prefer the google UI implementation. Months or years after something was written, I need to go find it. I may not know if it was ppt, xls, doc, pdf, or xml. I may not have written. I may not have even read it, but I need to find it.
;-).
Increasingly I find that mult-media files matter. MP3 recordings of meetings, images of whiteboards, videos of presentatons are all fair game. (My hobbies include photography and genealogy. So findings pictures of people and places, correlated with GPS tracklogs is also of interest.)
I also find that, although I structure my file system according to something that makes sense initially, it won't be the way I want to search for it later. I usually file things accordning to client and project. But later I may need to 'find all system specifications where the DRP recovery time requirement was longer than 30 minutes'. This kind of search would require a lot of my time.
I need way better search, way better meta-data (which means system created because people don't), and more disk space
When I first saw the spotlight demo, I was thrilled. A very small step to help me with my job.
Does anyone know how they are going to deal with security? Will the indexed information inherit the same security attributes as the underlying files? Do the indexers run as root?
Hmmm. This sounds a little dangerous. It will make it much too easy for the wife to find your porn collection, your AOL-IM sessions with that weird Goth chick, the draft of your divorce papers, etc. I AM NOT UPGRADING TO TIGER.
How well this system works will in part depend upon how many data format plug-ins are provided. For example, take something like the SID audio format. It's relatively unknown, but has an officially registered MIME type with IANA giving it a status above many other file format types, and it is used to provide background sounds on some web sites. Will it make the cut?
This is just one file format chosen at random. There are thousands out there, some of which are used pretty heavily for documentation in certain circles. How about all of the OpenOffice file formats, or the AbiWord format?
I can see this feature being hugely useful if Apple does a good job of providing plug-ins, and making it easy for third-parties to add more.
"...how often does one do a blind search of the whole system anyway?" well, you've got to realize that this will be the big topic for the next boring couple of years. even google, not to mention apple, MS and every 'nix flavor are working on solutions. managing your information.
;)
i have to admit that i have crapola all over my harddrive that i will never go back to -- the files just keep getting buried and copied over to my newest computer. even if spotlight is kinda flawed, engineers have to start looking for better ways to manage information.
and besides, it gives MS something to do besides f-ing up browser standards
The kind of metadata that was almost deprecated by Apple isn't quite same thing as the "modern" concept of metadata. The classical HFS metadata covered concepts like file type, file creator, and "Finder bits" that aren't handled at the file system level in other OSes. This, combined, with the Mac OS's historical use of resource forks for storing developer defined data records, made perserving such data difficult or impossible in heterogenous environments like the Internet. It's really a shame; I've always thought this concept was the most elegant attempt to solve the problem of "rich data" associated with data files without requiring the data in the file itself to have some form of universal container format.
The metadata concept used by Spotlight is going to be based in part on a plug-in system that allows the Mac OS to reconstruct metadata information from the data within files themselves, rather than just using the metadata facilities provided by HFS and Mac OS resource forks. That means that each different kind of file, from Word documents to PDFs to Postscript jobs, needs its own special kind of processing to read its own format of storing such data. It's less elegant and more processor intensive that just using the historical HFS system, but it's more likely to to be useful for extracting metadata from files provided by Windows and other Unix variant users.
Those who complain about affect & effect on
I'm not convinced yet apple is going to get Spotlight right, i.e. truely revolutionary. It has potential (smart Finder folders is on the right path) but at the moment, it seems they are more interested in simply trying to duplicate Quicksilver/Launchbar technology, which is the wrong way to do this.
I'm tired of apple ripping off ideas from developers without (A) Giving them credit or (B) developing something equivalent so the new as at least as feature-full as the old. Based on apple's history, the first version of Spotlight will likely be a horribly dumbed down version of Launchbar in terms of tech, since apple is obsessed with "ease of use": i.e. a three year old has to be able to work it.
Rant aside, there are a few key pieces I think apple is missing:
(1) User-created metadata. I should be able to tag anything I want with any metadata I want so the organization system follows ME and MY preferences, instead of the system determining it for me. Apple should be thinking about taking the insanely wonderful metadata system they created in iTunes and applying that to the finder. It is essential you be able to tag metadata in, because you don't always access the same objects for the same purposes.
(2) Flexible file system. This is a concept I've developed which basically says that the file system should be dynamic and adaptable to match the thought flow of the user (only possible with a good metadata file system). If you've ever seen this app on the PC, think: "The Brain". What that means is that if apple does #(2) right, it should be easy as hell to tag things, and then basically I can create relationships which let me "flow" through my files by navigating CONCEPTS instead of folder heirarchy. A good app that does this is Devonthink. Devonthink will grab the contents out of your files, and when you do a search, you can not only see your search term but "related" search terms. Click on a new search term and you get a new listing. So as you come up with ideas about what you want to do, you can easily and naturally branch off into other parts of your file system. This methodology models the way the human brain actually works- thinking in concepts and spacial organization, rather then structure. (The "flexible" comes because the system takes your tags and adapts the search around them, allowing you to change how the "flow" works, depending upon what topics are most important to you.)
(3) The next level after metadata search is a new way of visually interpreting the metadata and relationships between. Which means a NEW FINDER. I can't believe Steve actually threw this comment out after demoing Spotlight: "With this, you probably won't even need to use the finder any more." Well then why even have the Finder at all, Steve?! There IS a reason for the finder, which is why it's stayed around all these years, and that is that people think SPACIALLY. People are creatures of habit, and one way we remember where things are is if we know where to look for it and it's always in the same place. Which means there needs to be a visual grounding to the above dynamic files system, to give people a sure footing to all of this. I'm talking about things like a window that always stays in the same spot and always performs the same task, like showing you what new files have been added to the system, or actively updating your list of word documents wherever they are. Right now in the finder, a window is a window is a window. That shouldn't be. If a search is applied to a window, then that window isn't just showing you files, it's performing an active function. The finder needs to evolve to take on the new roles and responsiblities it should have in the context of a metadata files system. Spotlight should replace the finder: the two should work together seamlessly.
The good news is that Spotlight is built into the system, so even if apple screws up the implimentation (likely), the next generation of 3rd party apps will hopefully be able to fill in the gaps.
Filesystem metadata is great, but "instantly" updated search indexes sounds like a solution to a problem that doesn't really exist.
Doesn't exist *for you* perhaps. Perhaps you don't have a lots of user data, or you have taken time to sort it into useful folders. I'd say it's about as useful as the incremental seach in iTunes is. Sure I could remember what artist did a track, and access a track by scrolling down to that artist, then finding the track. Or I could scroll down the list of thousands of track names, remembering my alphabet ordering, and locate the track that way. Assuming I've remembered the exact wording of track name. But I've always found it easier to type whatever word comes to mind first from artist or track into the search box.
And so it is with documents. Even if I do remember the file name and folder that a particular piece of information is stored in, I still need to navigate there. Most times it will be quicker just to type in whatever it is you remember about the data you want into a search box - even if you know where the data is stored.
This sounds like an attempt by Apple to do on HFS what they've done for years on the Newton, er, *did* for years on the Newton. On the Newt there is no file system: there's only a database system, and each application maintains its own database of entries. When you issue a search, the operating system queries each of the applications in turn, asking them to search their entries in an appropriate fashion looking for a particular string or whatnot. Then it assembles the entries and the user can choose them and launch the application opening the entry. Nice.
On the Mac, that'd be expensive. Querying all the apps means running the apps. So instead Apple has lightweight app proxies (the "plugins") which provide metadata information rather than directly searching the files. Blah.
I'm very fuzzy on the details, but I know that Apple played a leadership role, back in the mid-90s, in lobbying the FCC for the radio spectrum allocations for what we now call WiFi.
Spotlight can support arbitrary file types, entirely dependant on what an application developer decides to supply, and you decide to install. Google is limited to the file types Google implements.
WinFS is an overly complicated pile of steaming pooh, that Microsoft are having trouble delivering.
Either until you code it up, or you buy a Mac next year?
GPL Deconstructed
The technologies are barely related; Apple is not ripping off QS/LB in the least here. Spotlight is a technology for searching through files based on their conent and metadata. QS/LB are utilities for finding files based on easily typed mnemonics. You are looking at one aspect of Spotlights appearance (the dropdown search pane in the corner) and assuming it's a ripoff based on some similarity to the appearance of the other utilities.
In fact, the Spotlight indexing technology will be a boon to the utilities, as they will be able to leverage this newly available metadata to execute even more powerful searches. Quicksilver is already invaluable to me, and I expect it to just get better.
Yes. System Preferences > Keyboard & Mouse > Keyboard Shortcuts. Check the box marked "Turn on full keyboard access".
This allows you to tab between gui elements. Ctrl-F2 activates the menus for keyboard access. And you can edit shortcuts for every application you have.
Veteran, Bermuda Triangle Expeditionary Force, 1992-1951
"As someone replied earlier, this is a new paradigm in app management: the top menu controls the application, and the window menu controls the window."
Actually, this behavior is not a new paradigm as it has been a feature of the Mac OS back before it was Mac OS -- all the way back to The Beginning.
There are a few reasons for this behavior, but the most important one is that in good UI design, each widget should serve a clear purpose. On a Mac, the "close window" widget closes windows and that's it (unless the app has only one possible window). Aside from making the app appear to "launch" faster, it's a cleaner UI implementation that leaves little room for ambiguity. Plus, lets say you are downloading a big file in your web browser but don't want the display or the Dock cluttered with windows. On the Mac, you can close all the windows but still not quit the browser and keep the download active and out of sight.
Per Square Mile, a blog about density
AFAIK they use some kind of search on the lexicon for the inverted index. For instance, the string "nut" is matched to "nutmeg", "donut", etc., and the document lists for those terms are merged together. Phrase search would also be done using all matching words, eg "nut hol" would expand to phrase searches like "donut hole", "peanut holder", etc.
The exact method for matching the search string to the lexicon isn't clear. It could be a suffix tree, but it may be as simple as grep-like scanning of the words, since there aren't that many relative to the text size.
Looking at mail.app it seems to do this process on each keystroke. It's not terribly fast, but it gets the job done.
---- "If we have to go on with these damned quantum jumps, then I'm sorry that I ever got involved" - Erwin Schrodinger
So now Apple's given my wife a way to INSTANTLY find all my porn.
I guess I now have to go back to a "download as needed then delete" paradigm.
Sheesh, I wish they'd think these things through.
However, from what I've seen, that's not the sort of thing Spotlight is about. The plugins we're talking about make use of intrinsic metadata - information extracted from the datastream itself. Many common file types include some descriptive information: EXIF data in pictures, MP3 tags in audio files, meta tags in HTML files, and so on. Spotlight is a way of extracting and using that data.
The practical differences include, OTTOMH:
- Spotlight's information won't be lost when files get stored on other file systems, sent over email, processed on other platforms, &c.
- Spotlight uses information that's already in the files - you won't have to set it up manually.
- You can use existing tools to see and edit the metadata - MP3 taggers, photo editors, whatever. And you can do so on any machine and OS.
This is probably one of those rare cases when that foul word 'leverage' might be appropriate -- Spotlight should allow you to make much better use of an existing resource. As such, it sounds like a jolly neat idea!Ceterum censeo subscriptionem esse delendam.
There's been reference from the beginning of the computer revolution to this solution we've all been waiting for... and credit to evolutionary steps taken by apps such as,Quicksilver, Launchbar, BeOS, etc... but one application that predates AND which most closely matches the feature set is:
Simson Garfinkle's "Sbook.app" from NeXT in the 90's.
The usefulness of Sbook.app ability to add tokens in a flat file for instantaneous searches enabled people to apply Sbook.app outside its realm of address book that it originally was designed.
Abstracting its functionality and interoperating at the kernel level is pure Apple polish on the brand. Until people start using "Spotlight", the verdict will be out on adoption across the platform.
I will venture it will be one of the defining characteristics of the Mac platform into the future.
Of course, this metadata will be so much cooler when something like spotlight is there to take advantage of it...
Die Menschen verhoehnen was sie nicht verstehen. -- Goethe.
While schematized semi-structured DAGs of data may be overkill for many applications, you might be surprised how often something like this is needed, and how few developers actually have the skill to build it when it is necessary.
It is not uncommon for Windows developers to use a Jet database as their "file format", and just rename the extension to something else. Right off the top of my head I can think of three [1,2,3] apps that do this. CityDesk and ContentSaver would both be much better served by something like WinFS, as their data are not particularly relational in nature. Jet is also not easily fulltext searchable, doesn't give you eventing, is not scalable past 2GB...
The team behind Chandler (Mitch Kapor et al) have probably spent at least a man-year or two working on a repository with similar features to those intended for WinFS. From what I've heard, it's a nice piece of work, and they're hoping other developers will use it (i.e. not just for Chandler).
I myself spent much of last year working on a similar repository for version 1.0 of my company's application. It was an expensive task, but the result was well worth it, as our 2.0 product adds very different functionality and yet was easily built on the same storage foundation.
You can bet many others have tackled subsets of WinFS functionality for their applications. (Sleepycat's customer list would probably lead you to many of them.) The problem with everyone doing this on their own is not only duplication of effort, but it essentially closes the door on interoperability, since each implementation is in effect another proprietary file format. Not to mention that some of these problems are truly subtle and difficult, such as allowing concurrent access to sub-file-level items (fine-grained locks), replication and synchronization, etc.
[1] Diebold GEMS - http://www.diebold.com/dieboldes/GEMS.htm
[2] Fog Creek CityDesk - http://www.fogcreek.com/CityDesk/index.html
[3] Macropool ContentSaver - http://www.macropool.com/en/index.html
Speaking as someone who married a girl geek, I've had to find workarounds for this set of annoying situations already. She's crafty and won't fall for the 'put the stuff in /etc/' trick so that my hypothethical goth and asian schoolgirl porn won't show via a normal search.
Solution?
Save the porn / super personal stuff on an encrypted disk image saved somewhere inconspicuous, and set cronned (or logout) scripts to scrub your various histories and recent items. Make sure that the machine logs you out after no more than 10 minutes of activity.
Hypothetically, that is.
Hi sweety!
Does anyone know how this will work with Backups/Restores? OS X backup programs have enough problems with resource files, yet alone this additional data.
._ prefix. Will the metadata be useable on an NFS mounted filesystem.
Also, how about remote file systems (nfs for example). Resource files are mapped as regular files with a
I just stick all my porn in a disc image [encrypted]. It's very good, except when I have to resize it.
"[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz
Sure search engines are killer apps for the Internet but that's because the web is intrinsically disorganised and distributed.
Is search really so relevant for a single computer and the average desktop user? Most people already organise their files in a somewhat structured way, and generally know where to find stuff. (Especially if they use OS X)
Sure powerful file search might be useful occasionally, but i don't see it as a huge issue that companies like M$ think it is.
If you have a mac with a ton of files, various "Previous System Folders" etc...follow along :)
I have smart folders for pdfs, avis, mpgs, and wmvs
I have these sorts of files *all over the place*...movie clips, test files, you name it.
I go to the finder, "open" the Windows Media Files folder, and they are all "there"
Or all the "archive" files (zip, rar, sit/sitx etc) i've collected and not erased in the last year...
or all of the emails i've received from japanese users...
it goes on and on.
To me, its like the whole star trek "Computer..find all of the blah blah blah for sector Whatever"
It concentrates on the "what you want" as opposed to the current paradigm of where did i pit it/what app did i use, etc
In Mac OS X Panther, I hear fast-user switching is a boon for this type of illicit activity. Create a separate user, say, "batman", with no admin rights, turn-on file vault for that specific user to make sure everything gets encrypted. When the urge comes ... fast-user-switch, do thy bidding, fast user-switch-back ... lah lah lah.
Extraordinary Vacations. Exceptional Prices
you could just divorce the bitches and get together with someone who doesn't mind you looking at porn.
I have such a big collection of porn that it doesn't fit on any one computer. and I run my own porn site and she doesn't care.
Two things that will always stay in my life before my woman: porn and comics. and computer gear. and other gadgets. ok, four. she's number 5. oo. that doesn't sound too good does it? ah, if she finds out, she'll leave me, and I'll be with someone else until they figure it out.
either way -- the porn stays.
have I gone off topic here?
obnote: if my computer makes it easier for me to find my porn, I'm happy. if she gets upset that she finds porn on my computer because of advanced metadata, then she can buy her own damn machine with her own damn data.
The reason girls and Windows users don't understand UNIX is because all the documentation is in Man files.