Slashdot Mirror
WinFS' Spot on Back Burner Nothing New
osViews.com writes "Charles Arthur of Independant.co.uk has an interesting editorial which analyzes Microsoft's recently postponed 'WinFS,' the file system that Microsoft had been planning to implement in Longhorn. His editorial reminds us that this technology, previously referred to as the 'NT Object Filing System' was intended for a previous version of one of Microsoft's operating system's code named 'Cairo.' Microsoft first spoke of the 'NT Object Filing System' in 1992 and scheduled a beta release in 1996 and then a full release in 1997. But limitations cause it to continue being delayed."
From what I know of WinFS, it really won't be all that important anyway. It is supposed to provide a way for all files to be treated the same by the OS (roughly) right? Thus making it easier for users to search, browse, or otherwise find these files?
Well, I don't know all of the juicy details of WinFS but I have played with the new Longhorn build. The search tool that is in the Alpha release (MSDN) is much improved over the current WinXP search. It was pretty cool, although some of it can be chalked up to eye candy. It still had a certain ease of use to it.
I doubt WinFS will ever be complete, personally. But I am sure some of the innovation and development benefits will still reach us as consumers. I know where I work, we spend time doing things the customers will never see. But they will still reap many of the benefits.
I'm working on a object file system right now, and it's really not easy.
It's a simple concept:
Store on a standard journaled b-tree (or similar) filesystem the binary data, and store in a database all sorts of meta-information about the data. Also if you want, store a reverse index of the textual info and maybe another 'index' of image features if it's an image.
Then if you want to get anything, no need to go through the filesystem's tree, you can hit the DB indexes and get info instantly.
The real problem is keeping all of this in synch, with almost flawless atomic operations. (of course it's pretty much impossible to be flawlessly atomic, but one should come as close as the current journaled filesystems are).
So if you're using 2 components, let's say, a filesystem and a SQL database, then you need to open a SQL transaction, do your inserts/updates/deletes, then do the filesystem operation, then do the SQL transaction commit. If anything fails, you can revert the SQL modifications and everything goes back to normal. But if the filesystem has problems, then you can't keep the damn DB synchronized, and at some point you'll have to resynch both.
On 100k files, no problem. On 200MM files (what I'm aiming for), you're pretty much screwed. Then you have to start thinking of a self-healing system with a constantly-running checker that must ensure that it's very resource-efficient, etc...
It's just a huge problem. Supposedly Apple is solving this by Q1 2005, but I wouldn't be surprised if we see a massive increase in filesystem corruption bugs for a while on OS X (unless the DB indexing piece is just that, an indexer that runs x times a day and isn't atomically joined to the filesystem operations).
Let's put this in perspective. In '92 MS was looking at the Sybase source code and thinking about building a new filesystem around a database engine. Chicago AKA Win95 was almost out the door and it seemed reasonable to shoehorn this into Cairo (NT4). They were absolutely the dominant and fastest growing player.
I commented to a collegue in '93 (paraphrasing Robert Heinlein) that I did business with MS for the same reason I obeyed Newton's laws.
What happened around 1995? The internet became a commercial entity. Suddenly, MS needed to provide new applications (like IIS, IE, Outlook Express, an SMTP aware Exchange server, etc.) not just dork with cool OS technologies. A few years later, they are comfortable again after playing catch-up and start thinking about filesystems again, this time in "Longhorn". Again, they started talking about the capability two OS releases into the future.
However, this isn't a feature that is going to drive sales. MS needs to keep developers of home and office apps happy so they develop yet another new graphics system to replace DirectX. The perception of Windows security has never been lower and is starting to affect sales. IIS is losing ground again to Apache/Linux.
It's time to focus on revenue streams again and the revolutionary, expensive, difficult-to-build features get axed. It's probably not a bad idea. Think about the problems they've had with MS-SQL and ask yourself if you want a similar technology built into every teenager's game and grandmother's email box.
I used to work a lot with Lotus Notes, which is sorta a half-solution to the problem, and one that's been around for decades. (Notes, like the WWW, was based quite a bit on Ted Nelson's Xanadu idea.)
Notes basically gives you network-enabled document stores with indexed metadata and fulltext searching. The problem (other than the asstastic and totally broken UI) is that Notes doesn't integrate well with other software, either in exposing interfaces to users or pulling in random documents from the Internet or MS Office or whatever. Basically they pushed the hard problems back on the enduser, and Notes ended up as another island of data rather than a solution.
Anyway the idea was out there, and I think some people in MS understood it.
Microsoft, OTOH, is in the unique position to implement such an idea on the 'system' level and provide a transition plan for existing software. But it sounds like WinFS got beached because they still don't have real answers to the hard problems of pulling random data & metadata into such a system.
The other big issue for Microsoft is that they'd probably have to rewrite Outlook and Access for the thing to be effective. (I find it slightly funny that Outlook lacks even basic fulltext searching while MS was running their mouth off about WinFS.)
Business. Numbers. Money. People. Computer World.
The key here is this: I am not at all interested in a system that fundamentally assumes I am stupid. I will be utterly devoted to a system that fundamentally assumes I am lazy.
WinFS and masses of metadata assumes the stupid and not the lazy. The reason I don't want to have complicated trees of directories is that i am too damn lazy to do so and maintain it. Requiring me to add masses of metadata instead of a directory heirarchy does not address the problem: I am lazy!
Such a system will work well for limited uses - anything that has self populating metadata (such as music collections where files will either come with suitable metadata attached, or if I rip a CD I'll automatically attach suitable metadata via FreeDB or what have you. Similarly for a certain amount of video etc.
Such a system will work passingly well when you have a reasonable amount of attached metadata automatically, for instance email.
It won't work well for general user created documents and the like.
In the end a lot of data is purely user created - from speadsheets and letters to photos downloaded off digital cameras.
Find a way for me to be lazy and still have quick and easy access to all of those, and then you'll have my interest.
Jedidiah.
Craft Beer Programming T-shirts
I wrote the first design document for ReiserFS in 1984....
The nice thing about being slow in solving a hard problem is that others are also slow....