Slashdot Mirror
WinFS' Spot on Back Burner Nothing New
osViews.com writes "Charles Arthur of Independant.co.uk has an interesting editorial which analyzes Microsoft's recently postponed 'WinFS,' the file system that Microsoft had been planning to implement in Longhorn. His editorial reminds us that this technology, previously referred to as the 'NT Object Filing System' was intended for a previous version of one of Microsoft's operating system's code named 'Cairo.' Microsoft first spoke of the 'NT Object Filing System' in 1992 and scheduled a beta release in 1996 and then a full release in 1997. But limitations cause it to continue being delayed."
...is a solution in search of a problem?
It reminds me of the old saying
Or for that matter the ORIGINAL goal of the Gnu project?
What's your point here? Why are you trying to bash Microsoft just because they decided to delay or abandon something?
Best Buy can have you arrested
duke nukem forever and winfs are fighting for the throne... of... stupid delays
http://ipod.fresh27.net/
From what I know of WinFS, it really won't be all that important anyway. It is supposed to provide a way for all files to be treated the same by the OS (roughly) right? Thus making it easier for users to search, browse, or otherwise find these files?
Well, I don't know all of the juicy details of WinFS but I have played with the new Longhorn build. The search tool that is in the Alpha release (MSDN) is much improved over the current WinXP search. It was pretty cool, although some of it can be chalked up to eye candy. It still had a certain ease of use to it.
I doubt WinFS will ever be complete, personally. But I am sure some of the innovation and development benefits will still reach us as consumers. I know where I work, we spend time doing things the customers will never see. But they will still reap many of the benefits.
I'm working on a object file system right now, and it's really not easy.
It's a simple concept:
Store on a standard journaled b-tree (or similar) filesystem the binary data, and store in a database all sorts of meta-information about the data. Also if you want, store a reverse index of the textual info and maybe another 'index' of image features if it's an image.
Then if you want to get anything, no need to go through the filesystem's tree, you can hit the DB indexes and get info instantly.
The real problem is keeping all of this in synch, with almost flawless atomic operations. (of course it's pretty much impossible to be flawlessly atomic, but one should come as close as the current journaled filesystems are).
So if you're using 2 components, let's say, a filesystem and a SQL database, then you need to open a SQL transaction, do your inserts/updates/deletes, then do the filesystem operation, then do the SQL transaction commit. If anything fails, you can revert the SQL modifications and everything goes back to normal. But if the filesystem has problems, then you can't keep the damn DB synchronized, and at some point you'll have to resynch both.
On 100k files, no problem. On 200MM files (what I'm aiming for), you're pretty much screwed. Then you have to start thinking of a self-healing system with a constantly-running checker that must ensure that it's very resource-efficient, etc...
It's just a huge problem. Supposedly Apple is solving this by Q1 2005, but I wouldn't be surprised if we see a massive increase in filesystem corruption bugs for a while on OS X (unless the DB indexing piece is just that, an indexer that runs x times a day and isn't atomically joined to the filesystem operations).
Storage would be one example. I bet there are others.
Trust the Computer. The Computer is your friend.
I'll pull out the link again: Storage (a GNOME project) uses some nice algorithms to let you look up anything from '1960s music' or 'films directed by Francis Ford Coppola' to 'pdfs from joe'. All in natural language and over a wide range of formats, although evidently it's still a work in progress.
Let's put this in perspective. In '92 MS was looking at the Sybase source code and thinking about building a new filesystem around a database engine. Chicago AKA Win95 was almost out the door and it seemed reasonable to shoehorn this into Cairo (NT4). They were absolutely the dominant and fastest growing player.
I commented to a collegue in '93 (paraphrasing Robert Heinlein) that I did business with MS for the same reason I obeyed Newton's laws.
What happened around 1995? The internet became a commercial entity. Suddenly, MS needed to provide new applications (like IIS, IE, Outlook Express, an SMTP aware Exchange server, etc.) not just dork with cool OS technologies. A few years later, they are comfortable again after playing catch-up and start thinking about filesystems again, this time in "Longhorn". Again, they started talking about the capability two OS releases into the future.
However, this isn't a feature that is going to drive sales. MS needs to keep developers of home and office apps happy so they develop yet another new graphics system to replace DirectX. The perception of Windows security has never been lower and is starting to affect sales. IIS is losing ground again to Apache/Linux.
It's time to focus on revenue streams again and the revolutionary, expensive, difficult-to-build features get axed. It's probably not a bad idea. Think about the problems they've had with MS-SQL and ask yourself if you want a similar technology built into every teenager's game and grandmother's email box.
ReiserFS version 4 is a database at heart. Its basic structure is just a table of FileName | Binary but it also contains a modular system where it can be expanded for many uses. There is a lot of talk of including meta data in ReiserFS for such a system.
http://www.namesys.com/whitepaper.html
I used to work a lot with Lotus Notes, which is sorta a half-solution to the problem, and one that's been around for decades. (Notes, like the WWW, was based quite a bit on Ted Nelson's Xanadu idea.)
Notes basically gives you network-enabled document stores with indexed metadata and fulltext searching. The problem (other than the asstastic and totally broken UI) is that Notes doesn't integrate well with other software, either in exposing interfaces to users or pulling in random documents from the Internet or MS Office or whatever. Basically they pushed the hard problems back on the enduser, and Notes ended up as another island of data rather than a solution.
Anyway the idea was out there, and I think some people in MS understood it.
Microsoft, OTOH, is in the unique position to implement such an idea on the 'system' level and provide a transition plan for existing software. But it sounds like WinFS got beached because they still don't have real answers to the hard problems of pulling random data & metadata into such a system.
The other big issue for Microsoft is that they'd probably have to rewrite Outlook and Access for the thing to be effective. (I find it slightly funny that Outlook lacks even basic fulltext searching while MS was running their mouth off about WinFS.)
Business. Numbers. Money. People. Computer World.
There's a system that exists already and that's not vaporware. ReiserFS 4.
You can "cd" into a file like a directory and see the metadata. Things like bitrate for MP3, and all that stuff.
SQL doesn't fit that well with filesystems, btw. Relational databases work great with rigid categories. But beyond very rudimentary classification it won't work well because everybody has their own idea of what a good classification should look like.
Although it makes a nice tagline and dig in the ribs for Microsoft -- same delayed technology, different century, yuck, yuck -- the Cairo Object File System (OFS) and WinFS bear no resemblance to one another. Having worked in the Cairo/NT group at the tail end of the former and suffered through uncountable meetings about the goals/architecture/benefits of the latter prior to leaving MS, I can say this with some certainty. Saying they're the same internally or architecturally because both strive(d) to provide the ability to find any document by any properties or content (aka "information at your fingertips"... remember that?) is just vacuous -- you might as well talk about similarities between file-systems that support shell wildcard expansions * and ?.
OFS was about a lot of things, probably too many things. It was designed during the "object wars" and things like copeland and pink and opendoc were in the headlines. Document-centered work was the proposed user paradigm, where structured documents contained nested opaque data from many different applications, and so applications wouldn't need or want to know the difference between a top-level document or a sub-part of a document. This user paradigm did not entirely come to pass, and so an entire file and object-system architecture and shell user-experience premised on it was canned.
That said, a few features from "OFS" did survive into NT/XP, including:
From what I saw to date, WinFS seemed to be about the data/XML paradigm of data format transparency, not about opaque nested/contained data like OFS. It seems to be pursuing a different usage paradigm. At least I think so.
It's a confusing thing, and it shouldn't be. The basic idea of fusing a DB and a FS is dead simple, and if every OS offered structured and unstructured data, a set of simple core schemas, federated query across the two forms of data, and transactional/ACID cross-references between them, you could build many applications more easily. Why WinFS keeps taking so many more bits to describe itself than this is beyond me.
n@
Well, actually Reiser6 is vaporware, and Reiser4 is just the first storage layer that is really suitable for supporting adding database and search engine functionality into the filesystem. I grant you, it puts us years ahead of where MS is, but MS has much more funding, and our guys are spending at least half their time dealing with grubbing for money instead of coding.
Oh, and, let us not forget that there is a definite lack of political support for our work in the kernel community (especially among the other FS developers, ahem), and implementing the semantics would take 3-5 years if we got the funding today.
But hey, we do have a really sweet storage layer that blows away the other filesystems, while MS has seemingly given up on the serious algorithm issues we solved, and MS is now talking about putting the metadata into a layer above the FS rather than getting their tree algorithms right. Also, their semantics are probably going to be a confused hodge-podge of search engine and SQL shaped by turf battles with no single architect behind the design.
So, I have to say that things are looking interesting. I wish we had the funding I need to focus more of my time on coding and design.