Slashdot Mirror
WinFS' Demise Not a Bang Or a Whimper
Shadowruni writes "The Seattle-PI confirms with Mircosoft what MS bloggers and pundits have been saying all along. WinFS simply isn't going to happen. Some of its features have been 'merged' with other projects." From the article: "WinFS was dropped from Vista in what company executives described at the time as a trade-off to get the operating system completed in a timely manner. The release of Vista has since been delayed again and is now scheduled for November for large customers and January 2007 for the general public, though some observers say it may be out even later." Final confirmation of a story from last month.
So what you want to see is an compilation of immature academic technologies into a mature stable production system. Why not just wish for a gold house?
How we know is more important than what we know.
Meanwhile, other marketing people are looking at the feature set of distributed link tracking.... And another set of marketing weasels are looking at DRM respect... and attributes for near-line storage management... and (name any competitor's advantage, and expect Marketing to want to add it to the feature set).
The failure isn't in Engineering - it's in Management. Someone promised too much complexity.
Given a year or two per feature set, done incrementally, with product releases that allow the code to be tested and refined, WinFS probably could be engineered into a fine solution.
But the deadline is too close now. They need to cut their losses and bug-check what they have, now, so that the file system that does ship is stable, and not a huge disaster.
Interestingly, the open source solution of file systems is far better at trying out new ideas and making progress. It may take longer to make the features integrated - but that integration hasn't been a defining requirement for success or failure.
"The most sensible request of government we make is not, "Do something!" But "Quit it!"
Why WinFS failed to deliver...
When the concepts of relational database FS were being thrown around back in the mid 90s, there was a need for this technology. WinFS was to be the next progression of this work, but in its new form a non-structure, non-relational database FS technology.
WinFS was designed to sit on NTFS, never to replace it. In fact none of the proposed MS FS technologies were ever to replace NTFS.
WinFS did develop several inroads in database technology to move past relational and object oriented database storage concepts; however, this was not enough for it to succeed, but rather for its technology to be used in database and data access technologies like MSSQL and the ADO models.
There are two big reasons WinFS was stopped before ever seeing the light of day.
1) Efficiency over functionality
2) Business & Networked File Systems
The first is probably the biggest nail in the coffin, but yet also the hardest one to get through to people.
In current computing environments, adding in a good indexing technology, you can provide 99.9% of the functionality of WinFS and the overhead in doing so turns out to be less than if a full WinFS was implemented.
For example, it is easier and more efficient to have a database indexing backend that references the standard FS and FS contents than it is to put the FS contents into a database. This can be witnessed in products like MS Desktop Search, the Vista Desktop Search, and Apple's Desktop Search as well. (Although the Apple incarnation at this point is a bit more poky than it should be.)
The second part of this is the added functionality. One of the promises of WinFS was the ability to tag and relationally add content to files and file listings. Again, this does not offer 'enough' of an edge compared to the current FS technologies. Most of these features are already supported in NTFS, so you can add tagging, and additional fields of information to the files stored on an NTFS volume, basically providing the same features as adding new fields as a database FS would offer.
The only portion that is somewhat left behind in current technology that WinFS would have provided is the 'relational' nature of items in the FS. But again, the database indexing engine that is used for searching can also provide a certain level of these relational aspects to the file and contents.
So when you look at just these basic issues, you can see why in the end MS pulled WinFS as it exists today, and instead has put the functionality of WinFS in the current technologies, as you find in Vista already. (Fast search, relations between files and file contents, tagging using NTFS, etc.)
It may not be the best PR move for Microsoft in the long run, as people here will have a field day with WinFS being abandoned in its current form as an add-on to NTFS. But if you were Microsoft and could provide 99% of the functionality of WinFS with the database indexing services in Vista (and XP) and do it faster than having to add on a new WinFS layer to NTFS, they why would you progress with a product that isn't going to offer what they can already offer with the current technologies.
If computing power was on par with 1995, then something like WinFS would have more viability as Hard Drives and Processors could more efficiently do all that Vista is doing in a Database structured storage. However today, the overhead of doing this outside a database store is fairly non-existent.
On to the second reason, which is business. Implementing localized database stores for files and documents and keeping these in sync with corporate stores is a rather big hurdle when you consider that businesses are not average Joe users and have tons of applications and infrastructure to coordinate Files spread across networks that are outside of existing MS technologies. WinFS would break many business tools and models rather badly.
As for WinFS and Database FS concepts being 'vaporware' or dead, simply is a myth for the MS haters
After 10 years working to create a suitable storage layer for implementing semi-structured data queries in the FS for Linux, I gotta tell you, this stuff is harder than I ever thought it would be. :-/
(See Reiser4 for our storage layer, and our Future Vision paper for what semantics we are going to add.)
5 years to do the first draft (ReiserFS V3), and then another 5 years to get it finally right (Reiser4).
To do enhanced semantics cleanly, you want keywords to be just another kind of file (see our Future Vision paper for why. That means you need to store files that contain phone number sized objects and keywords reasonably efficiently. Because of network effect economics creating a barrier to entry, you have to at the same time make traditional file system usage patterns at least as fast. That is hard. How hard? Oracle tried to do it without deeply changing their tree algorithms, amd implemented an FS on top of their database engine, and found that it was half the speed of a traditional filesystem. Others also found it hard. I tried to do it with V3, and found that for files in the 0-10k size range, I had many of the performance problems that FFS had when they created fragments. Thing was, I never knew they had performance problems, because it was not in their paper.... The problem was that when you combined fragments from multiple files, you add seeks, and one added seek is deadly to performance. The approach used in most databases, BLOBS, suffers from the same problem as FFS combining fragments, and yet more, because BLOBs unbalance the tree (see our website for details and nice diagrams). The usual transaction technology employed for databases, it is just wrong for filesystems, what you need in an FS is to fuse multiple transactions together into batches. And more....
There are so many different areas where if you take a wrong step, performance goes through the floor. You cannot imagine how depressing it is to work on a project where the performance is terrible until the very end, after 5% to rarely 20% at a time you've dragged it into something decent over years of time. I look back on it, and I see that we were incredibly lucky, because all the mistakes I made, were mistakes that took days or weeks to fix, and except for one thing (BLOBs), all the major things that would take years to fix, I got right. There is no reason for this other than luck. And BLOBS cost us years.
So we have for Linux the storage layer that MS could not develop because they quit before 10 years had passed, and perhaps weren't lucky enough at. Now, with technology working, and balance trees that can emulate file system semantics at twice the speed of the real thing (see our benchmarks ), sigh, if only we can overcome the politics. Yup, the WinFS team had to deal with corporate managers that quit before 10 years are past, but we have to deal with..... better unfinished as a sentence.
The only consolation in this field is that everyone else seems to find it just as hard. Probably that includes even the politics.
What's VISTA an acronym for?
I thought everybody knew this by now.
Viruses, Infections, Spyware, Trojans and Adware.
I'll probably be modded down for this...