Slashdot Mirror
WinFS' Demise Not a Bang Or a Whimper
Shadowruni writes "The Seattle-PI confirms with Mircosoft what MS bloggers and pundits have been saying all along. WinFS simply isn't going to happen. Some of its features have been 'merged' with other projects." From the article: "WinFS was dropped from Vista in what company executives described at the time as a trade-off to get the operating system completed in a timely manner. The release of Vista has since been delayed again and is now scheduled for November for large customers and January 2007 for the general public, though some observers say it may be out even later." Final confirmation of a story from last month.
After 10 years working to create a suitable storage layer for implementing semi-structured data queries in the FS for Linux, I gotta tell you, this stuff is harder than I ever thought it would be. :-/
(See Reiser4 for our storage layer, and our Future Vision paper for what semantics we are going to add.)
5 years to do the first draft (ReiserFS V3), and then another 5 years to get it finally right (Reiser4).
To do enhanced semantics cleanly, you want keywords to be just another kind of file (see our Future Vision paper for why. That means you need to store files that contain phone number sized objects and keywords reasonably efficiently. Because of network effect economics creating a barrier to entry, you have to at the same time make traditional file system usage patterns at least as fast. That is hard. How hard? Oracle tried to do it without deeply changing their tree algorithms, amd implemented an FS on top of their database engine, and found that it was half the speed of a traditional filesystem. Others also found it hard. I tried to do it with V3, and found that for files in the 0-10k size range, I had many of the performance problems that FFS had when they created fragments. Thing was, I never knew they had performance problems, because it was not in their paper.... The problem was that when you combined fragments from multiple files, you add seeks, and one added seek is deadly to performance. The approach used in most databases, BLOBS, suffers from the same problem as FFS combining fragments, and yet more, because BLOBs unbalance the tree (see our website for details and nice diagrams). The usual transaction technology employed for databases, it is just wrong for filesystems, what you need in an FS is to fuse multiple transactions together into batches. And more....
There are so many different areas where if you take a wrong step, performance goes through the floor. You cannot imagine how depressing it is to work on a project where the performance is terrible until the very end, after 5% to rarely 20% at a time you've dragged it into something decent over years of time. I look back on it, and I see that we were incredibly lucky, because all the mistakes I made, were mistakes that took days or weeks to fix, and except for one thing (BLOBs), all the major things that would take years to fix, I got right. There is no reason for this other than luck. And BLOBS cost us years.
So we have for Linux the storage layer that MS could not develop because they quit before 10 years had passed, and perhaps weren't lucky enough at. Now, with technology working, and balance trees that can emulate file system semantics at twice the speed of the real thing (see our benchmarks ), sigh, if only we can overcome the politics. Yup, the WinFS team had to deal with corporate managers that quit before 10 years are past, but we have to deal with..... better unfinished as a sentence.
The only consolation in this field is that everyone else seems to find it just as hard. Probably that includes even the politics.