Slashdot Mirror
WinFS Gets the Axe
commander salamander writes "Over at the WinFS Team Blog, Quentin Clark states that Microsoft no longer plans to ship WinFS as a standalone software component. Instead, portions of the underlying technology will be included with the next release of SQL Server (codename Katmai) and ADO.NET. Does this spell the end for the true relational storage paradigm that Microsoft has been promising since Windows 95?"
How long has the promise of WinFS been on the table? Microsoft has dragged this teaser on 10-lb test in front of drooling long-time loyalists as the newest and amazingly innovative piece of their "best OS ever". Aside from the fact it really wasn't amazingly innovative (well, in vernacular maybe it was), now they're close to closing the door on this. I wonder how many sales they've pulled off with these lies?
HINT: Here's a snippet from an October 2003 PC World article:
Microsoft may not have thought they were lying at the time but they must have had an idea they not only weren't on target but they weren't even close! It's amazing a company can get away with this -- call it genius marketing, I call it deception at all costs to keep their customer base intact.
Sometimes these outcomes seem to say more about the Microsoft loyalists than Microsoft.
Google has a video of Hans Reiser talking about his vision of the Holy Grail of file systems, rather similar to what Microsoft has been promising. The difference is that he's moving toward it in baby steps (Reiser1, 2, 3, 4) and releasing those steps as he goes.
They say the mind is the first thing to
Not that I'm blaming them -- all software designs have limits, past which they can't be stretched any further and still be made to work. But perhaps Microsoft should be looking at starting over with a fresh new OS design (with backwards compatibility provided via virtual machine emulation only, a la MacOS Classic running in MacOS/X)?
I don't care if it's 90,000 hectares. That lake was not my doing.
A relational file systems is the next generation of OS design and a necessary evolution of the concept.
Put it this way, your computer stores hundreds of thousands of files, the current paradigm of treating them as files stored in a folder tree is absolutely antiquated and ridiculous.
I should be able to ask my operating system, "Show me all my picture files", and it simply can list ALL the image files on my computer, regardless of how or where they are stored. Features like Spotlight in OS X or Google Desktop are "nice" ways of trying to deal with this problem in a folder tree, but they are just an expensive to generate index file and it takes way too much time to return a result. Spotlight not only has to return if the index entry for a file matches, but it also has to verify if the file still exists on disk. I could take minutes for spotlight or Google desktop to return ALL image files on your computer. You will also notice that these systems often display something like (and 5000 more) link, this means that in order to have the search return results quick enough, it didn't REALLY find all 5000 files, it just says that according to its index file, there appears to be 5000 more image files, when you click on the link, it take more time to finally list all these files. Indexing a folder based tree structure is a solution, but its not an ideal solution. It is limited by the limitations of an antiquated file tree structure.
In a relational file system, if I ask for all image files stored on my computer, the result should be instantaneous, or near to it, as the fact that the file exists as a database entry means the file exists in reality. The time required for the results is simply the time required to build a query and return a result from a database.
Also, why do we even have to name files? Why do we have to give them a file extension. These are all antiquated file system concepts which are completely meaningless for a modern OS. A relational file system stores more then just a file name and a file type, I should be able to search for a file by date, description, keyword in the file, etc, etc, etc. I should not only be allowed to name the file, but provide any meta tags I want to help me locating that file quickly. An extension was a cheap way to get the OS to launch or open a file related to a specific program, but it would be completely unnecessary if the file itself embedded its type or had an entry in a database record. The name of a file would purely be a description and only one of many ways to identify a file.
Ultimately, a relational file system will allow such concepts as "Show me the letter about taxes I wrote to Bob Smith last week." and it will return the email or document you wrote, period. You don't care what the file name is. You don't care what type of file it may be, whether it was an email or text document. A file system should know that a file exists on your computer that is a texted based document, including keywords taxes and Bob that was generated within a week of the current date. This is a sorely needed concept in ANY OS, no OS to date has anything near that powerful a concept. There is no reason for a file system not to be able to handle these requests, and if we EVER want something like what we have seen in Star Trek, where people can ask a computer real language queries, we NEED a relational file system.
Relational files systems will bring a whole new level of superior storage capability to computers that will eventually start storing millions of files. We can't just keep a "lean and mean" tree based folder structure, that paradigm was never intended to manage millions of files.
I applaud Microsoft for at least trying, because unlike Google or Apple, they realize that the future is in a database driving relational file system and not stop gap pseudo-solutions like indexing. Its obviously a difficult concept to implement, but once anyone is able to implement the idea, it will be a VERY welcomed concept and improve the functionality and usability of an operating system. I for one would switch to and swear by ANY OS that implements this idea properly, whether its Linux, OS X, or yes, even Windows.
I haven't thought of anything clever to put here, but then again most of you haven't either.
This announcement makes more sense than I think people recognize, particularly if you posit that they ran into performance difficulties.
Remember that they were going to put it all into the filesystem? Then they put it into a layer above the FS. Bet you a bitflip that they found performance problems resulting from mixing large files and small files in the same filesystem. It is very easy to have such problems, you have to get quite clever to avoid them, and expend a lot of effort on it. Rather than take the extra time, they pulled the enhanced semantics out of the filesystem. That was the wrong thing. Then, look at the descriptions of how some of the queries they supported took too long. It is really easy to design things in that area wrong, and get unacceptable performance impacts. Rather than solving the deeply challenging problems, they punted and put the stuff into SQL server. Why? Because people who don't want things to be slow can just not use the feature until they figure out how to make it not cost performance. Of course, that means no OS integration but..... it is so nice to not be designing Reiser4 by committee.
I see Microsoft responding to difficult technical problems not by solving them, but by running from them, and that explains the entire trajectory of WinFS.
Another consideration you can see between the lines is that they don't want to lose the revenue from SQL Server by doing everything that it does in the OS and doing it better. Marketers will do things like make the first release of something only available at a higher price. They do that a lot. They'll do it even if it robs Vista of most of its excitement to do it.
Large corporations often have real problems handling tough research projects.
Reiser4 took 5 years to get into working at all (v3), and 10 years of sustained development to get right (reiser4), and it is just the storage layer. You can't do that in a large corporation.
In a large corporation you are thinking that you need 3 years to do a project that is a paradigm change, and you go talk to management, and you sense that they have patience for 9-18 months, and you really want to do the project, so you tell them you can do it in 9-18 months.
18 months go by, and you are 1/2 of the way through the first version (you think you are 90% of the way through), and the first version is going to suck badly and take years to be well optimized. Now, if your product is the first in its market, you can make it even though it sucks, and get the money for the version 2. If you are going into a mature market, well, things are tough. Very tough. WinFS is going into a mature market.
Now, into this reality throw corporate managers. They think that if they intimidate the programmers a lot, products ship sooner. So, technical shortcuts get taken. Only problem is, in a product like WinFS, going into a mature market, taking technical shortcuts kills things. Especially since for a product like WinFS the technical shortcuts affect DEEP decisions that you will never be able to reverse out of. Like, whether the enhanced semantics are in the FS layer. Or whether the whole OS is designed around using the enhanced semantics in every component. Then, managers feel the need to prove they are tough about schedules, and they cancel for being late projects that everyone should have known were going to take a long long time because they were hard. There is some very interesting recent research suggesting that if you want an accurate project length forecast, you don't ask for an estimate, you create a betting pool.
The sad thing is, since everyone copies Microsoft, now there will be more people saying that Reiser4 shouldn't do what WinFS backed away from. We can do it. We solved the hard storage layer design problems, our stuff works. Now we can finally go after the enhanced semantics. It took 10 years, but we got the storage layer into the shape we want it in, and one plugin at a time the enha