Slashdot Mirror
WinFS' Demise Not a Bang Or a Whimper
Shadowruni writes "The Seattle-PI confirms with Mircosoft what MS bloggers and pundits have been saying all along. WinFS simply isn't going to happen. Some of its features have been 'merged' with other projects." From the article: "WinFS was dropped from Vista in what company executives described at the time as a trade-off to get the operating system completed in a timely manner. The release of Vista has since been delayed again and is now scheduled for November for large customers and January 2007 for the general public, though some observers say it may be out even later." Final confirmation of a story from last month.
I would lay even money on Spring 2008. How long did Win2k take to stabilize? Granted XP went a little quicker, but the explosion of mal-ware made this an almost impossible, and some say unachievable task. I am sorry, but 10+ million lines of code just do not strike me as reasonably predictable and thus stable. At some point, the combinatorily explosion even might give the code sentinence...
With more and more announcements like these, does anyone else think it is inevitable that Linux will overtake Microsoft on all bases one day? I mean, it's starting to show Microsoft is only one company devoting a portion, large but just a portion, of it's resources to its OS while Linux is an entire industry with a bunch of diverse people working on small parts seperately.
I wonder if the Vista's voyage is any kind of vindication to the Linux side, who was always ballyhooed as having "too many distros" earlier, but at least we could depend on someone, somewhere releasing some small update with some type of progress (small but frequent steps) rather than the monolothic approach of large but infrequent steps.
How about a Poot?
It's not offtopic, dumbass. It's orthogonal.
So MS has was founded just over 31 years ago. Wouldn't a company that has spanned that many decades have a better understanding of software engineering and have a better grasp at making deadlines? I just don't get it. I'm not a fan of MS, but I'm trying to look past that: I just don't get how they can keep underestimating Vista the way they are.
:wq
I would love to see an OS released for the market that combines all of the research done within the past 10-15 years in kernels, file systems, HCI, application development, programming languages and APIs, virtual machines and virtualization, etc. However, look where we are at now. We're still using (for the most part) monolithic kernels, old file systems, old development tools, etc. There hasn't been any radical improvements in commerical OSes for quite some time. (One could say that OS X is a dramatic improvement, but much of OS X is based off NeXTSTEP, which had existed for quite some time before Apple bought them out).
I would like to see a new NeXTSTEP (technologically, not in terms of business success). NeXT was able to look at all of the current CS research of the time and integrate that into their operating system. NeXTSTEP was far ahead of its competition and, if it weren't for hardware support and the need for modern software, I'd probably run it as an everyday OS. Mac OS X is still ahead of its competition because of its NeXTSTEP roots, as well as Apple's improvements to the OS since 1997. Imagine if there was a new OS that took advantage of all of today's CS research, was very easy to use, and was compatible with existing software. I'd be the first person in line to buy it.
Until then, I can dream about my ultra-secure, exokernel OS with a database file system, flexible yet safe programming language, very easy to use UI, "boxes" to run Windows and *nix software....
So I suppose the purpose of having a road map is so that we can see where we didn't go.
Xix.
"Everything is adjustable, provided you have the right tools"
Overtaking Microsoft is not enough to become the dominant OS, for example see OSX.
Philosophy.
At least NTFS is somewhat understood now and drivers (although imperfect) exist and are being improved.
- anyway rubbish in order to function as well as Microsoft's own offerings.
I understand that WinFS was going to have NTFS as the backend but this avoids the necessity to reverse engineer another closed and obfusicated layer of almost-compliant-with-the-spec-which-you-cant-see
Think of the Children; Sleep with your Sister
Filesystems are for posers. Just write one big tar file to /dev/hda.
So what you want to see is an compilation of immature academic technologies into a mature stable production system. Why not just wish for a gold house?
How we know is more important than what we know.
Meanwhile, other marketing people are looking at the feature set of distributed link tracking.... And another set of marketing weasels are looking at DRM respect... and attributes for near-line storage management... and (name any competitor's advantage, and expect Marketing to want to add it to the feature set).
The failure isn't in Engineering - it's in Management. Someone promised too much complexity.
Given a year or two per feature set, done incrementally, with product releases that allow the code to be tested and refined, WinFS probably could be engineered into a fine solution.
But the deadline is too close now. They need to cut their losses and bug-check what they have, now, so that the file system that does ship is stable, and not a huge disaster.
Interestingly, the open source solution of file systems is far better at trying out new ideas and making progress. It may take longer to make the features integrated - but that integration hasn't been a defining requirement for success or failure.
"The most sensible request of government we make is not, "Do something!" But "Quit it!"
That OS already exists. There is so much cutting edge work going into Linux that Windows seems archaic in comparison. Yeah, the kernel is monolithic, that's because research over the past 10-15 years shows that microkernels are slooooow. File systems ... pick one, they ALL exist for Linux. HCI ... XGL anyone? Application development ... there are more IDEs and toolkits on Linux than one could learn in a lifetime. Programming languages ... all there. APIs ... broad question ... but anything that's not MS (and even some that are ... WINEAPI) are there. Virutal machines ... Bochs, VMWare, Win4LinPro, etc. Virutalization ... KML and XEN.
... I think you get the point. Now, having spewed all that, my impression is that you're waiting to see that "OS" from MS, nobody else, so you have to expect to be waiting a very long time, if ever for it. The fact is, if you want to be on the cutting edge, drop the past and use Linux. If you want to play games ... stay on Windows, it's DESIGNED for people who want something familiar, doesn't obselete any software compiled 15 years ago, and isn't so revolutionary as to scare grandma or the receptionist.
You can lock down Linux as tight as you want, use the Oracle IFS db based file system, use Ruby, KDE, VMWare
Good people do not need laws to tell them to act responsibly, while bad people will find a way around the laws-Plato
1) While the filesystem architecture is pretty horrible, there have been successes there by other companies.
There's are file system interfaces to NFS, FTP, EXT2, UDF, and a probably a few more that I can't think of right now. This has nothing to do with the previously badly written code.
The problem with WinFS is WinFS. It's got features in it that would make it unacceptably slow and easily corrupted. That won't fly. I think they thought that they could overcome these obvious problems through genius. Apparently its still hard.
2) Like every OS trailing back almost to the invention of the compiler, Windows is modular. And by that I don't mean "it has modules, or even dlls" I mean that the ideas within it are divided into real (and occasionally conceptual) pieces.
Some of the pieces are new and shiny and well written. Some are old and spaghetti-like. There's no reason to throw out everything to get one new piece. The fundamental design of the Windows kernel is neat even if the registry isn't. The network stack works pretty well even if the filesystem interface doesn't.
Along those lines,
I think they should stop selling windows as one thing. I'd like to know what new thing it is I'm getting in the latest version of Windows. Because they do occasionally throw out the old and replace it with something new and fresh that works great. But sometimes they only sell things that are exactly the same as the old, but with things I don't care about at all, or sell me lots of things I don't care about and only one that I do.
Mod me down and I will become more powerful than you can possibly imagine!
> And how often have we heard rumours of WinFS appearing in the next Windows OS?
Ever since Jim Allchin Microsoft announced Project Cairo in 1991. It was scheduled for release in 1994, and it was to include an object oriented file system similar to what is now referred to as WinFS. Now, 15 years later, these capabilities are STILL missing in action. Here's a chart showing Microsoft scheduled and actual ship dates.
Of course, in the mean time Microsoft has been chasing other innovations and their ever expanding appetites ever since that announcement...Internet connectivity and its related clients (www, mail, ftp, nntp...), security, improving stability, not to mention all the kinds of new hardware devices that had to be supported. Many of them were first supported by Windows.
Nonetheless, WinFS is one of the most obvious cases of Microsoft having "dropped the ball."
Why WinFS failed to deliver...
When the concepts of relational database FS were being thrown around back in the mid 90s, there was a need for this technology. WinFS was to be the next progression of this work, but in its new form a non-structure, non-relational database FS technology.
WinFS was designed to sit on NTFS, never to replace it. In fact none of the proposed MS FS technologies were ever to replace NTFS.
WinFS did develop several inroads in database technology to move past relational and object oriented database storage concepts; however, this was not enough for it to succeed, but rather for its technology to be used in database and data access technologies like MSSQL and the ADO models.
There are two big reasons WinFS was stopped before ever seeing the light of day.
1) Efficiency over functionality
2) Business & Networked File Systems
The first is probably the biggest nail in the coffin, but yet also the hardest one to get through to people.
In current computing environments, adding in a good indexing technology, you can provide 99.9% of the functionality of WinFS and the overhead in doing so turns out to be less than if a full WinFS was implemented.
For example, it is easier and more efficient to have a database indexing backend that references the standard FS and FS contents than it is to put the FS contents into a database. This can be witnessed in products like MS Desktop Search, the Vista Desktop Search, and Apple's Desktop Search as well. (Although the Apple incarnation at this point is a bit more poky than it should be.)
The second part of this is the added functionality. One of the promises of WinFS was the ability to tag and relationally add content to files and file listings. Again, this does not offer 'enough' of an edge compared to the current FS technologies. Most of these features are already supported in NTFS, so you can add tagging, and additional fields of information to the files stored on an NTFS volume, basically providing the same features as adding new fields as a database FS would offer.
The only portion that is somewhat left behind in current technology that WinFS would have provided is the 'relational' nature of items in the FS. But again, the database indexing engine that is used for searching can also provide a certain level of these relational aspects to the file and contents.
So when you look at just these basic issues, you can see why in the end MS pulled WinFS as it exists today, and instead has put the functionality of WinFS in the current technologies, as you find in Vista already. (Fast search, relations between files and file contents, tagging using NTFS, etc.)
It may not be the best PR move for Microsoft in the long run, as people here will have a field day with WinFS being abandoned in its current form as an add-on to NTFS. But if you were Microsoft and could provide 99% of the functionality of WinFS with the database indexing services in Vista (and XP) and do it faster than having to add on a new WinFS layer to NTFS, they why would you progress with a product that isn't going to offer what they can already offer with the current technologies.
If computing power was on par with 1995, then something like WinFS would have more viability as Hard Drives and Processors could more efficiently do all that Vista is doing in a Database structured storage. However today, the overhead of doing this outside a database store is fairly non-existent.
On to the second reason, which is business. Implementing localized database stores for files and documents and keeping these in sync with corporate stores is a rather big hurdle when you consider that businesses are not average Joe users and have tons of applications and infrastructure to coordinate Files spread across networks that are outside of existing MS technologies. WinFS would break many business tools and models rather badly.
As for WinFS and Database FS concepts being 'vaporware' or dead, simply is a myth for the MS haters
I've been running on Vista, and most programs I've tried (Photoshop, Dreamweaver, Opera, WoW, Guild Wars, Trillian) run perfectly fine. Some I've had to run with admin access (WoW, utorrent), others just worked. The only program I've had trouble with is Nero. Nero 6.whatever doesn't load Nero Express, but the actual Nero Burning ROM program works. I tried installing the Nero 7 demo, but it won't run for some reason. I haven't tried MS WOrks.
Linux does have a ton of research-level development on it. You can do amazing things that you can't do anywhere else. Unfortunately the history of UNIX weighs in very heavily in almost all OS development. The fact is that the problems with UNIX are obscured by how horrendous Windows is. Think how much we could really move forward if we were to take some fresh ideas like Plan9. Unfortunately the software economy is too mature for a cutting-edge research OS to be able to get a critical mass of developers. No one wants to write software for a new OS when there's already so much open source out there for Linux/UNIX. If you could get paid to do pure research it would be pretty fun though.
"WinFS was dropped from Vista in what company executives described at the time as a trade-off to get the operating system completed in a timely manner."
Oops... Too late for that don't you think?!
The race isn't always to the swift... but that's the way to bet!
After 10 years working to create a suitable storage layer for implementing semi-structured data queries in the FS for Linux, I gotta tell you, this stuff is harder than I ever thought it would be. :-/
(See Reiser4 for our storage layer, and our Future Vision paper for what semantics we are going to add.)
5 years to do the first draft (ReiserFS V3), and then another 5 years to get it finally right (Reiser4).
To do enhanced semantics cleanly, you want keywords to be just another kind of file (see our Future Vision paper for why. That means you need to store files that contain phone number sized objects and keywords reasonably efficiently. Because of network effect economics creating a barrier to entry, you have to at the same time make traditional file system usage patterns at least as fast. That is hard. How hard? Oracle tried to do it without deeply changing their tree algorithms, amd implemented an FS on top of their database engine, and found that it was half the speed of a traditional filesystem. Others also found it hard. I tried to do it with V3, and found that for files in the 0-10k size range, I had many of the performance problems that FFS had when they created fragments. Thing was, I never knew they had performance problems, because it was not in their paper.... The problem was that when you combined fragments from multiple files, you add seeks, and one added seek is deadly to performance. The approach used in most databases, BLOBS, suffers from the same problem as FFS combining fragments, and yet more, because BLOBs unbalance the tree (see our website for details and nice diagrams). The usual transaction technology employed for databases, it is just wrong for filesystems, what you need in an FS is to fuse multiple transactions together into batches. And more....
There are so many different areas where if you take a wrong step, performance goes through the floor. You cannot imagine how depressing it is to work on a project where the performance is terrible until the very end, after 5% to rarely 20% at a time you've dragged it into something decent over years of time. I look back on it, and I see that we were incredibly lucky, because all the mistakes I made, were mistakes that took days or weeks to fix, and except for one thing (BLOBs), all the major things that would take years to fix, I got right. There is no reason for this other than luck. And BLOBS cost us years.
So we have for Linux the storage layer that MS could not develop because they quit before 10 years had passed, and perhaps weren't lucky enough at. Now, with technology working, and balance trees that can emulate file system semantics at twice the speed of the real thing (see our benchmarks ), sigh, if only we can overcome the politics. Yup, the WinFS team had to deal with corporate managers that quit before 10 years are past, but we have to deal with..... better unfinished as a sentence.
The only consolation in this field is that everyone else seems to find it just as hard. Probably that includes even the politics.
Yay - it's refreshing to see someone working for 'the other side' (for want of a better term) who reacts to this story in a realistic and honest way, without feeling the need to bash MS for their WinFS problems ("Ha ha! M$ are teh suck!", etc).
Perhaps, I don't know, it's because you've spent years working on this problem, and know the difficulties involved, rather than the average slashdot MS basher who read a magazine article about writing file systems once and can't see what's so hard about them, or, come to that, like some of the other posters here, who can't see what's so hard about managing one of the largest software projects on the planet.
It was probably recoverable, if you had contacted support@namesys.com, and if it was not an extensive hardware error. Most ReiserFS V3 corruptions (and most ext3 and XFS corruptions) occur due to hardware errors. The ones that cause only a block or two to be bad are usually recoverable. The ones that lose the whole drive, well, no FS will save you from that....