Slashdot Mirror
Next Windows to Have New Filesystem
ocipio writes: "Microsoft is currently planning a new filesystem. Its planned that the new filesystem will make searches easier, faster, and more reliable. Windows will also be less likely to break, and easier to fix when it does. The new technology will cause practically all Microsoft products to be rewritten to take advantage of it. Called Object File System, OFS will be found in the next major Windows release, codenamed Longhorn. More information can be found here at CNET."
Check that you're not running an Office documents reindex. This is on by default and _despite_ the name, what it does is search every file your document pool and attempts to add it to the index. Very not recommended.
Yes, moving the documents _will_ stop this process, but it's not the most practical method is it? Better, by a long, long way, to turn off automatic indexing of office documents.
Cheers
Chris
The GNOME project has an excellent overview of some of the issues with metatata in general.
Of course, this will mean a whole new found of application incompatibilities on Windows and a whole new round of reverse engineering to determine the filesystem and metadata layout.
Rather a good thing to know.
dinner: it's what's for beer
Yes, you are a troll. Is it wrong for Microsoft to advance File systems and state specific reasons and right to preach about the many choices in file systems linux/unix has?
When your talking
Say when you open a folder of 5,000+ mp3's and it searches the ID3 tags of every song and displays artist/title as part of the description, having an optimized file system for quicker searches of data on the disk will only streamline this more.
so yes, this is cool, and yes, there is alot more then just "fast searches" as you put it.
that they can simply take FFS and Soft Updates and embed it in Windows, and call it OFS.
NTFS has features like ACLs, streams, etc that aren't in FFS or UFS. Also, support for transparent compression and encryption, also sparse files. There's support for quotas in the filesystem, and it's quite resistant to the effects of fragmentation. It's journalled and supports Unicode. It's actually a very good filesystem, once of the better parts of NT.
Early versions of BeOS had a full object orientated file system and found performance was abysmal. This was from a company with no backwards compatibility to worry about and a small OS designed for speed.
In the end Be developed BFS which is basically a standard file system with support for indexes and attributes, an overall much better performing system with most of the benefits of an object orientated file system.
[)amien
Hey trollie, read the article. Or better yet, the Slashdot posting. This file system has nothing to do with DRM.
.. boy, don't even get me started on that.
Firstly, there is no such thing as "Microsoft Digital Rights Management Software" (Media Player supports DRM, but only for WMA's) and Microsoft has nothing what-so-ever to gain from including DRM features into the file system. They know and we know that Longhorn with DRM will go down the toilet, while Longhorn without DRM will sell just as well as WinXP, probably better.
The second thing you got wrong is that this system is not (just) about speeding up searches. It's about replacing an antiquated system that's been around since MS-DOS with something future-proof, faster, and more reliable. Considering they've been working on this for 10+ years, they'll probably succeed eventually. And when they do
Now, for something constructive. When will we see this in Linux? Surely, if Microsoft can do this, so can the people working on Linux. Riiight?
Quality, performance, value; you get only two, and you don't always get to pick.
Now for some speculation. The implementation is likely to be based on the same "pluggable" FS driver architecture first introduced in Windows 2000, where the NTFS and FAT32 drivers are just a layer on top of the kernel (you can actually buy a devkit from Microsoft that will allow you to implement, say, ReiserFS for Windows 200). This however poses an interesting question: do you make this newfangled FS to sit on top of tried-and-true NTFS, or do you implement it at the kernel level and make NTFS a layer on top of that? Either way, I think the article is overstating the devastating effect on existing apps. Microsoft is not about to shoot itself in the foot so massively. Whatever this ends up being it's a good bet it will be fully backwards compatible. Kinda like Win32, which can still run 16-bit apps, albeit slower (in 2K and XP). But this will make companies more likely to either port existing code or release newer versions that take advantage of the redesigned FS.
There's a interesting Register article here
The point of this new data store isn't necessarily faster searches, although that is one part of it. The idea is to have a common data storage mechanism, used by all programs.
The underlying technology is to replace the NTFS filesystem driver with SQL Server, with a few tweaks. SQL Server already supports using a RAW partition as a data store, so essentially you just have to move the transaction log and descriptive info for the databases into a specific area of the disk. Add a little bit of bootstrap code to ntldr, and slap the SQL Server stuff into the startup driver list, and it's a done deal.
The next step is creating an NTFS compatibility layer -- it would allow you to mount tables as drive letters or network shares. A lot of the information wouldn't be useful when viewed in that fashion, but it would give you a way to run older programs.
Once all your data is in a common data store and can be manipulated as such, it opens up a world of new possibilities. The change will be long and slow; no need to kid about that. It will take years for all the 3rd party programs (and even Microsoft's own apps) to catch up and start taking full advantage of it. It's the same situation Plug & Play was in back in 1995; it sorta worked sometimes, but you couldn't really take full advantage of it. But here in 2002, you really can expect to grab a piece of hardware and slap it in your box without hassles. It took some time, but it eventually paid off.
But... are you having trouble, as I did, thinking of ways to make use of this common data store? Part of that comes from the fact that we've been conditioned and trained to think of data storage in terms of files; it's hard to shift gears... to think outside of the "filesystem" box so to speak.
For one thing, I could see someone emailing me a project. Not some word documents, an excel spreadsheet, and a database zipped into a ZIP file; they just email me the project. When I get it, and open the message, the project opens up presenting me with the various documents (linked to the database of phone numbers for example), and a little yellow stickynote window that has the project leader's actual email text. I didn't have to deal with unzipping the data, rearranging it, then opening the documents separately. Since the "rows" are linked, they open and act as a unit until I tell them to do otherwise.
It gets better though... imagine if I could run a query such as "SELECT f.*, s.filename FROM Folder1 f INNER JOIN folder2 s ON f.datetime = s.datetime"
It can get even more useful because you now have full SQL syntax available to you for manipulating the filesystem, with queries that are lightning fast. Throw in some Stored Procedures, Functions, Views, etc and I can see real possibilities.
Natural != (nontoxic || beneficial)
I do this by putting everything on my webpage and letting google sort it out :)
-Yarn - Rio Karma: Excellent
> They want to get their Digital Rights
.NET and start dropping the crufty Win16-contaminated Win32 API and x86 ties.
> Management Software to infest every aspect of
> their OS as possible.
Right. You keep throwing your FUD about while the rest of us looks at things seriously.
> Do you honestly believe that the benifit of a
> faster search is enough incentive to rewrite
> such a major part of the OS?
Short Answer: Yes.
Long Answer: Filesystems haven't changed much in the past few decades, but one of the things they have tended to gain is arbitrary metadata. Adding indexing to that metadata is a natural progression of that.
Now your filenames are just a part of the metadata you'll want to play with different views of the system, which suddenly becomes much much cheaper. Believe it or not, lots of users have trouble understanding the current basic filesystem concepts and using them to organise their data; well, now you can do it automatically for them.
Of course, you want your other stuff to make use of these new ways of looking at the system, especially when you're MS and are running out of new features to put in (come on, what are they going to add to Word XP now? A paperclip with speech recognition? Yet another GUI redesign?), so you want to do something that provides a visible difference (and maybe even an advantage) for those expensive upgrade programmes.
So, yes, I think they do have a very good reason for such a major change, like they had good reason to introduce '95 and start dropping DOS, or NT and start dropping Win16, or
The biggest issue I have with it is that it's going to be a bitch to use in other OS's. Hopefully they'll do detailed specs and stick to them fairly closely (ah haha), which will at least make it easier.
NTFS is a very solid filesystem and seems to recover problems well when something bad does happen. The only complaints I have are slow searches and reports. It takes a LONG time to find a file on a big volume, or try and do reports on file system usage. A good database system should speed that up tremendously.
The idea of having to rewrite the apps is interesting though. That tells me this is at least 5 years off, and longer before it would be used widescale. But I guess that makes sense, would you be the first shop to put your big fileserver on a new filesystem like that? Not me.
When I read this article, I immediately had two thoughts:
Thought 1: "You know, they're right" Current file systems are outdated and are not really serving the needs of modern applications. Take for example, Microsoft Outlook (and Outlook Express). The programming teams for these pieces of software were forced to implement a "filesystem within a file" in order to achieve their design goals (I believe the files are called DBX files). Or take for instance, the Windows Registry, or, even better, the Gnome registry, GConf. Why do programmers have to implement dozens of different abstract filesystems in order to achieve their design goals? Simple, the present filesystems are not sufficient.
Thought 2: "Another way of attacking the Free Software Movement." By creating a new filesystem, Microsoft achieves many goals. First, they make Linux filesystem developers start from scratch again. I mean, the NTFS driver isn't even done, and this means we would have to start over. It gets even worse: From the sound of this article, it seems that OFS would be fundamentally incompatible with our conception of a filesystem today (possibly including features such as resource branches, GUID tags, and other metadata forks, ad nauseum). This would make it difficult to write a usable Linux driver for OFS. And finally, to top it off, my gut tells me that the POSIX file access calls would _not_ be sufficient to access such a rich filesystem. The introduction of a new, richer file access API by Microsoft would make writing cross-platform software much more difficult.
Microsoft can kill two birds with one stone here.
Ben
The Register had and article about this ages ago.
Think SQL Server 2003 = OFS
Not only do they get their new FS with nifty new features (DRM yada...yada...) but imagine this scenario...
MS Sales Rep: "You need a database?"
Potential Sucker^H^H^H^H^H^HCustomer: "We're looking at Oracle."
Rep: "Oracle's OK and all, but you know...the TCO is over the top. With Windows NG [Next Generation], a top-notch, state-of-the-art DB is included for free."
Customer: "Really? Hmm...well, if it's already in there, I might as well use it."
Don't knock it, it worked with IE.
Getting rid of extensions is not necessarily a good thing...
.avi files. Of course she checked a and then w and then mi* area and no guesses were right. The right answer was "Movie Clip" (way to generic, but anyway...) Now look at the same dialog under, say XP. You will see that things are sorted by extension. While it may seem clunky and inelegant conceptually, in practice it is elegant. I would say identifying type is important enough to belong in the filename.
/etc/magic would be prohibitively slow. If you suddenly designate a part of the file space as needing to contain type information, tons of legacy problems can arise, and that field better be pretty long, and have a standard organization dictating what gets to use what codes. You can keep the information out of the file and efficient through Extended Attributes (already possible with NTFS, XFS, Be's FS, among others), but as I mentioned before this would not work cross-platform.
First off, all kinds of things are already designed around the extension idea, redesigning everything won't work that easily. Also, users are used to the concept of extensions. MS is very much aware of this. If you take, say, Windows 98 and go to edit file type associations, the list is sorted by type description. This doesn't work, as the user is not likely to know the string attributed to the file type he is thinking of. For example, recently my fiancee wanted to change the default viewer for
Secondly, these extended attributes are not portable. Many widely used protocols would be unable to automatically notify client machine of this information, forcing the user on every file downloaded to set the type of the file manually. Sure you can embed them directly in the file, but who gets to dictate the format? I can bet you that MS would extend any standard to break compatibility with other systems if it existed. By tying in the reading of these extended attributes more tightly with the opening of files, you are inviting MS to come and make life harder on non-MS platforms, as well as technologies such as DRM to have more success...
Finally, what about performance? As it stands, a system based on
The system as it stands now works quite well. Windows explorer already works to "protect the user from himself" by not allowing renaming of extension on a file easily. We have an established, cross-platform standard for identifying file types, we don't need to blow that...
XML is like violence. If it doesn't solve the problem, use more.
That's what they said about the registry. It will solve all of the problems with ini files.
But as everyone knows, with totally undisaplined usage of the registry, the registry is a nightmare. In some cases it is impossible to clean it up and the only solution is a reinstall.
Ask any dba. Even with the most heavy duty industrial strength db, somebody can come up with a schema and application that will bring that db to its knees. Prepare for deja vu.
[user] Meet my friends Mr. Debian Boot Disk and Mr. Debian Root Disk
[windows] A priority alert has been dispatched to the BSA.
[Debian Boot Disk] So Boss, do ya want to just rough him up a little or completely murdalize the bum?
"We've been working hard on the next file system for years [since the early 1990's], and -- not that we've made the progress that we've wanted to -- we're at it again," Ballmer said.
While the Cairo project eventually resulted in Microsoft's Windows 2000 operating system, the file system work was abandoned because of complexity, market forces and internal bickering. "It never went away. We just had other things that needed to be done," Jim Allchin, the group vice president in charge of Windows development, told News.com.
Those other things most likely included battling "Netscape and Java and the challenge of the Internet and the Department of Justice," Gartner Group analyst David Smith said--issues that continue to persist today.
<snip>
The more important reasons for the renewed development effort, however, are strategic. If the plan succeeds, it will give Microsoft a huge technological advantage over the competition by making its products more attractive to buyers and giving large companies another reason to install Windows-based servers.
So if they hadn't been trying so hard to kill off Netscape, they would have had the time to spend on creating this. Something that seems to offer actual advantages to the user, and that would be "a huge technological advantage over the competition by making its products more attractive to buyers."
I wonder how many other genuine advances have been put on hold in the name of detroying someone else first.
Nope, no sig
The extension, like the rest of the filename, is metadata. The output of a pipe or socket, in case you haven't noticed, doesn't include that metadata, but most apps getting their data piped in manage without it. For those that can't, most transport mechanisms move that metadata with the file, and so things work all right. If transport mechanisms need to be upgraded to include metadata... okay, we can do that. It's why most good standards are extensible.
The problem is that it's inflexible, and individual file types include their own formats for additional metadata (like id3 tags). Hate to break it to 'ya, but having a whole bunch of little incompatible metadata formats sucks. It sucks a lot. If I want to write a search tool that can look through files' metadata, I don't want to teach it one format for MP3s, one format for Word documents, one format for Joe's Accounting System datafiles... well, you get the point.
Files sharing names but with differing metadata are Bad. That you generally aren't allowed to have them is a Good Thing -- if the file is different enough to need different metadata (like foo.txt and foo.rtf), they should have different names (as foo.txt and foo.rtf do, incidentally -- how handy! -- but doing that for *all* metadata is inappropriate).
You've "debunked" exactly nothing.
I'm told this file system won't need to be de-fragmented.
Oh wait, that email was 7 years old.
- Dan I.
A lot of people seem to've totally missed the point of what would be different about a database-oriented filesystem. File extensions? Not bloody well relevant! Let's consider the issue of searching. A database-oriented filesystem might allow you to create directories that are basically "views" of your filesystem, perhaps including all files that meet certain name, attribute or content criteria (like Evolution's vFolders but available to any app). These views would be up-to-the-instant accurate at all times, with no dead links and no problem with apps replacing links with actual files instead of updating the file that the link pointed to. Filesystems could also benefit from other things like referential-integrity checks, triggers, and cross-file transactional behavior. In fact, there has been a lot of work in the kernel-hacker community to figure out how just that last feature could be added to Linux filesystems. Basing a filesystem on a database also allows you to leverage all of the tools (e.g. efficient snapshots and replication) that have been developed for the database. There's a lot more here than just journaling and BeOS-style metadata.
It's not that I think basing a filesystem on a database is a great idea. For one thing, it's a pretty good bet that performance is going to suck because of all the extra DB-related overhead. Administration might become more of a PITA too. I'm just trying to explain that the idea of a database-oriented filesystem has much broader implications than the trivial crap (much of which is relevant to neither filesystems nor databases) that people seem to be focusing on in this thread so far.
Slashdot - News for Herds. Stuff that Splatters.
MSDE _is_ SQL Server throttled for = 10 users and without the visual tools.
I have a website. It's about Macs.