Slashdot Mirror
Next Windows to Have New Filesystem
ocipio writes: "Microsoft is currently planning a new filesystem. Its planned that the new filesystem will make searches easier, faster, and more reliable. Windows will also be less likely to break, and easier to fix when it does. The new technology will cause practically all Microsoft products to be rewritten to take advantage of it. Called Object File System, OFS will be found in the next major Windows release, codenamed Longhorn. More information can be found here at CNET."
I hope this doesn't cause as many problems as the last feature they added.
Check that you're not running an Office documents reindex. This is on by default and _despite_ the name, what it does is search every file your document pool and attempts to add it to the index. Very not recommended.
Yes, moving the documents _will_ stop this process, but it's not the most practical method is it? Better, by a long, long way, to turn off automatic indexing of office documents.
Cheers
Chris
It looks like BeFS with XML descriptions instead of MIME types. I think.
[o]_O
The GNOME project has an excellent overview of some of the issues with metatata in general.
Of course, this will mean a whole new found of application incompatibilities on Windows and a whole new round of reverse engineering to determine the filesystem and metadata layout.
From what I understand, Veritas essentially rewrote NTFS version 5 (shipped with win2000 and winXP) and
integrated built-in volume management (dynamic disks) with some abstract layer to maintain the clunky drive letter schemes.
I dont' really see the reasoning by mentioning that all Windows applications will require a rewrite; they only need to abstract the Win32::File APIs to handle the internal OFS changes... the changes that they document appear to do essentially what the Indexing services do for win2000 and winXP now. I assume it is just extra metadata strings that they associate with each file inode.
It seems a bit arrogant of MS to think they can improve (or trash and rewrite) what is essentially Veritas' domain, file systems. But then again, we're talking about MS...
Everyone would have to buy new versions of all their office software! Isn't that handy for MS?
I'll pass. I may be running (pre-installed) XP on my Dell but I'm still using Office 97. Why?
BECAUSE IT WORKS JUST FINE.
I don't need to "upgrade" to something even more bloated and bug ridden.
Rather a good thing to know.
dinner: it's what's for beer
I hope any changes that happen to the file system also include the removal of the antiquated concept of file extensions for type association. Here is another thing that Mac does very well. Imbed the type of a file IN the file. Why not give me a version number and some way to know what program created it.
Back to the original topic, I can't wait for an OFS. Just for my MP3's. Figuring out which folder hierarchy to use for genre/group/album/track is a pain. Let the file system group them for me.
A speech...
I hope they haven't discovered the unified file system, something which all the unixes use to store their data. We might be looking at the demise of unix!
oh wait, where do we put Progra~1 then?
- runs away to see if it's patentable
Yes, you are a troll. Is it wrong for Microsoft to advance File systems and state specific reasons and right to preach about the many choices in file systems linux/unix has?
When your talking
Say when you open a folder of 5,000+ mp3's and it searches the ID3 tags of every song and displays artist/title as part of the description, having an optimized file system for quicker searches of data on the disk will only streamline this more.
so yes, this is cool, and yes, there is alot more then just "fast searches" as you put it.
You can achieve this goal by the following process:
1. Store all of your documents in a simple, text based format, and not in some overly complex propriatary format such as ".DOC", ".PDF", etc.
1. This text based format is known as "American Standard Code for Information Interchange" (AKA "ASCII")
2. If you require more complex presentation of information, you might want to use something called HyperText Markup Language. (Which doesn't do much markup these days... but I digress)
3. There is a program built in to Windows 98 and above called "Find" (usually accessed by hitting F3), and in other environments known as "grep" which can search by content.
Use the tools you have, you won't have to "upgrade" to the latest bugs, and the computer remains useful.
--Mike--
that they can simply take FFS and Soft Updates and embed it in Windows, and call it OFS.
NTFS has features like ACLs, streams, etc that aren't in FFS or UFS. Also, support for transparent compression and encryption, also sparse files. There's support for quotas in the filesystem, and it's quite resistant to the effects of fragmentation. It's journalled and supports Unicode. It's actually a very good filesystem, once of the better parts of NT.
Early versions of BeOS had a full object orientated file system and found performance was abysmal. This was from a company with no backwards compatibility to worry about and a small OS designed for speed.
In the end Be developed BFS which is basically a standard file system with support for indexes and attributes, an overall much better performing system with most of the benefits of an object orientated file system.
[)amien
Hey trollie, read the article. Or better yet, the Slashdot posting. This file system has nothing to do with DRM.
.. boy, don't even get me started on that.
Firstly, there is no such thing as "Microsoft Digital Rights Management Software" (Media Player supports DRM, but only for WMA's) and Microsoft has nothing what-so-ever to gain from including DRM features into the file system. They know and we know that Longhorn with DRM will go down the toilet, while Longhorn without DRM will sell just as well as WinXP, probably better.
The second thing you got wrong is that this system is not (just) about speeding up searches. It's about replacing an antiquated system that's been around since MS-DOS with something future-proof, faster, and more reliable. Considering they've been working on this for 10+ years, they'll probably succeed eventually. And when they do
Now, for something constructive. When will we see this in Linux? Surely, if Microsoft can do this, so can the people working on Linux. Riiight?
Quality, performance, value; you get only two, and you don't always get to pick.
Now for some speculation. The implementation is likely to be based on the same "pluggable" FS driver architecture first introduced in Windows 2000, where the NTFS and FAT32 drivers are just a layer on top of the kernel (you can actually buy a devkit from Microsoft that will allow you to implement, say, ReiserFS for Windows 200). This however poses an interesting question: do you make this newfangled FS to sit on top of tried-and-true NTFS, or do you implement it at the kernel level and make NTFS a layer on top of that? Either way, I think the article is overstating the devastating effect on existing apps. Microsoft is not about to shoot itself in the foot so massively. Whatever this ends up being it's a good bet it will be fully backwards compatible. Kinda like Win32, which can still run 16-bit apps, albeit slower (in 2K and XP). But this will make companies more likely to either port existing code or release newer versions that take advantage of the redesigned FS.
There's a interesting Register article here
The point of this new data store isn't necessarily faster searches, although that is one part of it. The idea is to have a common data storage mechanism, used by all programs.
The underlying technology is to replace the NTFS filesystem driver with SQL Server, with a few tweaks. SQL Server already supports using a RAW partition as a data store, so essentially you just have to move the transaction log and descriptive info for the databases into a specific area of the disk. Add a little bit of bootstrap code to ntldr, and slap the SQL Server stuff into the startup driver list, and it's a done deal.
The next step is creating an NTFS compatibility layer -- it would allow you to mount tables as drive letters or network shares. A lot of the information wouldn't be useful when viewed in that fashion, but it would give you a way to run older programs.
Once all your data is in a common data store and can be manipulated as such, it opens up a world of new possibilities. The change will be long and slow; no need to kid about that. It will take years for all the 3rd party programs (and even Microsoft's own apps) to catch up and start taking full advantage of it. It's the same situation Plug & Play was in back in 1995; it sorta worked sometimes, but you couldn't really take full advantage of it. But here in 2002, you really can expect to grab a piece of hardware and slap it in your box without hassles. It took some time, but it eventually paid off.
But... are you having trouble, as I did, thinking of ways to make use of this common data store? Part of that comes from the fact that we've been conditioned and trained to think of data storage in terms of files; it's hard to shift gears... to think outside of the "filesystem" box so to speak.
For one thing, I could see someone emailing me a project. Not some word documents, an excel spreadsheet, and a database zipped into a ZIP file; they just email me the project. When I get it, and open the message, the project opens up presenting me with the various documents (linked to the database of phone numbers for example), and a little yellow stickynote window that has the project leader's actual email text. I didn't have to deal with unzipping the data, rearranging it, then opening the documents separately. Since the "rows" are linked, they open and act as a unit until I tell them to do otherwise.
It gets better though... imagine if I could run a query such as "SELECT f.*, s.filename FROM Folder1 f INNER JOIN folder2 s ON f.datetime = s.datetime"
It can get even more useful because you now have full SQL syntax available to you for manipulating the filesystem, with queries that are lightning fast. Throw in some Stored Procedures, Functions, Views, etc and I can see real possibilities.
Natural != (nontoxic || beneficial)
> They want to get their Digital Rights
.NET and start dropping the crufty Win16-contaminated Win32 API and x86 ties.
> Management Software to infest every aspect of
> their OS as possible.
Right. You keep throwing your FUD about while the rest of us looks at things seriously.
> Do you honestly believe that the benifit of a
> faster search is enough incentive to rewrite
> such a major part of the OS?
Short Answer: Yes.
Long Answer: Filesystems haven't changed much in the past few decades, but one of the things they have tended to gain is arbitrary metadata. Adding indexing to that metadata is a natural progression of that.
Now your filenames are just a part of the metadata you'll want to play with different views of the system, which suddenly becomes much much cheaper. Believe it or not, lots of users have trouble understanding the current basic filesystem concepts and using them to organise their data; well, now you can do it automatically for them.
Of course, you want your other stuff to make use of these new ways of looking at the system, especially when you're MS and are running out of new features to put in (come on, what are they going to add to Word XP now? A paperclip with speech recognition? Yet another GUI redesign?), so you want to do something that provides a visible difference (and maybe even an advantage) for those expensive upgrade programmes.
So, yes, I think they do have a very good reason for such a major change, like they had good reason to introduce '95 and start dropping DOS, or NT and start dropping Win16, or
The biggest issue I have with it is that it's going to be a bitch to use in other OS's. Hopefully they'll do detailed specs and stick to them fairly closely (ah haha), which will at least make it easier.
NTFS is a very solid filesystem and seems to recover problems well when something bad does happen. The only complaints I have are slow searches and reports. It takes a LONG time to find a file on a big volume, or try and do reports on file system usage. A good database system should speed that up tremendously.
The idea of having to rewrite the apps is interesting though. That tells me this is at least 5 years off, and longer before it would be used widescale. But I guess that makes sense, would you be the first shop to put your big fileserver on a new filesystem like that? Not me.
NTFS has features like ACLs, streams, etc that aren't in FFS or UFS. Also, support for transparent compression and encryption, also sparse files. There's support for quotas in the filesystem, and it's quite resistant to the effects of fragmentation. It's journalled and supports Unicode. It's actually a very good filesystem, once of the better parts of NT.
Right. This begs the question of why bother ?
The push et al is just a load of hype to push the upgrade path. They are going to engineer a database into their file system "to make searches faster" because doing it the slocate way would not force another round of complete system upgrades on consumers.
You may have also noticed that Outlook and Office will need to be rewritten to "take advantage" of the new file system. So not only will they leverage OS upgrades, but Office upgrades as well. They are planning to rip out a perfectly good file system (which is called "antiquated" in the article) to make billions of dollars, and the press releases are all about consumer benefit.
And consumer benefit, as you have noted, is essentially nil.
It is funny, we accuse Microsoft of using other people's ideas - but are we really any better? How much of Open Source development is really just reimplementations of other people's ideas?
When I read this article, I immediately had two thoughts:
Thought 1: "You know, they're right" Current file systems are outdated and are not really serving the needs of modern applications. Take for example, Microsoft Outlook (and Outlook Express). The programming teams for these pieces of software were forced to implement a "filesystem within a file" in order to achieve their design goals (I believe the files are called DBX files). Or take for instance, the Windows Registry, or, even better, the Gnome registry, GConf. Why do programmers have to implement dozens of different abstract filesystems in order to achieve their design goals? Simple, the present filesystems are not sufficient.
Thought 2: "Another way of attacking the Free Software Movement." By creating a new filesystem, Microsoft achieves many goals. First, they make Linux filesystem developers start from scratch again. I mean, the NTFS driver isn't even done, and this means we would have to start over. It gets even worse: From the sound of this article, it seems that OFS would be fundamentally incompatible with our conception of a filesystem today (possibly including features such as resource branches, GUID tags, and other metadata forks, ad nauseum). This would make it difficult to write a usable Linux driver for OFS. And finally, to top it off, my gut tells me that the POSIX file access calls would _not_ be sufficient to access such a rich filesystem. The introduction of a new, richer file access API by Microsoft would make writing cross-platform software much more difficult.
Microsoft can kill two birds with one stone here.
Ben
The SQL server files have to be on a "file system".. so what's THAT going to be?
Not if your database server has its own datastore functionality built in -- and many do. You know how Oracle can work with raw partitions directly? Think 'bout that kind of functionality, just done by MS.
The Register had and article about this ages ago.
Think SQL Server 2003 = OFS
Not only do they get their new FS with nifty new features (DRM yada...yada...) but imagine this scenario...
MS Sales Rep: "You need a database?"
Potential Sucker^H^H^H^H^H^HCustomer: "We're looking at Oracle."
Rep: "Oracle's OK and all, but you know...the TCO is over the top. With Windows NG [Next Generation], a top-notch, state-of-the-art DB is included for free."
Customer: "Really? Hmm...well, if it's already in there, I might as well use it."
Don't knock it, it worked with IE.
That's what they said about the registry. It will solve all of the problems with ini files.
But as everyone knows, with totally undisaplined usage of the registry, the registry is a nightmare. In some cases it is impossible to clean it up and the only solution is a reinstall.
Ask any dba. Even with the most heavy duty industrial strength db, somebody can come up with a schema and application that will bring that db to its knees. Prepare for deja vu.
[user] Meet my friends Mr. Debian Boot Disk and Mr. Debian Root Disk
[windows] A priority alert has been dispatched to the BSA.
[Debian Boot Disk] So Boss, do ya want to just rough him up a little or completely murdalize the bum?
"We've been working hard on the next file system for years [since the early 1990's], and -- not that we've made the progress that we've wanted to -- we're at it again," Ballmer said.
While the Cairo project eventually resulted in Microsoft's Windows 2000 operating system, the file system work was abandoned because of complexity, market forces and internal bickering. "It never went away. We just had other things that needed to be done," Jim Allchin, the group vice president in charge of Windows development, told News.com.
Those other things most likely included battling "Netscape and Java and the challenge of the Internet and the Department of Justice," Gartner Group analyst David Smith said--issues that continue to persist today.
<snip>
The more important reasons for the renewed development effort, however, are strategic. If the plan succeeds, it will give Microsoft a huge technological advantage over the competition by making its products more attractive to buyers and giving large companies another reason to install Windows-based servers.
So if they hadn't been trying so hard to kill off Netscape, they would have had the time to spend on creating this. Something that seems to offer actual advantages to the user, and that would be "a huge technological advantage over the competition by making its products more attractive to buyers."
I wonder how many other genuine advances have been put on hold in the name of detroying someone else first.
Nope, no sig
The extension, like the rest of the filename, is metadata. The output of a pipe or socket, in case you haven't noticed, doesn't include that metadata, but most apps getting their data piped in manage without it. For those that can't, most transport mechanisms move that metadata with the file, and so things work all right. If transport mechanisms need to be upgraded to include metadata... okay, we can do that. It's why most good standards are extensible.
The problem is that it's inflexible, and individual file types include their own formats for additional metadata (like id3 tags). Hate to break it to 'ya, but having a whole bunch of little incompatible metadata formats sucks. It sucks a lot. If I want to write a search tool that can look through files' metadata, I don't want to teach it one format for MP3s, one format for Word documents, one format for Joe's Accounting System datafiles... well, you get the point.
Files sharing names but with differing metadata are Bad. That you generally aren't allowed to have them is a Good Thing -- if the file is different enough to need different metadata (like foo.txt and foo.rtf), they should have different names (as foo.txt and foo.rtf do, incidentally -- how handy! -- but doing that for *all* metadata is inappropriate).
You've "debunked" exactly nothing.
I'm told this file system won't need to be de-fragmented.
Oh wait, that email was 7 years old.
- Dan I.
No, NTFS was not a rip-off of OS/2's HPFS. It was an update. Microsoft wrote HPFS (and did a good job), and updated it to NTFS. IIRC most of the updates just made it incompatable with HPFS, but there were probably one or two things done better in their defense.
A lot of people seem to've totally missed the point of what would be different about a database-oriented filesystem. File extensions? Not bloody well relevant! Let's consider the issue of searching. A database-oriented filesystem might allow you to create directories that are basically "views" of your filesystem, perhaps including all files that meet certain name, attribute or content criteria (like Evolution's vFolders but available to any app). These views would be up-to-the-instant accurate at all times, with no dead links and no problem with apps replacing links with actual files instead of updating the file that the link pointed to. Filesystems could also benefit from other things like referential-integrity checks, triggers, and cross-file transactional behavior. In fact, there has been a lot of work in the kernel-hacker community to figure out how just that last feature could be added to Linux filesystems. Basing a filesystem on a database also allows you to leverage all of the tools (e.g. efficient snapshots and replication) that have been developed for the database. There's a lot more here than just journaling and BeOS-style metadata.
It's not that I think basing a filesystem on a database is a great idea. For one thing, it's a pretty good bet that performance is going to suck because of all the extra DB-related overhead. Administration might become more of a PITA too. I'm just trying to explain that the idea of a database-oriented filesystem has much broader implications than the trivial crap (much of which is relevant to neither filesystems nor databases) that people seem to be focusing on in this thread so far.
Slashdot - News for Herds. Stuff that Splatters.
Another unverified, just my personal experience, YMMV tip is removing any files from your desktop. If you store files there, especially large ones, it will slow windows down. Even a large number of shortcuts can have an effect.
Yes, I know you are not supposed to store things on the desktop, but windows makes it far too easy to do so. Plus it has the advantage that once you have downloaded a file, you can see it immediately without having to navigate to the right directory.
First Slashdotter: MS is coming out with a new "revolutionary" file system! *bash* *bash* *bash*
All Slashdotters: Burn it! Burn it!
Rational observer: Well, how do you know it's so bad?
First Slashdotter: Well, it's from...MS!
All Slashdotters: Yeah! Yeah! Burn it! Yeah!
It's 10 PM. Do you know if you're un-American?
MSDE _is_ SQL Server throttled for = 10 users and without the visual tools.
I have a website. It's about Macs.