Slashdot Mirror
Next Windows to Have New Filesystem
ocipio writes: "Microsoft is currently planning a new filesystem. Its planned that the new filesystem will make searches easier, faster, and more reliable. Windows will also be less likely to break, and easier to fix when it does. The new technology will cause practically all Microsoft products to be rewritten to take advantage of it. Called Object File System, OFS will be found in the next major Windows release, codenamed Longhorn. More information can be found here at CNET."
Check that you're not running an Office documents reindex. This is on by default and _despite_ the name, what it does is search every file your document pool and attempts to add it to the index. Very not recommended.
Yes, moving the documents _will_ stop this process, but it's not the most practical method is it? Better, by a long, long way, to turn off automatic indexing of office documents.
Cheers
Chris
The GNOME project has an excellent overview of some of the issues with metatata in general.
Of course, this will mean a whole new found of application incompatibilities on Windows and a whole new round of reverse engineering to determine the filesystem and metadata layout.
From what I understand, Veritas essentially rewrote NTFS version 5 (shipped with win2000 and winXP) and
integrated built-in volume management (dynamic disks) with some abstract layer to maintain the clunky drive letter schemes.
I dont' really see the reasoning by mentioning that all Windows applications will require a rewrite; they only need to abstract the Win32::File APIs to handle the internal OFS changes... the changes that they document appear to do essentially what the Indexing services do for win2000 and winXP now. I assume it is just extra metadata strings that they associate with each file inode.
It seems a bit arrogant of MS to think they can improve (or trash and rewrite) what is essentially Veritas' domain, file systems. But then again, we're talking about MS...
I remember MS talking about this years ago, even before NT came out. They were calling it Cairo back then.
In the end, IIRC, the UI elements of the Cairo project were recycled into the Windows 95 shell and the Object File System concept disappeared entirely... until now, it seems.
Is no-one else disturbed at the short memories in the industry? I was at the launch of Visual Studio.NET in Ireland a few weeks back and there was a Microsoft goon waving a Tablet PC around his head like it was some completely new thing. I mentioned the Go Corp/ Windows for Pen Computing FUD from the early '90s to the guys I was there with and was met with blank stares.
that they can simply take FFS and Soft Updates and embed it in Windows, and call it OFS.
NTFS has features like ACLs, streams, etc that aren't in FFS or UFS. Also, support for transparent compression and encryption, also sparse files. There's support for quotas in the filesystem, and it's quite resistant to the effects of fragmentation. It's journalled and supports Unicode. It's actually a very good filesystem, once of the better parts of NT.
Now for some speculation. The implementation is likely to be based on the same "pluggable" FS driver architecture first introduced in Windows 2000, where the NTFS and FAT32 drivers are just a layer on top of the kernel (you can actually buy a devkit from Microsoft that will allow you to implement, say, ReiserFS for Windows 200). This however poses an interesting question: do you make this newfangled FS to sit on top of tried-and-true NTFS, or do you implement it at the kernel level and make NTFS a layer on top of that? Either way, I think the article is overstating the devastating effect on existing apps. Microsoft is not about to shoot itself in the foot so massively. Whatever this ends up being it's a good bet it will be fully backwards compatible. Kinda like Win32, which can still run 16-bit apps, albeit slower (in 2K and XP). But this will make companies more likely to either port existing code or release newer versions that take advantage of the redesigned FS.
There's a interesting Register article here
Largely due to the migration of much of the needed code into the core OS DLLs. If it's in the OS, the apps are smaller, and they load faster.
After all, wouldn't you rather allocate the memory once when the DLLs are loaded at startup...or repeatedly every time you run the app?
Sadly, OS X uses the glorious idea of file extensions. Metadata for storing file information is only just tolerated. So much for progress.
I just have this feeling that completely replacing the file system can't make it less likely to break. I've had no trouble with FAT32, and I seriously doubt MS's ability to write a filesystem that has no bugs. But maybe I'm just being pessimistic today.
I'm surprised nobody's posted this yet, but ReiserFS is working on something similar, described in this whitepaper.
The SQL server files have to be on a "file system".. so what's THAT going to be?
Not if your database server has its own datastore functionality built in -- and many do. You know how Oracle can work with raw partitions directly? Think 'bout that kind of functionality, just done by MS.
Lets get things straight here.
There is no point of comparision between regular filesystens, and journaled ones. It's not a matter of Win x Linux. I would not dare pull the power cord of a ext2 based computer, but I would do it on a ext3 based one. And what is ext3 besides adding journaling capabilities to ext2 ?
You see, journaling filesystems are all slower than regular ones. Of course, I new developted journaled fs tends to be faster the a filesystem developed 20 years ago (usualy).
On the other hand, it's still possible to lose data on a journaled fs. Not as likely as in a regular one, so don't trust it too much. It's the worst thing you can do.
morcego
BeFS...
BeFS was/is essentialy a database, which in conjunction with features like extended attributes will allow you to store your mail as files in a folder (actually any folder). The same can be done w/ mp3:s where you can store the ID3 tag as file attributes.
All this makes for lightning fast search of mail, mp3:s etc because it's all in the FS, there is no need for looking at the actually contents of a file.
So now, do not say a database as a file system will not be to any use for the consumer...
And, oh, ReiserFS seems to be heading this way too...
The extension, like the rest of the filename, is metadata. The output of a pipe or socket, in case you haven't noticed, doesn't include that metadata, but most apps getting their data piped in manage without it. For those that can't, most transport mechanisms move that metadata with the file, and so things work all right. If transport mechanisms need to be upgraded to include metadata... okay, we can do that. It's why most good standards are extensible.
The problem is that it's inflexible, and individual file types include their own formats for additional metadata (like id3 tags). Hate to break it to 'ya, but having a whole bunch of little incompatible metadata formats sucks. It sucks a lot. If I want to write a search tool that can look through files' metadata, I don't want to teach it one format for MP3s, one format for Word documents, one format for Joe's Accounting System datafiles... well, you get the point.
Files sharing names but with differing metadata are Bad. That you generally aren't allowed to have them is a Good Thing -- if the file is different enough to need different metadata (like foo.txt and foo.rtf), they should have different names (as foo.txt and foo.rtf do, incidentally -- how handy! -- but doing that for *all* metadata is inappropriate).
You've "debunked" exactly nothing.
My company currently uses ObjectStore in several applications for its simplicity. With it, we are able to persist our objects in storage quickly and easily. And since the complete object is stored, we can retrieve it on demand without much hassel. On top of all that, 99% of the administration is transparent to the user.
Relational has its uses, but don't make the blanket statement that OO databases are useless in production environments. Check your facts first.
actually, it doesnt beg the question
No, NTFS was not a rip-off of OS/2's HPFS. It was an update. Microsoft wrote HPFS (and did a good job), and updated it to NTFS. IIRC most of the updates just made it incompatable with HPFS, but there were probably one or two things done better in their defense.
It also has mount points as well as reparse points.
Very few people actually use them, but that doesn't mean they're not there.
Also, say what you want about DOC/PDF, but they have supports for these new and fancy things called fonts and something called layout, which makes them slightly more useful for serious stuff.
As do TeX, LaTeX and various other formats that are stored in plain text.
A lot of people seem to've totally missed the point of what would be different about a database-oriented filesystem. File extensions? Not bloody well relevant! Let's consider the issue of searching. A database-oriented filesystem might allow you to create directories that are basically "views" of your filesystem, perhaps including all files that meet certain name, attribute or content criteria (like Evolution's vFolders but available to any app). These views would be up-to-the-instant accurate at all times, with no dead links and no problem with apps replacing links with actual files instead of updating the file that the link pointed to. Filesystems could also benefit from other things like referential-integrity checks, triggers, and cross-file transactional behavior. In fact, there has been a lot of work in the kernel-hacker community to figure out how just that last feature could be added to Linux filesystems. Basing a filesystem on a database also allows you to leverage all of the tools (e.g. efficient snapshots and replication) that have been developed for the database. There's a lot more here than just journaling and BeOS-style metadata.
It's not that I think basing a filesystem on a database is a great idea. For one thing, it's a pretty good bet that performance is going to suck because of all the extra DB-related overhead. Administration might become more of a PITA too. I'm just trying to explain that the idea of a database-oriented filesystem has much broader implications than the trivial crap (much of which is relevant to neither filesystems nor databases) that people seem to be focusing on in this thread so far.
Slashdot - News for Herds. Stuff that Splatters.
This is really quite a scary development. I used to be an Oracle/ MS SQL Server DBA. Doing this is a (in some respects) brilliant tactical move for MS, but a disaster for users potentially. It could also be very dangerous. Adding database technology to the filesystem in the OS gives them a lot of power over your computer:
- makes it easier to kill off oracle
(I bet the real reason they stopped doing this was that the DOJ would have viewed this as anticompetitive behavior during the MS investigations). Why buy oracle when you get a MS database for free?
- makes it really easy to spy on the users of the OS. Database triggers and stored procedures
could be written so that every time you save an MP3 on your hard drive, it checks to see if it is copyrighted material and if so sends the info to the RIAA (for example). You will not even know these triggers run.
This is no worse than having some spy softwarein existing FS related apis, but with a SQL database in the back, the quality of information would be much better, and the risk is much higher, especially if software is allowed to install their own triggers into this filesystem. Imagine the havoc then. Now imagine tying that fuctionality into IE so that such triggers and such can get installed remotely over the web without your consent. Imagine the viruses you can get then. And good luck at figuring out what is there - it will be impossible. If you thought the windows registry was hard to manage, well, this will be an order of magnitude worse.
I am not sure if I would want this technology built into my OS. It is too dangerous if it is not implemented well, and I don't trust MS enough to believe that they (or someone else if others can put triggers into your os) will not attempt to spy on their users. The upside is that SQL gives really great query capability and maybe searching for files will be 5% easier. Is it worth it?
My hidden files summed up to a couple of hundreds of megs of wasted diskspace from my primary disk...
The owls are not what they seem
It's not perfect, but it relies on magic(5), the hints file with signatures of the various types of files. If you have a small magic(5) file you get stuff wrong, and some signatures need to be made "deeper" or smarter tests built in so that otherwise similar files can be told apart.
I can see where it would be advisable to have your file manager or desktop GUI be able to update the magic(5) file by being told that one or more of the files are different and that better signatures need to be generated -- eg, find out what's the same about files a and b but isn't the same as c and d.
Anything is naturally going to have some glitches, and some choices will always need to have an arbitrary definition since the definition of what they are may vary depending on interpretation.
I just think that the overwhelming majority of files *will* be machine-typable based on contents with hints and that adding a lot of extra data to the filesystem will cripple its speed in the long run.
I think the major change from HPFS to NTFS was that HPFS was only capable of locking down permssions to the directory level, and NTFS could do it to the file level.
"History doesn't repeat itself, but it does rhyme." Mark Twain
I think you are right, at least in part. I think that the other issue to consider is the close relationship between WNT and VMS.
The last major release of VMS with a major improvement was back in about '91 when they added in the low level support for transactions in the file system. This was a truly major breakthrough. it is a pity that it came towards the end of the VMS era and as a result was never really exploited by applications.
The advantage of a transaction based file system is that you can write updates out to disk without having to worry about partial completion. You know that the write will either succeed completely or fail completely. It is not possible for the program to crash in mid write and leave the data in a corrupted state.
This would be a major advantage for application programmers. You could get ACID properties without having to mess arround changing your data structures to use SQL. You would not need to deal with the twits who think that Entity Relationships are the latest thing in comp sci rather than what they really are, an obsolete throwback from the COBOL era. You would not need UML or any of that rubbish either. And you can hand your Oracle DBA their pink slip.
All in all a win win situation, unless that is you are an Oracle DBA or hold Oracle stock.
Looking for an Information Security student project suggestion?
Try http://dotcrimeManifesto.com/
And 30 years ago the bleeding-edge filesystems (non-UNIX) did have metadata and resource forks and other whizbang ideas. Guess what? They sucked. You can't pipe a multistream file. You can't send it over the network. You can't dump it to tape. First you have to convert your whiz-bang metadata enabled file into a "bag of bytes" file. This means you need to manually add an additional "convert to bag of bytes" step before you do anything unusual with a file. UNIX removes the complexity by forcing the applications to save into the "bag of bytes" file format from the start. The metadata is still within the file, but the application put it there, and the application knows how to get it out.
The UNIX people chose simplicity over complexity. History has shown that if people want metadata then they can implement it inside their applications (ID3 tags, TAR, GZIP headers). This is one of the many reasons why UNIX still survives: it dictates the means to store data, without enforcing a policy about what data is stored. Applications can then grow and evolve to include new-and-unthought-of metadata. This would not be possible if the UNIX filesystem had forced a restricted set of metadata onto everybody from the start.
People who don't learn history are doomed to repeat it.
I think:
- enough people who know a thing or two more about this than me are talking about it
- now MS is talking about creating an Object File System
- and we know SQL Server research has been going into making it handle less structured data more effectively
I think it all points to credibility. That's not to say there won't be some add-on for additional SQL Server functionality, but I think it's believable that the core functionality of storing and retrieving objects in OFS will come from SQL Server.MSDE _is_ SQL Server throttled for = 10 users and without the visual tools.
I have a website. It's about Macs.
Ummmm, it certainly will if you do "xhost +localhost" or whatever.
Better yet, disable the Active Desktop. (Uncheck "view as web page".) If you just have to have a jpg/gif as a background, convert it to a bitmap and set it up the traditional way.
When AD is installed, it uses processor power to generate the desktop view (refreshing the screen). When using a bmp the image is cached in video ram and the video card is responsible for refreshing the desktop view. After all, that's what it's made for, but AD basically runs your desktop in Internet Explorer in a hidden window. At least that's what I'm guessing.
If you don't understand what I just said, you are too stupid to use a computer. Buy a pencil.