Slashdot Mirror
Next Windows to Have New Filesystem
ocipio writes: "Microsoft is currently planning a new filesystem. Its planned that the new filesystem will make searches easier, faster, and more reliable. Windows will also be less likely to break, and easier to fix when it does. The new technology will cause practically all Microsoft products to be rewritten to take advantage of it. Called Object File System, OFS will be found in the next major Windows release, codenamed Longhorn. More information can be found here at CNET."
Yes, I'm cynical. But really, why shouldn't I?
:Peter
political_news.c: warning: comparison is always true due to limited range of data type
I hope this doesn't cause as many problems as the last feature they added.
Note the historical sidebar on the article. It traces the on-again, off-again history of OFS. MS has been playing with it for over half a decade (!), and doesn't yet have anything to show for it. They've backpedalled and caught up again so many times that I think this article can be safely labelled as speculation.
In other words, it sounds cool. I'll believe it when I see it. (and only at that point judge whether it really makes Windows less likely to break)
"People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban
Check that you're not running an Office documents reindex. This is on by default and _despite_ the name, what it does is search every file your document pool and attempts to add it to the index. Very not recommended.
Yes, moving the documents _will_ stop this process, but it's not the most practical method is it? Better, by a long, long way, to turn off automatic indexing of office documents.
Cheers
Chris
To be honest, NTFS seems to be a tip-top file system to me. The only thing I can imgaine it missing is hardcore digital rights management (cant wait).
What a clever way to force DRM down every consumers throat: break every single windows program created prior to OFS.
fuckers.
(2,3-Benzopyrrole)
It looks like BeFS with XML descriptions instead of MIME types. I think.
[o]_O
From the article:
Replacing its antiquated file system with modern database technology...
Now, if you were going to base a file system on a DB, what would you use? An Object-Oriented DB? Where organization is key (which you want for a file system), or a Relational DB for speed (which is why they are claiming to switch)?
I'm sure they are going to make a custom system, yes, but wouldn't it have to be based on one of the two major DB designs?
For the record, I'm no DB Admin, but, as I understand it, relational is the choice of DB for almost all projects for its sheer speed, OO is only good for academic reasons to show off organization...
Good quote, too many chars. Seriously, the slashdot 120 char limit sucks!
The GNOME project has an excellent overview of some of the issues with metatata in general.
Of course, this will mean a whole new found of application incompatibilities on Windows and a whole new round of reverse engineering to determine the filesystem and metadata layout.
From what I understand, Veritas essentially rewrote NTFS version 5 (shipped with win2000 and winXP) and
integrated built-in volume management (dynamic disks) with some abstract layer to maintain the clunky drive letter schemes.
I dont' really see the reasoning by mentioning that all Windows applications will require a rewrite; they only need to abstract the Win32::File APIs to handle the internal OFS changes... the changes that they document appear to do essentially what the Indexing services do for win2000 and winXP now. I assume it is just extra metadata strings that they associate with each file inode.
It seems a bit arrogant of MS to think they can improve (or trash and rewrite) what is essentially Veritas' domain, file systems. But then again, we're talking about MS...
Everyone would have to buy new versions of all their office software! Isn't that handy for MS?
I'll pass. I may be running (pre-installed) XP on my Dell but I'm still using Office 97. Why?
BECAUSE IT WORKS JUST FINE.
I don't need to "upgrade" to something even more bloated and bug ridden.
How long do ./ readers think it will be until the Linux kernel and/or Samba will be able to read OFS shares?
I don't think that this is what they are after. It is possible that they will have to open their file i/o api's soon because of the anti-trust case here and in the EU. I truly (naively?) believe that they are moving to a better filesystem because it's a better filesystem, not because they want to break interop between *nix and MS. I'm also an eternal optimist.
I remember MS talking about this years ago, even before NT came out. They were calling it Cairo back then.
In the end, IIRC, the UI elements of the Cairo project were recycled into the Windows 95 shell and the Object File System concept disappeared entirely... until now, it seems.
Is no-one else disturbed at the short memories in the industry? I was at the launch of Visual Studio.NET in Ireland a few weeks back and there was a Microsoft goon waving a Tablet PC around his head like it was some completely new thing. I mentioned the Go Corp/ Windows for Pen Computing FUD from the early '90s to the guys I was there with and was met with blank stares.
Gimme a D, gimme a R, gimme an M. . . /. ads ;)
What's it spell?
I also see my ad filter is working on the new
1q2w3e4r5t6y7u8i9o0pqawsedrftgthyjukilo;p'azsxdcf
Rather a good thing to know.
dinner: it's what's for beer
I hope any changes that happen to the file system also include the removal of the antiquated concept of file extensions for type association. Here is another thing that Mac does very well. Imbed the type of a file IN the file. Why not give me a version number and some way to know what program created it.
Back to the original topic, I can't wait for an OFS. Just for my MP3's. Figuring out which folder hierarchy to use for genre/group/album/track is a pain. Let the file system group them for me.
A speech...
and the FAT32 took a major hit.
XP is an NT-based OS...why were you using FAT32 at all when NTFS is available?
C-X C-S
I hope they haven't discovered the unified file system, something which all the unixes use to store their data. We might be looking at the demise of unix!
oh wait, where do we put Progra~1 then?
- runs away to see if it's patentable
Yes, you are a troll. Is it wrong for Microsoft to advance File systems and state specific reasons and right to preach about the many choices in file systems linux/unix has?
When your talking
Say when you open a folder of 5,000+ mp3's and it searches the ID3 tags of every song and displays artist/title as part of the description, having an optimized file system for quicker searches of data on the disk will only streamline this more.
so yes, this is cool, and yes, there is alot more then just "fast searches" as you put it.
Completely agreed. This extends the OO nature of Windows down to the FS level. .NET extends it up to the network level. It's a huge play by MS and it's a huge step forward.
If only Windows Scripting Host gave you a more dead-easy way to script/tinker with the Windows objects...
... right before the sledgehammer hits.
You can achieve this goal by the following process:
1. Store all of your documents in a simple, text based format, and not in some overly complex propriatary format such as ".DOC", ".PDF", etc.
1. This text based format is known as "American Standard Code for Information Interchange" (AKA "ASCII")
2. If you require more complex presentation of information, you might want to use something called HyperText Markup Language. (Which doesn't do much markup these days... but I digress)
3. There is a program built in to Windows 98 and above called "Find" (usually accessed by hitting F3), and in other environments known as "grep" which can search by content.
Use the tools you have, you won't have to "upgrade" to the latest bugs, and the computer remains useful.
--Mike--
that they can simply take FFS and Soft Updates and embed it in Windows, and call it OFS.
NTFS has features like ACLs, streams, etc that aren't in FFS or UFS. Also, support for transparent compression and encryption, also sparse files. There's support for quotas in the filesystem, and it's quite resistant to the effects of fragmentation. It's journalled and supports Unicode. It's actually a very good filesystem, once of the better parts of NT.
Early versions of BeOS had a full object orientated file system and found performance was abysmal. This was from a company with no backwards compatibility to worry about and a small OS designed for speed.
In the end Be developed BFS which is basically a standard file system with support for indexes and attributes, an overall much better performing system with most of the benefits of an object orientated file system.
[)amien
[user] Open Excel document "Personal_Finances"
[windows] Please log into Passport so the $0.50 file usage fee can be charged to your credit card.
[user] But this is my file, I created it.
[windows] NO, I store your crap file, it is mine.
[user] THAT's IT, I'm formatting you!!
[windows] Please log into Passport so the $999.99 reformatting fee can be charged to your credit card
[windows] And have a nice day!
-- Find the Truth...
Hey trollie, read the article. Or better yet, the Slashdot posting. This file system has nothing to do with DRM.
.. boy, don't even get me started on that.
Firstly, there is no such thing as "Microsoft Digital Rights Management Software" (Media Player supports DRM, but only for WMA's) and Microsoft has nothing what-so-ever to gain from including DRM features into the file system. They know and we know that Longhorn with DRM will go down the toilet, while Longhorn without DRM will sell just as well as WinXP, probably better.
The second thing you got wrong is that this system is not (just) about speeding up searches. It's about replacing an antiquated system that's been around since MS-DOS with something future-proof, faster, and more reliable. Considering they've been working on this for 10+ years, they'll probably succeed eventually. And when they do
Now, for something constructive. When will we see this in Linux? Surely, if Microsoft can do this, so can the people working on Linux. Riiight?
Quality, performance, value; you get only two, and you don't always get to pick.
Everyone would have to buy new versions of all their office software!
Ummm...Why?
Changing the FS would really only affect the way the data is stored on the drive, the AppFS interface should be abstracted by the OS.
C-X C-S
Maybe the rest of the world has a short memory, but I don't.
According to what Microsoft had announced some time ago, they are HALTING further new development while they focus on bug fixes. Can I then take this as a sign they've fixed all the problems with Windows?
So now they are working on new developments again? I'm pretty sure existing Windows is not completely stabiized...I know mine isn't. So what does it mean? Micrsoft can't maintain focus? They can't tell the truth? What?
Now for some speculation. The implementation is likely to be based on the same "pluggable" FS driver architecture first introduced in Windows 2000, where the NTFS and FAT32 drivers are just a layer on top of the kernel (you can actually buy a devkit from Microsoft that will allow you to implement, say, ReiserFS for Windows 200). This however poses an interesting question: do you make this newfangled FS to sit on top of tried-and-true NTFS, or do you implement it at the kernel level and make NTFS a layer on top of that? Either way, I think the article is overstating the devastating effect on existing apps. Microsoft is not about to shoot itself in the foot so massively. Whatever this ends up being it's a good bet it will be fully backwards compatible. Kinda like Win32, which can still run 16-bit apps, albeit slower (in 2K and XP). But this will make companies more likely to either port existing code or release newer versions that take advantage of the redesigned FS.
There's a interesting Register article here
It really is an interesting problem. My Wife's iMac only has a 6Gb drive. She's always saving info form the web into AppleWorks files. She has generating a LOT of little itty bitty files. Now our PC file server stores our digital photos and mp3s, and both iTunes and iPhoto make managing that mess quite easy and Sherlock is SUPPOSED to make finding in files easier (it kinda does), but does a poor job at it.
Now, my point is, I've actually thought about setting up some form of database so that my Wife can find her info for years to come. But my biggest question is NOT would a database help, I'm sure it would. What I would like to know, is how would the interface for that database look?
Considering what I have seen of XP (I got a copy sitting on a 2GB drive that sits on my shelf), MS knows very little about information management in the UI, and I would expect this problem to not get any better for the majority of PC uses, even if the entire file system was one big database.
Burn Hollywood Burn
The point of this new data store isn't necessarily faster searches, although that is one part of it. The idea is to have a common data storage mechanism, used by all programs.
The underlying technology is to replace the NTFS filesystem driver with SQL Server, with a few tweaks. SQL Server already supports using a RAW partition as a data store, so essentially you just have to move the transaction log and descriptive info for the databases into a specific area of the disk. Add a little bit of bootstrap code to ntldr, and slap the SQL Server stuff into the startup driver list, and it's a done deal.
The next step is creating an NTFS compatibility layer -- it would allow you to mount tables as drive letters or network shares. A lot of the information wouldn't be useful when viewed in that fashion, but it would give you a way to run older programs.
Once all your data is in a common data store and can be manipulated as such, it opens up a world of new possibilities. The change will be long and slow; no need to kid about that. It will take years for all the 3rd party programs (and even Microsoft's own apps) to catch up and start taking full advantage of it. It's the same situation Plug & Play was in back in 1995; it sorta worked sometimes, but you couldn't really take full advantage of it. But here in 2002, you really can expect to grab a piece of hardware and slap it in your box without hassles. It took some time, but it eventually paid off.
But... are you having trouble, as I did, thinking of ways to make use of this common data store? Part of that comes from the fact that we've been conditioned and trained to think of data storage in terms of files; it's hard to shift gears... to think outside of the "filesystem" box so to speak.
For one thing, I could see someone emailing me a project. Not some word documents, an excel spreadsheet, and a database zipped into a ZIP file; they just email me the project. When I get it, and open the message, the project opens up presenting me with the various documents (linked to the database of phone numbers for example), and a little yellow stickynote window that has the project leader's actual email text. I didn't have to deal with unzipping the data, rearranging it, then opening the documents separately. Since the "rows" are linked, they open and act as a unit until I tell them to do otherwise.
It gets better though... imagine if I could run a query such as "SELECT f.*, s.filename FROM Folder1 f INNER JOIN folder2 s ON f.datetime = s.datetime"
It can get even more useful because you now have full SQL syntax available to you for manipulating the filesystem, with queries that are lightning fast. Throw in some Stored Procedures, Functions, Views, etc and I can see real possibilities.
Natural != (nontoxic || beneficial)
And IBM's AS400's have been doing this for years. Not to mention BeOS. ReiserFS is of perticular intrest because it will allow for attaching of arbitrary objects to any node. Only problem is we have five next generation file systems duking it out so generic Linux will most likely not see the benifits for some time as nobody will want to program specificly for a filesystem that reaches only a fraction of the usebase. It would be sure nice though to not to have to see .nautilus-metafile.xml stewn about my file system. sigh! What would be nice is generic system calls for filesystem metadata that would write out .nautilus-metafile's for FS that don't support metadata and node metadata for FS's that do. Of course we would need a standard format but it would be instantly useful.
The submission form is acting weird so here is the link again: http://www.pbs.org/cringely/pulpit/pulpit20010802. html.
This New File system sounds to me like something similar.
They proved with AS/400 that using a DB for the file system was the way to go. It's too bad they did it way ahead of their time.
I'm personally glad MS is finally changing their OS. Now that my workstation has 70GB of files, searches are taking an incredibly long time.
I have less than 100,000 files on my workstation. Each has maybe 10 searchable attributes. A full search on this can take over five minutes. (Athlon 800Mhz w/ 7200 RPM IBM drives on a Promise controller)
I know from experience that querying an Oracle database (on a cheap 500mhz linux box) on 100,000 records with 30 non-indexed columns/attributes generally takes around 2-3 seconds.
Imagine if MS were able to build a file system with such capabilities.
> They want to get their Digital Rights
.NET and start dropping the crufty Win16-contaminated Win32 API and x86 ties.
> Management Software to infest every aspect of
> their OS as possible.
Right. You keep throwing your FUD about while the rest of us looks at things seriously.
> Do you honestly believe that the benifit of a
> faster search is enough incentive to rewrite
> such a major part of the OS?
Short Answer: Yes.
Long Answer: Filesystems haven't changed much in the past few decades, but one of the things they have tended to gain is arbitrary metadata. Adding indexing to that metadata is a natural progression of that.
Now your filenames are just a part of the metadata you'll want to play with different views of the system, which suddenly becomes much much cheaper. Believe it or not, lots of users have trouble understanding the current basic filesystem concepts and using them to organise their data; well, now you can do it automatically for them.
Of course, you want your other stuff to make use of these new ways of looking at the system, especially when you're MS and are running out of new features to put in (come on, what are they going to add to Word XP now? A paperclip with speech recognition? Yet another GUI redesign?), so you want to do something that provides a visible difference (and maybe even an advantage) for those expensive upgrade programmes.
So, yes, I think they do have a very good reason for such a major change, like they had good reason to introduce '95 and start dropping DOS, or NT and start dropping Win16, or
The biggest issue I have with it is that it's going to be a bitch to use in other OS's. Hopefully they'll do detailed specs and stick to them fairly closely (ah haha), which will at least make it easier.
Everthing is a file, says Unix.
But that was 30 years ago. Perhaps its time to extend the unix-doctrin: Everthing is a file and a directory.
Why? Metadata.
Todays file-formats store informations about the file inside the file (thing id3-tags) or abuse file-attributes (such as the filename(.html)).
With files as file and directory, there would be no need for that. Imagine: you store informations about the authors of a file inside metadata-attributes. There would be simple possibilities to search for these informations, so one could easy pick up all "draft" files inside a direcory (ls -al *../status=draft, maybe)
p.
NTFS is a very solid filesystem and seems to recover problems well when something bad does happen. The only complaints I have are slow searches and reports. It takes a LONG time to find a file on a big volume, or try and do reports on file system usage. A good database system should speed that up tremendously.
The idea of having to rewrite the apps is interesting though. That tells me this is at least 5 years off, and longer before it would be used widescale. But I guess that makes sense, would you be the first shop to put your big fileserver on a new filesystem like that? Not me.
In the process, the plan could boost Microsoft's high-profile .Net Web services plan and pave the way to enter new markets for document management and portal software, while simultaneously dealing a blow to competitors."
OK I know FAT is antiquated, but NTFS is modern. In fact I recall it was announced at some point 3-4 years ago that OFS wasn't necessary because all the relevant features were being merged into NTFS? Maybe that was an internal announcement, one of the annual "we are finally merging our data stores" emails the top Microsoft brass would send out to the troops.
Anyway I don't see why this would make Windows less likely to break or easier to fix, or what it has to do with .NET...why does that kind of marketing fluff have to be included in a pretty reasonable article (and the sidebar is very nice)?
- adam
NTFS has features like ACLs, streams, etc that aren't in FFS or UFS. Also, support for transparent compression and encryption, also sparse files. There's support for quotas in the filesystem, and it's quite resistant to the effects of fragmentation. It's journalled and supports Unicode. It's actually a very good filesystem, once of the better parts of NT.
Right. This begs the question of why bother ?
The push et al is just a load of hype to push the upgrade path. They are going to engineer a database into their file system "to make searches faster" because doing it the slocate way would not force another round of complete system upgrades on consumers.
You may have also noticed that Outlook and Office will need to be rewritten to "take advantage" of the new file system. So not only will they leverage OS upgrades, but Office upgrades as well. They are planning to rip out a perfectly good file system (which is called "antiquated" in the article) to make billions of dollars, and the press releases are all about consumer benefit.
And consumer benefit, as you have noted, is essentially nil.
It is funny, we accuse Microsoft of using other people's ideas - but are we really any better? How much of Open Source development is really just reimplementations of other people's ideas?
When I read this article, I immediately had two thoughts:
Thought 1: "You know, they're right" Current file systems are outdated and are not really serving the needs of modern applications. Take for example, Microsoft Outlook (and Outlook Express). The programming teams for these pieces of software were forced to implement a "filesystem within a file" in order to achieve their design goals (I believe the files are called DBX files). Or take for instance, the Windows Registry, or, even better, the Gnome registry, GConf. Why do programmers have to implement dozens of different abstract filesystems in order to achieve their design goals? Simple, the present filesystems are not sufficient.
Thought 2: "Another way of attacking the Free Software Movement." By creating a new filesystem, Microsoft achieves many goals. First, they make Linux filesystem developers start from scratch again. I mean, the NTFS driver isn't even done, and this means we would have to start over. It gets even worse: From the sound of this article, it seems that OFS would be fundamentally incompatible with our conception of a filesystem today (possibly including features such as resource branches, GUID tags, and other metadata forks, ad nauseum). This would make it difficult to write a usable Linux driver for OFS. And finally, to top it off, my gut tells me that the POSIX file access calls would _not_ be sufficient to access such a rich filesystem. The introduction of a new, richer file access API by Microsoft would make writing cross-platform software much more difficult.
Microsoft can kill two birds with one stone here.
Ben
The SQL server files have to be on a "file system".. so what's THAT going to be?
Not if your database server has its own datastore functionality built in -- and many do. You know how Oracle can work with raw partitions directly? Think 'bout that kind of functionality, just done by MS.
If you take a look at the XP interface, it feels (to me at least) a lot like a candied up BeOS -- a lot of the icons have a similar look, there's the grouped taskbar items a la the BeOS tracker, etc. And seeing as BeOS has been around for years, it makes a lot more sense that the Microsoft engineers would have been able to start reimplementing ideas like this by this point.
And now we start seeing articles like this one, and it becomes clear that just as the XP interface has started to resemble BeOS, the XP native filesystem is starting to resemble BFS. This isn't the first time in recent months that we've seen reports of this -- not long ago there were articles saying that MS wanted to ditch Access and it's Jet engine (or whatever it runs now), and turn the SQLServer engine into the core of the next generation filesystem. This is of course exactly what Be wanted to do, but couldn't due to performance constraints, so they went with the scaled back object oriented system instead. Hey look at that, now we hear that Microsoft is also going with an OO-FS instead of a full SQL-FS.
Microsoft already ran Be out of the market, and are rightfully getting sued now for doing so. I wonder if Be would be willing to use this increasingly familiar evolution for Windows as evidence that Microsoft wanted to eliminate their strongest OS competition while ripping off all their good ideas. As much as it's vindicating to see that BeOS's best features will live on in new versions of Windows, I'd rather have the chance to see the original around today...
DO NOT LEAVE IT IS NOT REAL
Look at it this way - some of us may wish that the whole world used Linux, but it doesn't. It uses Windows. So when MS announces that they're taking a big risk (and it is a big risk) to try and make such an enormous upgrade to Windows, I think we should be happy that a few years down the road, if Windows is still dominant then at least people will be benefiting from this technology.
But ... this doesn't mean we should just sit back and go - well done Microsoft! After all, I recall reading about something similar over at the reiserFS page... how long until Linux users get this technology also?
That's what they said about the registry. It will solve all of the problems with ini files.
But as everyone knows, with totally undisaplined usage of the registry, the registry is a nightmare. In some cases it is impossible to clean it up and the only solution is a reinstall.
Ask any dba. Even with the most heavy duty industrial strength db, somebody can come up with a schema and application that will bring that db to its knees. Prepare for deja vu.
[user] Meet my friends Mr. Debian Boot Disk and Mr. Debian Root Disk
[windows] A priority alert has been dispatched to the BSA.
[Debian Boot Disk] So Boss, do ya want to just rough him up a little or completely murdalize the bum?
"We've been working hard on the next file system for years [since the early 1990's], and -- not that we've made the progress that we've wanted to -- we're at it again," Ballmer said.
While the Cairo project eventually resulted in Microsoft's Windows 2000 operating system, the file system work was abandoned because of complexity, market forces and internal bickering. "It never went away. We just had other things that needed to be done," Jim Allchin, the group vice president in charge of Windows development, told News.com.
Those other things most likely included battling "Netscape and Java and the challenge of the Internet and the Department of Justice," Gartner Group analyst David Smith said--issues that continue to persist today.
<snip>
The more important reasons for the renewed development effort, however, are strategic. If the plan succeeds, it will give Microsoft a huge technological advantage over the competition by making its products more attractive to buyers and giving large companies another reason to install Windows-based servers.
So if they hadn't been trying so hard to kill off Netscape, they would have had the time to spend on creating this. Something that seems to offer actual advantages to the user, and that would be "a huge technological advantage over the competition by making its products more attractive to buyers."
I wonder how many other genuine advances have been put on hold in the name of detroying someone else first.
Nope, no sig
BeFS...
BeFS was/is essentialy a database, which in conjunction with features like extended attributes will allow you to store your mail as files in a folder (actually any folder). The same can be done w/ mp3:s where you can store the ID3 tag as file attributes.
All this makes for lightning fast search of mail, mp3:s etc because it's all in the FS, there is no need for looking at the actually contents of a file.
So now, do not say a database as a file system will not be to any use for the consumer...
And, oh, ReiserFS seems to be heading this way too...
The extension, like the rest of the filename, is metadata. The output of a pipe or socket, in case you haven't noticed, doesn't include that metadata, but most apps getting their data piped in manage without it. For those that can't, most transport mechanisms move that metadata with the file, and so things work all right. If transport mechanisms need to be upgraded to include metadata... okay, we can do that. It's why most good standards are extensible.
The problem is that it's inflexible, and individual file types include their own formats for additional metadata (like id3 tags). Hate to break it to 'ya, but having a whole bunch of little incompatible metadata formats sucks. It sucks a lot. If I want to write a search tool that can look through files' metadata, I don't want to teach it one format for MP3s, one format for Word documents, one format for Joe's Accounting System datafiles... well, you get the point.
Files sharing names but with differing metadata are Bad. That you generally aren't allowed to have them is a Good Thing -- if the file is different enough to need different metadata (like foo.txt and foo.rtf), they should have different names (as foo.txt and foo.rtf do, incidentally -- how handy! -- but doing that for *all* metadata is inappropriate).
You've "debunked" exactly nothing.
I share your concerns about interoperability and the upgrade cycle. However, this move is a good thing. All the major OSes out there need to think about file systems as more than just a filename and data. This is because humans are capable of doing more than that, and we shouldn't be limited by what computers used to be able to do.
For example, think of a boring filing cabinet. While you can just create dividers and add more filing cabinets, humans do much more than that. For example, at my doctor's office, they color code each folder. In addition, they apply stickers to the tab on the folders for certain indicators (last update of the file by year, insurance type, which doctor I usually see, since there are several there, etc). As a result, simply by scanning the shelves, they can tell a number of things about the files without having to pull them out and open them up.
Same thing about the FS. It would be nice to be able to tell something about the file without having to issue the open call. That's a good thing. Currently, most apps limit themselves to one hint (the extension). What's wrong with more?
Everything you pointed out in your message, while valid questions, are mostly elementary engineering problems. For example, the two file/different extension problem can be solved a number of ways (MacOS already has to deal with this condition, for example).
In the future, it might be better to have RFCs for an new, standard FTP or whatever that allows a metadata section as part of the DATA transfer. This wouldn't be too hard either (HTTP already could get away with this, since you can define whatever headers, more or less, that you want).
The real concern is interoperability. I can imagine a "compatibility mode" for network aware services, like file sharing or FTP/Web, that present file names to the remote user in the old filename.extension format. That's actually almost trivial.
Namesys is already working on reiserfs (which does something similar). BeOS had something similar, too. NTFS already ran into part of the problem (which stream do you want). We shouldn't hold back because it might require some compatibility for a short period...
Realize that I'm a big skeptic when it comes to Microsoft... I'm worried that they won't do anything to help non-Windows interop. But as an idea, I'm all for the updated FS. To borrow your own phrasing, a file is a file, but I want more hints to help user applications like searching through files (how do you know a file contains an ID3 tag?)
But I like this concept.
Sujal
politics, food, music, life: FatMixx
A lot of people seem to've totally missed the point of what would be different about a database-oriented filesystem. File extensions? Not bloody well relevant! Let's consider the issue of searching. A database-oriented filesystem might allow you to create directories that are basically "views" of your filesystem, perhaps including all files that meet certain name, attribute or content criteria (like Evolution's vFolders but available to any app). These views would be up-to-the-instant accurate at all times, with no dead links and no problem with apps replacing links with actual files instead of updating the file that the link pointed to. Filesystems could also benefit from other things like referential-integrity checks, triggers, and cross-file transactional behavior. In fact, there has been a lot of work in the kernel-hacker community to figure out how just that last feature could be added to Linux filesystems. Basing a filesystem on a database also allows you to leverage all of the tools (e.g. efficient snapshots and replication) that have been developed for the database. There's a lot more here than just journaling and BeOS-style metadata.
It's not that I think basing a filesystem on a database is a great idea. For one thing, it's a pretty good bet that performance is going to suck because of all the extra DB-related overhead. Administration might become more of a PITA too. I'm just trying to explain that the idea of a database-oriented filesystem has much broader implications than the trivial crap (much of which is relevant to neither filesystems nor databases) that people seem to be focusing on in this thread so far.
Slashdot - News for Herds. Stuff that Splatters.
Let's see...Office uses "filesystems within files" named "OLE"...Microsoft is re-vamping the filesystem for Windows...the new Windows FS standard is named "OFS"...see a connection? Hint: if all your major apps are already effectively implementing their own filesystem in userspace (read: slowly), why not move those capabilities into the kernelspace drivers?
Personally, I think this is the right direction to go; just like Reiser's papers on "next-generation" versions of ReiserFS (with keyword and metadata searching built-in), along with the extensive work being done of VFS abstractions for almost every OS. Hell, you could go back to Be's attribute indexing on BeFS, or even the old MacOS resource fork.
Basic UNIX filesystem trees are far from the last word in this area; if they were, no one would need MySQL to support all their webapps. (Actually, it would be interesting to get the MySQL guys involved in some filesystem design work; I'm sure they would have an interesting perspective to offer.)
:)
hawk
Another unverified, just my personal experience, YMMV tip is removing any files from your desktop. If you store files there, especially large ones, it will slow windows down. Even a large number of shortcuts can have an effect.
Yes, I know you are not supposed to store things on the desktop, but windows makes it far too easy to do so. Plus it has the advantage that once you have downloaded a file, you can see it immediately without having to navigate to the right directory.
First Slashdotter: MS is coming out with a new "revolutionary" file system! *bash* *bash* *bash*
All Slashdotters: Burn it! Burn it!
Rational observer: Well, how do you know it's so bad?
First Slashdotter: Well, it's from...MS!
All Slashdotters: Yeah! Yeah! Burn it! Yeah!
It's 10 PM. Do you know if you're un-American?
It's not perfect, but it relies on magic(5), the hints file with signatures of the various types of files. If you have a small magic(5) file you get stuff wrong, and some signatures need to be made "deeper" or smarter tests built in so that otherwise similar files can be told apart.
I can see where it would be advisable to have your file manager or desktop GUI be able to update the magic(5) file by being told that one or more of the files are different and that better signatures need to be generated -- eg, find out what's the same about files a and b but isn't the same as c and d.
Anything is naturally going to have some glitches, and some choices will always need to have an arbitrary definition since the definition of what they are may vary depending on interpretation.
I just think that the overwhelming majority of files *will* be machine-typable based on contents with hints and that adding a lot of extra data to the filesystem will cripple its speed in the long run.
For every "denied" message you get as an admin, chances are you can give yourself access to do this.
An administrator (sysadmin, root, whatever) should never be denied access to anything ever, no way, no how, zip, zilch, nada.
Without full access to the machine, and every resource on it, it is impossible to properly administrate it.
"Permission Denied" shouldn't even exist for the administrator account.
I wonder how many R&D projects get canned in MS. Probably a whole lot. And considering building a new FS is nowhere near innovative. Are you suggesting that MS thought up the idea of this new type of FS? Puhleeze.
My beliefs do not require that you agree with them.
Now if we were talking about Microsoft coming out with a new obfuscated replacement for SMB (which is an evil hack and needs to be replaced with something less thoroughly bletcherous anyway), then you might have a point. But we're not talking about that at all.
--Ford Prefect
Not if your database server has its own datastore functionality built in -- and many do. You know how Oracle can work with raw partitions directly? Think 'bout that kind of functionality, just done by MS.
:-)
I'd rather not. I barely tolerate vfat only because I dual boot for games.
Humorless sig goes here.
Me: What happen?
Windows XP: Somebody set up us the upgrade.
Me: What !
Me: New operating system turn on !
Me: It's you !!
Windows NG (next generation): How are you gentlemen !!
Windows NG: All your file are belong to OFS.
Windows NG: You are on the way to .NET and DRM.
Me: What you say !!
Windows NG: Your mp3 have no chance to survive make your time
Windows NG: Have a nice day.
Me: Take off every boot disk
Me: Load Ranish Partition Manager
Me: Move Mandrake 12.0 install DVD
Me: For great justice
--
Don't throw the baby out with the bath water.
A "file" expresses a fundamentally useful idea: a clear demarcation of data that lives independent of the host filesystem. Once you start tieing and interweaving data tightly with the host filesystem, how do you export it without a significant, altering transformation?
That is, when someone "just emailed you the project", what did you get? How much of the filesystem did or didn't come along with it? Have we openned the door for Version Hell? Also, can the data be compressed without having to know that it is?
Let's just be careful to clearly define what we want and how we get it.
A "file" lets us abstract the data from the filesystem. It is then trivial for that data to live on Ext2, Ext3, FAT16, NTFS, Juliet, in a zip, in a tar, as an email attachment, or in a pipe to an arbitrary process.
With a "common data storage", it sounds like what is really wanted is for each "object" to emit a standard, common interface. Once everything has that interface, we can wrap a database system around it to transform the data in lots of unique, interesting ways. Is there something implicit about this new abstraction that it has to live in the filesystem instead of on it (Is-A versus Has-A inheritence)? Does it require that we to throw out other, existing, useful abstractions ("files") to get it?
It sounds like an equivalent solution is to encapsulate each file in a platform independent, self-describing data structure. Then, impose the database query system on top of that. That both maintains the separation between file and filesystem provides all the features of the "common data storage".