Slashdot Mirror
Tom's Hardware Looks At WinFS
Alizarin Erythrosin writes "Tom's Hardware Guide has an article about the new WinFS file system. The article talks first about some of the problems and advantages with FAT[16|32] and NTFS, then talks briefly about WinFS. Here is the summary: 'Microsoft is breaking new ground with Longhorn, successor to XP. The upcoming WinFS file system will be the first to be context-dependent, and promises to make long search times and wasted memory a thing of the past. Today, THG compares it to FAT and NTFS.' Personally, I still have reservations about using a relational database to keep track of files. Unless they can keep the overhead to a minimum, I can't see it being as efficient as a file system should be."
BeOS used indexing for certain attributes, and it is GREAT. Maybe someone is just sour that linux didn't do it first?
Wasn't this supposed to be an SCO story? We're falling behind our quota today.
This article is bullshit. There isn't a shred of new information in it. It's like watching CNN.
In Soviet America the banks rob you!
As far as i was concerned, WinFS was not actually a real file system but something that just runs on type of an NTFS filesystem.
This was actually confirmed at WinHEC:
"Microsoft has scaled back its 'Big Bang', and its Future Storage initiative will build on, rather than supersede the NTFS file system, when the next version of Windows 'Longhorn' appears in 2005."
"WinFS is not a file system
NTFS will be the only supported file system in Longhorn, from a setup and deployment standpoint, though the OS will, of course, continue to support legacy file systems like FAT and FAT32 for dual-boot and upgrade purposes. The oft-misunderstood Windows Future Storage (WinFS), which will include technology from the "Yukon" release of SQL Server, is not a file system, Mark Myers told me. Instead, WinFS is a service that runs on top of--and requires--NTFS. "WinFS sits on top of NTFS," he said. "It sits on top of the file system. NTFS will be a requirement."
Interestingly, when WinFS is enabled, file letters are hidden from the end user, though they're still lurking there under the covers for compatibility with legacy applications. This reminds of when Microsoft added long file name (LFN) support in Windows 95, but kept using short (8.3) file names under the covers so 16-bit applications would still work. Expect this to be the first step toward the wholesale elimination of drive letters in a future Windows version."
This article is tripe.
The closest it gets to examining the (possible!) new Windows filesystem is calling it a relational database, and going on for a bit about how paths (ie: directory structures) will be irrelevant. Oh, and yeah, the closest thing he found to the implementation was called "winfs.exe" and did nothing but produce errors.
The bulk of the article is a (poor) attempt at explaining filesystems in general, and FAT and NTFS in particular. However, it gets a number of things wrong and - at best - garbles a lot of things. If you already know what he's trying to say, you *may* be able to pick out truths, but if you don't you'll walk away with misinformation.
I would suggest instead perusing arstechnica.com and aceshardware.com. I don't know if they've done any filesystem stuff, but if they have it'll be of reasonable quality.
Barclay family motto:
Aut agere aut mori.
(Either action or death.)
I read this article hoping for some real information on the WinFS file system, and instead I got an amature's review of Microsoft file systems I grew up with.
"There has been much speculation"
Uh huh.
"Win FS is modeled on the file system of the coming SQL server"
Uh huh.
"In its latest build (M4), Longhorn contains few hints of the technology's imminent implementation."
Uh huh. You're saying you don't know anything, yeah, I'm getting that part.
"One of those is more than 20 MB in size and bears the name winfs.exe."
Neat.
"In the end, Win FS will probably emerge as an optional file system beside FAT and NTFS. It's also possible that Win FS will supersede its predecessors, however."
So in the end, it'll be A... but it is also possible it'll be B. I see.
"That would most likely produce problems for multi-boot systems"
An astounding feat of logic Mr. Spock!
This is the most uninformative article I've ever had the displeasure of reading on Tom's Hardware. These people know exactly nothing more about WinFS than any of the rest of us have heard in rumors and vague press releases.
How about Clippy? "I see you're looking for your work files. You're fscked."
One line blog. I hear that they're called Twitters now.
Looks like I was wrong - or, actually, right all along. Musta been a slow news day?
1) NT4 (certainly from SP3) allows you to make partition alterations without a reboot. Even 2K requires a reboot for alterations to the boot partition, however.
2) 2K doesn't dispense with the drive letter concept, despite the implication in the article that this is the case. That you can mount partitions under folders doesn't change this.
3) You can specify the cluster size when formatting a drive under NT4.
Also, has anyone actually come across a data centre that is making use of multi-hundred-TB NTFS volumes?
And, will Longhorn finally do away with the whole drive letter concept?
"God, root, what is difference?" - Pitr, userfriendly
Linux can not safely write to NTFS, not even NTFSv1.2 even though it has been around since 1994. The lack of documentation makes it so that some aspects of updating the metadata and indexes cannot be done safely in all situations.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Yeah, but I bet you it'll be like Windows XP's theoretical ability to use FAT 32 - it can read it, but it won't actually let you format in FAT32 when doing an installation
So when is the last time you actually used WindowsXP? What you said is pretty silly.
You can install WindowsXP on FAT32, even create FAT32 partitions during setup etc, all the same options you get with NTFS during the install.
Are you trying to make up stuff or just never used it?
Geeshâ¦
A directory tree is a very useful structure, at least to the software. Similar stuff is grouped together, and easily cached. It provides a very clean and simple way of putting data somewhere and getting it back later. This should not lightly cast aside.
So, you want to use a relational database to keep track of files? Go for it, but instead of keeping track of the files themselves, keep track of their paths. Let the filesystem do the efficient storage, and the database do the efficient lookups. The database can be made faster and smaller, the filesystems can remain as fast as they are, and the files are still there even if the database gets corrupted.
Put hooks wherever necessary to update the database when the filesystem changes. For example, put a database in the root of each filesystem. Use a stacked mount to mount that disk, so when interesting things happen, the kernel tells a userspace process that updates the database. Then, make some standard libraries that use the database. Make file browsers that can query it, but pass the path to programs. Make save dialogs that can also save metadata about the file, and open dialogs that can search for it. Use LUFS or FUSE to make directories that correspond to queries.
This is just as effective as what MS is doing, but it's more efficient, it's more compatible, and it doesn't reinvent the wheel.
Litigious bastards
From what I've read and experienced, the performance hit only involves writes. Reads are unaffected.
Writes are about 10% less efficient, which a pretty good tradeoff for that peace of mind.
--
the strongest word is still the word "free"
Right, but in general, journaling is a hack. The BSD softupdates concept is both cleaner and faster, and accomplishes nearly the same thing. To be fair, all of NTFS, ext3, reiserfs, BeOS's FS, and the new OSX FS have some advanced journaling features, but the concept itself should be more cleanly handled. Much like databases should be ACID compliant, filesystem writes can and should occur in such a manner that no external journal is required.
--
Use Vobbo for Video Blogs
This is completely unfair.
How many gaping security holes have there been in NTFS?
By most accounts, NTFS is one of the better filesystems ever written. Journaling, ACLs, decent performance.
There is talent in Redmond. Ignoring that is as flawed as assuming the entire Linux community is represented by Sendmail and SCO (security holes and bad publicity). Pretty bad, right?
--
Use Vobbo for Video Blogs
I've been reading and contributing responses to Slashdot for years, but this is by far the worst article I've ever seen posted here. I can't believe whoever posted the article on the "front page" of Slashdot actually read the article -- they couldn't have even read the first page.
Right on the first page of the article, the "journalist" who wrote it describes disk storage as "memory". In the "Summary" of the article posted on every page, current file systems are described as wasting "memory". This reminds me of every-day users who confuse their computer running slowly (or literally telling them they don't have enough memory) with the need to delete files from the hard drive -- two completely separate things in most situations. This is all aside from the fact that the article doesn't actually tell you much that anybody who's used computers for more than six months doesn't already know. This guy sounds like some of the kids who come to me interviewing for I.T. positions thinking they've got a leg-up on everyone else because they've got some basic experience with PC's and Windows.
The bottom line is that the guy who wrote this article doesn't have any business writing tech articles without heavy supervision from someone who KNOWS tech, and I don't just mean someone who knows enough to rattle off performance numbers for CPU comparisons (read some other articles on the site). Lastly, Slashdot has no business posting amateurish and misinforming articles like this for the rest of us to waste our time on.
The limit is 32GBs.
FAT32 Formatting in Setup is an option, but is known to frequently fail or have significant errors
No it doesn't.
FAT 32 Formatting once you are in the OS requires 3rd Party wares.
Again, only if creating partitions bigger than 32GBs.
The sysinternals driver is NOT a reverse engineering, it is basically a DOS mode interface to the windows system dll's. When you use the NTFSDOS disk you have to run it under the version of windows for which you will be accessing the NTFS partition, it then snarfs up the system dll's and puts them on the disk. Then when you load up the disk you run ntfsdos.exe which is basically a wrapper around that code. This could probably be done for linux but it would probably not be redistributable for fear of legal reprecussions from MS (how sysinternals gets away with it I do not know, maybe they have an agreement with MS, or maybe they are just too small of fish and too usefull for MS to swat them)
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
I think the performance of WinFS will tell us how serious Microsoft is about really changing the way files are used. Performance is just a question of time and engineering resources - OS X's journaling is slow but HFS+ is an antique filesystem; in contrast BeOS had BFS, a journaled filesystem with all of the indexing buzzwords WinFS claims except free-text context searches and it was also extremely fast.
The difference isn't features - BeFS supported everything HFS+ does and arbitrary attributes, journaling, much larger file/filesystem support, and indexing and it was still faster. Be simply made performance a much higher priority than Apple has so far; fortunately they've hired the BeFS lead developer and perhaps 10.3 will have some surprises.
Another good example is ReiserFS - while some of their choices reflect overall design goals (e.g. targeting large numbers of small files instead of BFS's massive videos) they've largely passed the traditional filesystems in most areas despite having to do more work to keep all of the extra features going.
Microsoft has a number of engineers who do understand performance; the question is simply whether it'll be a significant priority for them to make WinFS fast enough that we'll realistically be able to use it.
Plenty of filesystems are out there that do ACLs. FreeBSD's UFS2 does, I'm pretty sure SGI's XFS and IBM's JFS do also.
I think you miss the point of WinFS. It is actually a layer abover NTFS, so the actual files will still be stored the same way they are presently (and NTFS is pretty good about reliability). NTFS is already a lot like ReiserFS even without WinFS.
What WinFS adds is more powerful indexing, not a new storage system. Whenever you add or modify a file, WinFS adds the file's attributes to its indexes. The attributes stored are customizable, and vary depending on file type (MP3s have their ID3 info indexed, etc.). For example: somerwhere on my 120 GB disk I have a file named "code.txt" but it will take 10 minutes to find it by scanning the directory structure. Instead, I do a "SELECT Path FROM Files WHERE Filename='code.txt'" and WinFS comes up with the answer right away. If I have full text indexing, I could search for a specific phrase. Even more useful, you don't have to make playlists anymore. Just put all of your MP3s in the same directory, and when you want to hear all of your Bon Jovi, just perform the appropriate query. (Obviously you don't have to know SQL to make this useful.)
Some of this is already present in a more limited form in Unix. For example, I still can't figure out why Windows doesn't have something like the LOCATE database that is set up by default on my FreeBSD box. But WinFS will blow LOCATE away (update is real time, not daily, and it has much more than just the file name).
However, from what I've heard, so far it is a bit of a dog performance-wise. I hope it gets up to speed by ship time... At least it can be turned off if you can't take the perf hit.
Time flies like an arrow. Fruit flies like a banana.
Usability engineers looked at what users were doing in Windows and they saw that tons of people weren't using the filesystem - at least not directly. They were just putting everything on the desktop. If it was on the desktop, they could find it. They kept folder structures to a minimum and organized things visually (or not at all).
This posed a significant problem, so indexing and searching and abstracting the filesystem was one of the solutions. Instead of having to navigate a filesystem (hard for many users), you just type in what you're looking for and *poof* it appears. Not sure what you're looking for? Start describing it... *poof* it appears.
I'm not saying this is the right solution, but technology is not always about cluster size and performance - especially if the system isn't usable. It will be interesting to see how user friendly this WinFS thing is...
It should be noted that the regular fsck should still be performed, even on a journaling filesystem. The fsck after an unclean shutdown is obsoleted by the journal, but the regular fsck is supposed to catch subtle errors in the filesystem code and hardware problems which can cause the filesystem structure to rot slowly - problems which a journal can't and won't fix.
Microsoft loves to create incredibly complicated, undocumented and fragile data structures. Consider word: the internal structure of a word document as itself a filesystem, it has a root, a FAT and a bad block list. If even 1 block is damaged the file is useless. There are few repair tools because the internal layout is secret and undocumented. Access and SQL files are worse. In data recovery we frequently recover 90% of the files from volumes that have thousands of bad blocks or other damage. That wont work with WinFS. Are you going to store you precious digital photos on your $80 WesternDigital 80gb drive with its new 1 year warranty? Microsofts (and WD's) attitude will be that you should have backed it up. How do you easily back-up 80gb anyway?
I'm posting this from a WinXP machine. Let me assure you that while it can read FAT32, the OS as default will not FAT32 format any drive over (I belive) 2Gb.
w to /gettingstarted/guide/installnew.asp
/?
/N:sectors] /FS:filesystem Specifies the type of the file system (FAT, FAT32, or NTFS).
FAT32 Formatting in Setup is an option, but is known to frequently fail or have significant errors.
FAT 32 Formatting once you are in the OS requires 3rd Party wares.
SCO has performed an illegal operation and will be shut down.
I hate to rain on your parade, but the information you are providing you are either mixing with NT4 information or just making it up as you go.
Nothing I can say is convincing you, so let me reference you to the information you need.
http://www.microsoft.com/windowsxp/pro/using/ho
It clearly provides information on the only limitations of doing a FAT32 installation of WindowsXP, and that is the inherent limitation of FAT32 only being able to recognize a 32GB partition size.
As for '(I belive)2Gb', you are referring to the FAT16 installation of NT4. It doesn't apply to WindowsXP.
FAT 32 Formatting once you are in the OS requires 3rd Party wares.
Here let me quote from a command prompt for your edificationâ¦
Format
Formats a disk for use with Windows XP.
FORMAT volume [/FS:file-system] [/V:label] [/Q] [/A:size] [/C] [/X]
FORMAT volume [/V:label] [/Q] [/F:size]
FORMAT volume [/V:label] [/Q] [/T:tracks
FORMAT volume [/V:label] [/Q]
FORMAT volume [/Q]
volume Specifies the drive letter (followed by a colon), mount point, or volume name.
Notice the last line?
So where do you get your information, tooth-fairy, or just make it up as you go? Third party utilities are NOT needed to format FAT32 in WindowsXP, as you said âonce you are in the OSâ(TM) â" BTW, did you know that even during Setup â" YOU ARE âIN THE OSâ(TM)? It is running a stub version of NT already at that point; you are already in the NT OS.
FAT32 Formatting in Setup is an option, but is known to frequently fail or have significant errors.
Since when? Where do you document this at? Funny in all the time our tech team has put into working with WindowsNT, they can find no reference to this, either online or in their own experience. Tooth-fairy again or just making it up as you go?
There are NO known issues or documented issues of WindowsNT EVER having a problem running on a FAT partition, either FAT16 from NT4 days or FAT32 of current days. The NT core has an installable File System, like most modern OSes, and its interaction with the file system underneath is irrelevant to how it operates in regard to failure or problems.
Besides, this is really a moot point. If you are running WindowsXP and it is your only OS, then there is NO reason not to use NTFS. It is faster with larger files, faster with large volumes, supports compression, supports encryption and can have partitions up to 16 exabytes in size.
If you are running WindowsXP on FAT32, you are missing half the benefits of WindowsXP and the security of NTFS as well as its reliability of sitting on a journalled file system, etc, etc.
Geesh.
I think you are confusing the terms. Or you just got the threads mixed up.
Journalling makes sure that the state of data, or meta-data is always defined. So even if you turn the computer off while in the middle of a write command the file-system will be in a correct state afterwards. (The write operation will be lost however.) It is basically a way of making HDD operations atomic. Either they succeed or they fail, the in between state is what puts your drive in a corrupted state.
When talking about 10% it is refering to space on the disk for the log file to make this possible.
Now storing the meta data in a database, which is essentially what WinFS and such are doing, is not as clear a benfit. Personally I can imagine that it would be a very practiacal FS for keeping movies and MP3's on. I don't really see the benefits of running the OS files on that FS though. A lot of unneccesary overhead. (I don't search for files in my OS partition very often.)
Much like databases should be ACID compliant, filesystem writes can and should occur in such a manner that no external journal is required.
And how would you propose to achieve that without performance going down the drain?
The guys writing these filesystems are not dumb, and the reasons why journals are used are well considered. Another thing is that ACID compliant databases also use something like a journal to achive the atomicity.
Oh, and forget softupdates, they are _not_ comparable to journaling filesystems, for instance you still need to fsck, it's just faster.
Compare that with one of the funnies manpages I know.
Am I the only one really annoyed at this guys use of "memory" for disk space? I _still_ find customers who can't tell the difference between RAM and HDD, and this guy goes and makes it worse for every twit who thinks they know everything. Yes, I know that one can make a theoretical argument for using the term, but really, in practice, memory is RAM. It's just annoying... I'm surprised to see it at Tom's Hardware...
jX [ Make everything as simple as possible, but no simpler. - Einstein ]
Take a look at www.namesys.com/v4/reiser4_the_atomic_fs.html
and at www.namesys.com/v4/v4.html.
We will be adding support for semi-structured data querying in the next major release, assuming that we find funding for it. The semantics for it are described at www.namesys.com/whitepaper.html, which also explains why I don't think the relational model is effective for semi-structured data stores such as a general purpose filesystem is normally used for.
Best,
Hans
NTFS is actually not bad with fragmentation. I'm running a Win2k Pro box and a WinXP Pro box; neither's partition has ever reached more than 12% fragmentation, and even that was after having the partitions 98% full. Most people won't notice NTFS file fragmentation as a problem until it reaches 50% to 60% anyway. FAT32, however, is quite a different story. I can start to notice performance hits around 9% fragmentation or so. Also, according to my MCSE training kit, the main cause of filesystem fragmentation on windows machines is using a page file that does not have a static size. Using a statically-sized page file can decrease defragmentation dramatically. (If you don't believe me, set your paging file to a static size between 1.5 and 3 times the amount of physical RAM you have, reboot, defragment, and test with every FS torture you can think of.)
Now storing the meta data in a database, which is essentially what WinFS and such are doing, is not as clear a benfit. Personally I can imagine that it would be a very practiacal FS for keeping movies and MP3's on. I don't really see the benefits of running the OS files on that FS though. A lot of unneccesary overhead. (I don't search for files in my OS partition very often.)
Indeed. It seems like what they are claiming as an improvement, (i.e., faster searching for files), does not appear to help what people actually do most of the time. It is similar to claims of "boots much faster!" that you used to hear about new versions of windows. I would think the thing that would be important to people is data integrity and access efficiency. I know my primary concern is "how safe is my data".
I also question the need to include the overhead of a database frontend to the filesystem. Seems like a catastrophe just waiting to happen.
Also, since the DB is always active, what issues do you have with backups? I'd be concerned about backup and restoral issues with this type of filesystem. I haven't seen that addressed at all.
This is an ex-parrot!