Slashdot Mirror
Tom's Hardware Looks At WinFS
Alizarin Erythrosin writes "Tom's Hardware Guide has an article about the new WinFS file system. The article talks first about some of the problems and advantages with FAT[16|32] and NTFS, then talks briefly about WinFS. Here is the summary: 'Microsoft is breaking new ground with Longhorn, successor to XP. The upcoming WinFS file system will be the first to be context-dependent, and promises to make long search times and wasted memory a thing of the past. Today, THG compares it to FAT and NTFS.' Personally, I still have reservations about using a relational database to keep track of files. Unless they can keep the overhead to a minimum, I can't see it being as efficient as a file system should be."
yeah, but you still get a choice--i don't use mac os x's journaling because of the overhead--you don't hve to use winfs if the performance penalty is too high.
----
http://www.hellection.com
Now that every Windows user is going to be running a SQL variant on their system, imagine the bugs and holes that are going to be in this. Now THIS will be interesting to see.
Fundamentalism stops a thinking mind.
BeOS used indexing for certain attributes, and it is GREAT. Maybe someone is just sour that linux didn't do it first?
Wasn't this supposed to be an SCO story? We're falling behind our quota today.
In any event, Microsoft still has a few years to refine this "Future Storage" file system, so all judgements concerning it's effectiveness are a bit premature on some levels. Then again, it's always good to start planning as early as possible - especially when you consider that it may be introduced into Windows Server 2003 some time during the next 12-18 months. For now, all we can base judegement off of is Microsoft marketing hype and comparisons to existing file systems that operate in a similar way.
A relational database setup should do wonders for file search and access. Most filesystems today weren't designed with 200 Gb drives and millions of files in mind.
I keep thinking back to my Amiga when a 40 Mb hard drive was huge. Hell, I have a keyfob with more storage space now! Can you imagine AmigaDOS (not FFS, the old, slow one) on a 200 Gb drive?
Learning HOW to think is more important than learning WHAT to think.
This article is bullshit. There isn't a shred of new information in it. It's like watching CNN.
In Soviet America the banks rob you!
Haven't they basically been trying to implement this since the days of Cairo? Seems the 'revolutionary new file system' gets announced for every Windows release that is several years away, then vanishes by the time the release takes place.
.technomancer
The upcoming WinFS file system will be the first to be context-dependent, and promises to make long search times and wasted memory a thing of the past.
Well, yes; we must preserve those system resources for the most recent incarnation of explorer.exe.
Do you like German cars?
This will be better than FAT32 and NTFS, but it is hardly "breaking new ground". A number of operating systems have used more-or-less relational databases as their file systems; it's a special purpose technology and has no place in a general purpose OS. I think ReiserFS makes the right kind of compromise here: it uses a little bit of database technology, but it mostly remains a traditional file system.
As far as i was concerned, WinFS was not actually a real file system but something that just runs on type of an NTFS filesystem.
This was actually confirmed at WinHEC:
"Microsoft has scaled back its 'Big Bang', and its Future Storage initiative will build on, rather than supersede the NTFS file system, when the next version of Windows 'Longhorn' appears in 2005."
"WinFS is not a file system
NTFS will be the only supported file system in Longhorn, from a setup and deployment standpoint, though the OS will, of course, continue to support legacy file systems like FAT and FAT32 for dual-boot and upgrade purposes. The oft-misunderstood Windows Future Storage (WinFS), which will include technology from the "Yukon" release of SQL Server, is not a file system, Mark Myers told me. Instead, WinFS is a service that runs on top of--and requires--NTFS. "WinFS sits on top of NTFS," he said. "It sits on top of the file system. NTFS will be a requirement."
Interestingly, when WinFS is enabled, file letters are hidden from the end user, though they're still lurking there under the covers for compatibility with legacy applications. This reminds of when Microsoft added long file name (LFN) support in Windows 95, but kept using short (8.3) file names under the covers so 16-bit applications would still work. Expect this to be the first step toward the wholesale elimination of drive letters in a future Windows version."
ummm, FAT 32 has enormous cluster sizes for large drives. That does affect the normal user.
Fat 16 was limited to 2GB partitions, that affects the normal users.
Now, the fact that if the database file system works the way I imagine it would it will be a bad thing for the normal user's more tech savy friend.
I have spent years explaining to relatives that the same file name in 2 places is 2 different files.
Now I must spend time explaining that if you brows, documents, taxes, and edit file blah it will effect important stuff, file blah and what not.
People will be confused by this I believe. And I also think the the techies saying it is stupid would benifit from this greatly, I know I would love to organize things with tons of logical ways to browse there.
But I am not some overpaid market researcher so what do I know.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
It sounds interesting, to say the least. Maybe Microsoft has finally come up with something innovative. I'd be interested to try it out and see how it feels, and if it really can do everything they say it can. As usual, though, security could be an issue. A virus could wreak havoc if it found a way into the database.
Also, I'm wondering if they'll finally give up on that stupid drive lettering. I don't see any reason why that ever had to exist, and now that they're doing an overhaul of the whole filesystem, it seems like a good oportunity to get rid of it. You'd think, since they try to be user friendly, that they would want to give devices and partitions names instead of letters. I do still see it in that screenshot, but things could change by the time it's released.
This article is tripe.
The closest it gets to examining the (possible!) new Windows filesystem is calling it a relational database, and going on for a bit about how paths (ie: directory structures) will be irrelevant. Oh, and yeah, the closest thing he found to the implementation was called "winfs.exe" and did nothing but produce errors.
The bulk of the article is a (poor) attempt at explaining filesystems in general, and FAT and NTFS in particular. However, it gets a number of things wrong and - at best - garbles a lot of things. If you already know what he's trying to say, you *may* be able to pick out truths, but if you don't you'll walk away with misinformation.
I would suggest instead perusing arstechnica.com and aceshardware.com. I don't know if they've done any filesystem stuff, but if they have it'll be of reasonable quality.
Barclay family motto:
Aut agere aut mori.
(Either action or death.)
I read this article hoping for some real information on the WinFS file system, and instead I got an amature's review of Microsoft file systems I grew up with.
"There has been much speculation"
Uh huh.
"Win FS is modeled on the file system of the coming SQL server"
Uh huh.
"In its latest build (M4), Longhorn contains few hints of the technology's imminent implementation."
Uh huh. You're saying you don't know anything, yeah, I'm getting that part.
"One of those is more than 20 MB in size and bears the name winfs.exe."
Neat.
"In the end, Win FS will probably emerge as an optional file system beside FAT and NTFS. It's also possible that Win FS will supersede its predecessors, however."
So in the end, it'll be A... but it is also possible it'll be B. I see.
"That would most likely produce problems for multi-boot systems"
An astounding feat of logic Mr. Spock!
This is the most uninformative article I've ever had the displeasure of reading on Tom's Hardware. These people know exactly nothing more about WinFS than any of the rest of us have heard in rumors and vague press releases.
His post is darn insightful, wish I had some mod points :-/
100% Crunchier
How about Clippy? "I see you're looking for your work files. You're fscked."
One line blog. I hear that they're called Twitters now.
Looks like I was wrong - or, actually, right all along. Musta been a slow news day?
It's really irritating to have a RAID that gets crippled to 5-10MB/sec when on other OS and FS combos it can do 80-100 because MS has decided "Oh performance isn't important, reliability is, so we'll force cache off for all SCSI miniport devices even if it says it's on."
f =2&t=1758&hl=slow+scsi+performance&s=9f0e65a3ff482 2032e4a63091694cc3f it never got fixed. Non-RAID IDE is unaffected supposedly due to a "bug" in the system where IDE devices ignore the OS commands to switch to write through caching. It's really ridiculous when a 700MB file takes almost 2 minutes to copy under XP and yet under BSD on the same system dual booted on the same array, it takes 11.2 seconds.
See http://forums.storagereview.net/index.php?act=ST&
Introducing the new Occam Fusion! Now with sqrt(-1) fewer blades!
NTFS has tons more advantage than FATxx. The official list can be found here. Granted, this benefits the corporate user more than home user.
At the very least, NTFS offers a quicker way to hide porn than FAT32.
Not the only one!
With MS database record they have got to be kidding! I know that Windows CE uses a DB format for storage but I want to see it under max load with "n" task accessing it and the planet worth of data to pull from with a good percentage being changing. Then crash it and try to restore the mess. What would the resulting speed be. Recovery time.
I guess you will need a 4 to 6 ghz system with an insane speed hd array and memory up the wazu!
Instead of revamping the wrapper why not improve on surviablity of both data/os/programs! When will they get it in their head that the OS does and should not be a swiss army knife with cheep blades that are dull, usless, break and hard to open!
I cant remember but they was something based on somthing called "tumblers" that was a way to access data. Read somting in a ancient issue of Byte mag. Had to do with Objects and mondering content.
Hopefully this will encourage more competitors (including open source) to go for the RDBMS-based filesystem model.
I don't understand the concerns of the poster regarding performance (at least without evidence of truly dismal performance): no one is forcing anyone to use the FS if they are not satisfied with performance.
For most users, they main bottleneck in storage is their own organizational faculties. I used to be exasperated when users didn't know where they put their files, but once you get past the 100GB mark, it becomes very understandable.
Consider what most people use their massive storage for these days: videos, music, multimedia, games. Not only is this the kind of content that SHOULD be stored in a database, it's the kind of content that is ALREADY being handled through a database because the filesystem is not enough: people are using their media players, P2P programs and other software to handle their files, up to the point they rarely ever interact with the filesystem unless they lost a file.
For most users, the performance penalty is well worth the price.
For those for whom it is not, it doesn't take a genius to realize you can use more than a single filesystem, and perhaps rediscover the joy of proper partition organization: keep the OS and applications separate from your data, and you can use your highly efficient filesystem for the first and your metadata-loaded one for the second.
Freedom is the freedom to say 2+2=4, everything else follows...
1) NT4 (certainly from SP3) allows you to make partition alterations without a reboot. Even 2K requires a reboot for alterations to the boot partition, however.
2) 2K doesn't dispense with the drive letter concept, despite the implication in the article that this is the case. That you can mount partitions under folders doesn't change this.
3) You can specify the cluster size when formatting a drive under NT4.
Also, has anyone actually come across a data centre that is making use of multi-hundred-TB NTFS volumes?
And, will Longhorn finally do away with the whole drive letter concept?
"God, root, what is difference?" - Pitr, userfriendly
Relational databases are better than conventional file systems in both performance and transaction management/journalling.
However, the best solution is that used by EROS, which is for the kernel not to provide a file system at all, but instead provide Orthogonal Persistence.
This is a much simpler layer for applications, since it doesn't require them to explicitly access the memory and disk separately. It is also much simpler to recover from because the entire state of the whole disk is always known to be coherent with itself at all given points in time, without an expensive journal.
In terms of performance - it beats the hell out of explicit disk access systems (Both conventional and database systems) because it performs big continuous reads and writes (that don't move the head much) rather than small writes on metadata and file data that forcibly jump the disk head around.
In EROS then, on top of the Orthogonal Persistence, you can create any arbitrary Objects you want easily - because they're just normal processes with normal memory. Conventional File Systems become useless and objects implemented by processes become a much better and more powerful alternative to files.
A relational database of the user objects is then much more powerful than a string hierarchy, but this is all the user's choice - and not hardcoded into a kernel.
I'm looking forward to this! I personally am sick and tired of filesystems as we know them today. Today's filesystems are a strict hierarchy, the existence of which is only necessary in the systems of yester-year.
:+) Maybe the initial implementation wonâ(TM)t get all this right. But at least it stands a chance.
A filesystem based on a relational database will have some characteristics to which today's filesystems can only aspire:
1. ACID - In every way that the underlying database supports Atomicity, Consistency, Isolation, and Durability, so now will the filesystem. In so far as the database is robust, the filesystem will be robust. Please spare me the comments about the supposed unreliability of SQL Server. Itâ(TM)s certainly more reliable than NTFS; which is itself very good.
2. As an offshoot of the above - Imagine multiple file updates to a filesystem which is transactional! Imagine that transaction failing and being able to just rollback the changes without touching every file in your program! Imagine being able to make file changes programmatically without having to worry about locking because the engine will do it for you (just handle any exceptions)! Yeah, you could do all that today if you like. But it takes extensive to make it happen.
3. Operational characteristics - We can run queries against databases. We can index them. We can cluster them. We can replicate them. We can access them easily from any development platform you can imagine. Now your filesystem is a database. The possibilities make me shiver!
4. Another offshoot from #3 - Security. Databases are inherently better than filesystems (IMNSHO) at enforcing security and enabling administration of security.
I only have reservations about one issue with the database as filesystem area: recovery. Currently, all good and low-tech filesystem recovery tools really are based on the filesystem allocation table sort of scheme. Obviously, databases usurp this category of tried and true tools. However, good tools already do exist that allow recovery of relational databases. Itâ(TM)s just a matter of getting easily accessible tools of this sort into the hands of professionals that need them. It's more of a training issue I guess, but it will still need addressing.
I know many people will have a knee jerk reaction to this idea, and I understand why. But I would encourage people to keep an open mind to this. While there will probably be some issues with the idea, there's so much more that could easily be done with a filesystem on top of a database than could be done easily (or well) with a traditional filesystem.
And for you hard-core naysayers out there, you have to ask yourself this: If this is such a bad idea, then why did Oracle provide this as a feature too?
Please mod this post only if you think others should/n't read this. I have enough ego^H^H^Hkarma. Thanks!
Near-Term: this thing should be just as stable as every other MS product prior to version 3.0 of it. (In short, damned lousy). To make it worse, it probably also enables DRM at a file system level...
Mid-Term: FS finally works, and allows easier retrivial by relevance, author, source, etc. in ways that we can just dream of now. It's the kind of thing we didn't realize we needed until we had it...until it inevitably blows up as all MS products must do eventually. But when it works, we will be fairly happy to have it...especially end users, most of whom can't figure out a hierchical file system in the first place.
Far-Term: FS is finally able to use it's relational roots to distribute filesystems over multiple processors in an cluster or over a network. Such a system would support atomic, distributed file updates by threads of processes on differing processors (including HyperThreaded procs). Imagine a virtual filesystem that can span your whole-house network, with a single file system image...in WINDOWS.
So I guess my view is: painful in the near-term, but may be cool to have when they get it right.
Does it matter? HPFS was created at MS. I guess it explains why HPFS hasn't been improved on recent OS/2 beyond HPFS386, and JFS is now an optional FS on OS/2. JFS is probably a much more capable FS than HPFS anyhow.
http://www.cs.wisc.edu/~bolo/shipyard/hpfs.html
I recently installed a Win2K server that is blindingly fast at finding documents and such... but horridly slow at serving up portions of files, for things like legacy database programs. Three of the customer's applications started running at 1/4 speed.
It got so bad, even after all the "fix win2k speed" patches, that we re-introduced the 200MHz NT4 server to feed the database apps, and the dual-processor 2GHz system just serves up documents!
Unless they can keep the overhead to a minimum, I can't see it being as efficient as a file system should be.
They may have goals other than efficiency. Security, probably. But probably also security's perverted uncle, DRM. As DRM becomes more common, and we "pirates" look for more innovative ways to get around it, locking us out of our hard drives would seem to be a logical if not downright necessary step. It's pretty obvious that a lot of entities out there would benefit greatly from a model where we don't really OWN our computers, we just lease the right to use them. I mean, look, they're already floating the notion out there, at least for software and entertainment media, that we don't really OWN ANYTHING, and it's not that much of a stretch for that to become literal truth, aside from the hardware, which will be as impervious to meddling as they can possibly make it. (You decide who "THEY" might be, but I have a long list with a lot of familiar names on it.)
How technically difficult would it be for, say Microsoft, to "rent" out portions of your hard drive to various media and software providers, using a combination of hardware and software controls to assure those companies that you and I ABSOLUTELY CANNOT meddle with "their" product while we (temporarily) posess it? A database-driven file system provides exactly the access control and accountability that would be required to successfully implement something like that.
To ensure perfect aim, shoot first and call whatever you hit the target
yeah,
Because these same people will really quickly adapt to a gear or a foot instead of START.
And they really like to call me with questions when they buy a new printer and the install CD doesn't work, and then I find out that it is incompatable.
But their favorite thing is when they decide to make the family tree the family tree maker they buy doesn't run.
Links in Windows (9x anyway) cause all sorts of confuision too, like when copying the whole start menu to a floppy does not allow them to pirate all of their programs to someone else.
No, that just points to the program it is not the real program. Really the easiest thing to do is buy another copy and give it to them (we are talking playing card games, not $300 office packages).
I like and use Linux nearly exclusivly at home (I have SanctionII installed at home on Windows for work reasons), but I do not like to foist it upon my relatives. That is just rude, I like having a huge library of decent slightly less pretty slightly harder to use apps for free (in every sense). Most people like having easier to install (if you don't know anything), cheap and ubiquitose apps and hardware. And I can respect that.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
It just can't be good. Using MS SQL as a database is bad enough, I couldn't imagine depending on it as a file system.
I've dealt with a lot of bad products from MS, but SQL Server is not one of them. What exactly is wrong with it? It's been rock solid to me, with good features. I suppose the licensing might be brutal, but I don't care so much about that.
Geeks everywhere have been doing this already for a long time. It's nothing new or innovative. It's even been done in the filesystem code by several different groups (and that's still a bad idea). I've been doing this with Linux and MySQL for years. Once again Microsoft is trying to claim innovation on something everyone else was already doing.
On the other hand I wouldn't mind seeing a standard library for this purpose if one doesn't already exist. I know Gnome has a VFS library. I'm not sure how much of it is reused by other projects or who originated it.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
A directory tree is a very useful structure, at least to the software. Similar stuff is grouped together, and easily cached. It provides a very clean and simple way of putting data somewhere and getting it back later. This should not lightly cast aside.
So, you want to use a relational database to keep track of files? Go for it, but instead of keeping track of the files themselves, keep track of their paths. Let the filesystem do the efficient storage, and the database do the efficient lookups. The database can be made faster and smaller, the filesystems can remain as fast as they are, and the files are still there even if the database gets corrupted.
Put hooks wherever necessary to update the database when the filesystem changes. For example, put a database in the root of each filesystem. Use a stacked mount to mount that disk, so when interesting things happen, the kernel tells a userspace process that updates the database. Then, make some standard libraries that use the database. Make file browsers that can query it, but pass the path to programs. Make save dialogs that can also save metadata about the file, and open dialogs that can search for it. Use LUFS or FUSE to make directories that correspond to queries.
This is just as effective as what MS is doing, but it's more efficient, it's more compatible, and it doesn't reinvent the wheel.
Litigious bastards
A Mac desk accessory / extension combination, I believe, that came out in 1990 or so. It allowed you to instantly retrieve lists of file on your hard disk based on name and content (by "instant" I mean the list of matching files changed as you typed your query).
On my IIci it was perfectly fast. Faster than BeOS queries on a dual 603 box.
It took a little time to build your index, but keeping it up-to-date was pretty painless. Apple's developer CDs used to ship with On Location indices on them.
and may I ask, how many people have 1000 40kb files on a hard drive let alone one that is 500MB in size?
Three words...
Internet Temporary Files
A thousand small files is nothing with the default IE cache settings if you have a large drive.
Learning HOW to think is more important than learning WHAT to think.
Then at the end there's a few paragraphs with no real info about the FS at all. What meta info will be stored? How will the files be laid out on disk? Is there going to be journaling? How about file system integrity and recovery?
The only thing _really_ learned was that there exists a 20MB beta executable that doesn't do anything. What the frell? It's two years before Longhorn is to be released. (As if Microsoft is going to get it right the first time anyway).
Mod the whole article down. Way down.
New file systems always worry me esp with regard to data loss. With FAT and NTFS they are old but they are stable. I've never seen the OS lose data, though if there is a sudden crash then yes, but not during normal operation.
New FS = New corruption?
Rus
Cheap UK and US VPS
It's every slashdotters dream to get a "Score:5 Troll" post.
Nerd: Derogatory term typically directed at anybody with a lower Slashdot ID than you.
...MSWindows inches closer to where BeOS was 7 or 8 years ago.
I've been reading and contributing responses to Slashdot for years, but this is by far the worst article I've ever seen posted here. I can't believe whoever posted the article on the "front page" of Slashdot actually read the article -- they couldn't have even read the first page.
Right on the first page of the article, the "journalist" who wrote it describes disk storage as "memory". In the "Summary" of the article posted on every page, current file systems are described as wasting "memory". This reminds me of every-day users who confuse their computer running slowly (or literally telling them they don't have enough memory) with the need to delete files from the hard drive -- two completely separate things in most situations. This is all aside from the fact that the article doesn't actually tell you much that anybody who's used computers for more than six months doesn't already know. This guy sounds like some of the kids who come to me interviewing for I.T. positions thinking they've got a leg-up on everyone else because they've got some basic experience with PC's and Windows.
The bottom line is that the guy who wrote this article doesn't have any business writing tech articles without heavy supervision from someone who KNOWS tech, and I don't just mean someone who knows enough to rattle off performance numbers for CPU comparisons (read some other articles on the site). Lastly, Slashdot has no business posting amateurish and misinforming articles like this for the rest of us to waste our time on.
I have ~ 160,000 files taking up 55 gigs on my NTFS partitioned hard drives. It took over 5 minutes on my 1.6ghz machine to come up with that.
To search for a specific file often takes much longer.
Personally I look forward to a better, faster file system on Windows. Although I'd still hold off judgement of the new system until it becomes available.
-
Rod
The concept of naming files, and sorting them in directories isn't a very good concept, and the proof of it is looking at how everyone here uses playlists to handle media files.
Another poster mentioned phone numbers. An even better analogy is email addresses. The concept of email addresses and typing them into a TO field isn't a very good concept, and the proof of it is looking at how many people use address books to handle email addresses. Why type in an email address when you can just type in (or select) "tom"?
Thus I have the situation at work where I can't find anyone's email address. I want to send an email to "John Smith" down the hall. But the company exchange server is global, so I have to scroll down an Outlook list poring over 50,000 John Smith entries to make sure I don't accidentally send it to the John Smith in Kuala Lampur instead of the John Smith down the hall. I finally find the right John Smith and I expect to see the email address so I can write it down for later. But no! It's not available! The way my company has Exchange set up there is no actual email to be found. So I can only send email to John Smith from Outlook, because Evolution, KMail, Netscape, etc., keep complaining that "John Smith" isn't a valid address.
If that's the kind of situation you want for my file system, then may I suggest you take a long walk off a short pier. There are valid reasons for file names just as there are valid reasons for email addresses. Just because you use a playlist does not mean that the filenames for your MP3s are irrelevant.
This obviously doesn't require a database filesystem, but I think it's gotten to the point where we *need* some way to assign metadata to files and then deal with files *soley* by metadata.
We (my, myself and the mouse in my pocket) are already dealing with files solely by metadata, because the path of a file is metadata. Maybe it's not metadata that you want or care for, but there's a lot of people that do. I'm all for having a lot of metadata attached to my data. But you haven't explained why we need to eliminate the current metadata in order to get new ones.
p.s. Have you ever seen how the typical Windows user works? There are fifty icons on the desktop in no particular order and every document they create goes into a single "My Documents" directory. What makes people think they won't just shove everything into the same metadata category of "unfiled"? I know if I had to specify four or five different categories every time I had to save some work, I would probably shove a crowbar through my monitor within the first week.
Right now I have the choice of using a file system or a database. Why must this choice be eliminated? Does the concept of a file system offend you so much that you have to eliminate everyone else's use of one? Can't you just go use a database and let the rest of use work the way we want to work?
A Government Is a Body of People, Usually Notably Ungoverned
I think we'd be better off replacing the relational database with a file system.
Just a joke SQLiers, just a little joke. I know they are indispensible. Really. I believe you.
-pyrrho
I did use command line copy on both OS's. I've also tried tons of "accelerator" programs that claim to be faster at copying. I tried same partition, different partitions, changing cluster size to every possible setting it allows, etc. With Atto, the drive performs well once write size gets over 256K, but the OS's internal copy routines for both command line and drag/drop apparently want to write one 512 byte cluster at a time and confirm it went to disk before writing the next one, which is crippling the write performance because the writes won't cache or stripe.
Just "right click, turn on write cache etc" (from a previous post to this) DOESN'T WORK. If you'd care to READ what I originally posted I mention that it indicates write cache is enabled when IT ISN'T. It's pretty obvious whether it is or isn't based on the write performance. When read is 80-90MB/s and write is 1/10th of that, there's a problem. It's called the OS is forcing write-through, i.e. confirm all data to physical disk instead of just write back to the cache.
As for the controller, it's a 3Ware Escalade 7500-4, not one of those POS promise things. The drives are 4 Western Digital 1200JBs in RAID5. My previous Escalade 6400 and 4 75GXPs in RAID0 had the same problem. (this was asked in one of the previous posts)
Introducing the new Occam Fusion! Now with sqrt(-1) fewer blades!
I think the performance of WinFS will tell us how serious Microsoft is about really changing the way files are used. Performance is just a question of time and engineering resources - OS X's journaling is slow but HFS+ is an antique filesystem; in contrast BeOS had BFS, a journaled filesystem with all of the indexing buzzwords WinFS claims except free-text context searches and it was also extremely fast.
The difference isn't features - BeFS supported everything HFS+ does and arbitrary attributes, journaling, much larger file/filesystem support, and indexing and it was still faster. Be simply made performance a much higher priority than Apple has so far; fortunately they've hired the BeFS lead developer and perhaps 10.3 will have some surprises.
Another good example is ReiserFS - while some of their choices reflect overall design goals (e.g. targeting large numbers of small files instead of BFS's massive videos) they've largely passed the traditional filesystems in most areas despite having to do more work to keep all of the extra features going.
Microsoft has a number of engineers who do understand performance; the question is simply whether it'll be a significant priority for them to make WinFS fast enough that we'll realistically be able to use it.
Personally, I still have reservations about using a relational database to keep track of files. Unless they can keep the overhead to a minimum, I can't see it being as efficient as a file system should be.
Microsoft already has a file system that is integrated within a relational database. If you install Exchange 2000, you will see an M: volume on your local drive. You can flip thru this using Explorer as if it were an actual drive. In fact, it is a representation of a back-end relational database. The speed seems adequate to me so long as you have less than 8 GB in your mailboxes and public folders.
jwg
Usability engineers looked at what users were doing in Windows and they saw that tons of people weren't using the filesystem - at least not directly. They were just putting everything on the desktop. If it was on the desktop, they could find it. They kept folder structures to a minimum and organized things visually (or not at all).
This posed a significant problem, so indexing and searching and abstracting the filesystem was one of the solutions. Instead of having to navigate a filesystem (hard for many users), you just type in what you're looking for and *poof* it appears. Not sure what you're looking for? Start describing it... *poof* it appears.
I'm not saying this is the right solution, but technology is not always about cluster size and performance - especially if the system isn't usable. It will be interesting to see how user friendly this WinFS thing is...
3 years before release: Product will do everything for everyone.
2 years before release: Product will do everything for the majority of users.
1 year before release: Product will do many things for many users.
Release: Well it does something.
"To those who are overly cautious, everything is impossible. "
I use journaling and still get filesystem corruption that is not automatically fixed (like overlapped extent allocation) once in a while. On the other hand, HFS+ seems to already use a B-tree and file search is quite fast.
Microsoft loves to create incredibly complicated, undocumented and fragile data structures. Consider word: the internal structure of a word document as itself a filesystem, it has a root, a FAT and a bad block list. If even 1 block is damaged the file is useless. There are few repair tools because the internal layout is secret and undocumented. Access and SQL files are worse. In data recovery we frequently recover 90% of the files from volumes that have thousands of bad blocks or other damage. That wont work with WinFS. Are you going to store you precious digital photos on your $80 WesternDigital 80gb drive with its new 1 year warranty? Microsofts (and WD's) attitude will be that you should have backed it up. How do you easily back-up 80gb anyway?
NTFS is based on attributes. The filename is an attribute of the file. The ACLs are an attribute of the file. The file data is also an attribute. Some attributes are special and are used by the file system itself, but otherwise a file is just a collection of attributes. So NTFS has always been able to act as a database in the way you mention -- the attributes in Win2K are simply stored as NTFS attributes.
The only thing missing has been indexing. NTFS is not designed to index everything the way a database does. And it doesn't have a query engine. This is what WinFS will add.
Time flies like an arrow. Fruit flies like a banana.
Does this mean that finally the Windows file transfer counter will give sensible answers? Right now the only thing you can be sure of is that whatever it says, it's going to be something else.
Beep beep.
This is about denying Linux and BSD compatibility to their file system. NTFS is being reverse engineered as we speak, write access is being worked on, so obviously Microsoft needs something new to get rid of them pesky Linux people.
All the new and great features could be done with a (simple?) layer between NTFS and the application. There is no reason why Microsoft needs to invent a new file system here.
Am I the only one really annoyed at this guys use of "memory" for disk space? I _still_ find customers who can't tell the difference between RAM and HDD, and this guy goes and makes it worse for every twit who thinks they know everything. Yes, I know that one can make a theoretical argument for using the term, but really, in practice, memory is RAM. It's just annoying... I'm surprised to see it at Tom's Hardware...
jX [ Make everything as simple as possible, but no simpler. - Einstein ]
Postgres as an rdbms backend to a filesystem front end.
3
Designed mainly for version control. Could easily be modified for other purposes though.
http://www.linuxjournal.com/article.php?sid=138
Government of the people, by corporate executives, for corporate profits.
Take a look at www.namesys.com/v4/reiser4_the_atomic_fs.html
and at www.namesys.com/v4/v4.html.
We will be adding support for semi-structured data querying in the next major release, assuming that we find funding for it. The semantics for it are described at www.namesys.com/whitepaper.html, which also explains why I don't think the relational model is effective for semi-structured data stores such as a general purpose filesystem is normally used for.
Best,
Hans
Avoiding deadlock, dealing with transaction timeouts, controlling permissions on who can keep a transaction open for how long, these are all very serious issues that have us first making transactions available to plugins and trusted processes only in the first release. However, there is a LOT you can do with just plugins and trusted processes to make user data more secure.
I noticed right clicking in Win Explorer in Win2K allowed you to apply attributes to files. Not sure if they were limited to MS Office files.(Sorry, I can't check this anymore ;-)
.AppleDouble directories for an approach to foreign datastreams...they can get ugly pretty quick.
I believe that NTFS supports multiple streams of data per filename much like HFS. However, Windows doesn't make the pervasive use of the capability that Apple did in OS 9. I think one reason is that pretty much anything can go in these multiple streams meaning that particular files are tied even more closely to particular applications. Another reason may be that multiple filestreams complicate transfering files from machine to machine.
You're correct about RPM. It is using a more conventional approach by chucking it's data in a database file. Read about Netatalk's
I don't think it is possible to configure ReiserFS wrongly and to lose data as the above suggests. What is very possible is to use it with the wrong kernel and obsolete utilities. If you use ReiserFS with RedHat, get an OFFICIAL kernel from Marcelo, not a redhat kernel, and get the latest utilities off of our website.
I would take a guess that the users complaining about losing data are redhat kernel users and the ones that are happy are SuSE or official kernel users. When you compare stability of ReiserFS and ext3, try to compare them using the same version of the official kernel. We have generally been more stable than ext3 in the official kernels at any given moment in time, in large part because we were about 6 months ahead of them in the development cycle. Remember that RedHat does not tend to keep up with ReiserFS updates in its nonofficial kernels.
Recent RedHat kernels are probably much more stable than the old ones because they have bugfixes from more recent official kernels, but remember that these are the guys who in one of their releases compiled reiserfs with debugging turned on to make it slower than ext3, so I would go with an official kernel if it was me.
V3 of reiserfs is very stable currently. We go for months without bug reports and we have a lot of users in Europe. This is in large part because we put all our new features into V4. V4 will be unstable for quite some time.
Personally, I still have reservations about using a relational database to keep track of files.
A hugely conservative Slashdot reader? No way!
Not having read Tom's Hardware review of WinFS, but from the sound of things of the post, if WinFS is supposedly using a relational database to keep track of files, this sort of already sounds like what MS is implementing in some of their exisiting software such as Exchange and Sharepoint. Sharepoint utilizes Exchange-like file system where you can store files into the Sharepoint repository. In the first delivery of Sharepoint, there was an upper bound on the size of an individual file to be stored (I believe it was an 4GB limit) but in the current release I believe there is no upper limit. From what someone had told me if i recall right, Sharepoint utilizes the SQL engine to keep track of the files that are stored in Sharepoint. Maybe MS is just taking what they have learned from Sharepoint and making it more 'general purpose' for day to day use.
Let's get one thing straight ... many of these features are already available in one form or another and no one uses them.
... I'm not against having an FS that enables you to annotate files outside of the programs that create them. I remember working many years ago with FSs that would allow you to automatically keep the last n revisions, which was very helpful when coding.
How many people click on Properites in M$Word and put in the information?? How many people download files and leave them lying in whatever directory they just happen to fall in.
Don't take me wrong
It's just that I have a hard time getting excited over something that is going to simply bloat a system and the odds are no one will use.
My girlfriend is fairly smart, but she still downloads all her pictures into the default folder, and uses thumbnails to find the ones she wants. She has about 1000 of them, and it only takes here a few minutes to find the ones she wants. It would take here no extra work than what this new FS is suggesting to rename the file and/or store it in another folder
Useful feature, bloatware, Linux beater, or disaster waiting to happen. My guess is all of the above, at one time or another. Some people will use it and spend hours cross-linking files. I'm sure the initial releases will have security or data loss issues until the bugs get worked out. It will take 0.10 minutes for some Linux hacker to reverse engineer the ability to at least read it. And it will probably take up gobs more memory.
It's all a matter of perspective....
I rarely read replies, it's my opinion and if you thought about your opinion a little more, I'm OK with that.
Microsoft can require the bootstrap call routines from a separate DMCA/Patent protected, Microsoft branded, bootstrap ROM, and can control motherboard vendors through licensing for that chip.
Nasty Microsoft if they try that. Someone better get some prior-art out there and QUICK!
--- Nothing clever here: move along now...
This idea isn't new... Microsoft as long been trying to nip at the heels of IBM's midrange platforms. The system/38 shipped with a relational-only filesystem over 20 years ago. This filesystem is still alive in the AS/400 and iSeries systems. The advantages really are nice... merging the filesystem and the database, combined with a really smart optimizer, allow you to run a high performance database without having a DBA constantly monitor it. You end up with no tablespaces or any of the rest of the cruft that occurs when you try to build a database using files.
If it were just that, it might be laudable... but what MS is trying to do here is really more nefarious... MS is really trying to rid itself of all non-MS databases on Windows. This is the old IE vs Netscape trick. Why would you buy a real database when you're forced to buy SQL server anyway? Enough people might not know the difference
Finally, I can stop naming my folders like: ...
;)
cute brunette
cute brunette in dress
hot blonde with guy 1
hot blonde with guy 2
hot blonde with dildo 1
Now I can attach meta data like
hair color,
props (clothing, toys, surroundings),
# partners,
partner gender(s),
whatever else... you get the idea
no comment
For most file data, perhaps.
I will use this, and to good effect, as well.
The point to take into consideration is that the context will also change depending on the metadata available. Your view of the aggregate file objects changes, depending on the context. Not to mention that this same metadata will be available, in the same format, to all participating applications. Your apps can have all the same view, if you like.
What this means in concrete terms is that your carefully sorted directory of MP3's can look like a file library in iTunes. There are searchable, sortable columns for Title, Album, bitrate, Cover Art, year, label, and whatever (note I did not say "filename", which is just another attribute under a modern filesystem). This is possible with only the most basic gestures on the part of the user, and is remembered for the next time you visit this same view.
Similarly, a tree of photographs appear in any participating file browser with whatever columns you want (bit depth, format, date taken, date published, ICC info). It's important to consider that you can do this with any arbitrary collection of data, even one's you define yourself (to take the BeFS example, anyway).
So you can take your collection of widgets, define attributes about these widgets, and your file browser applet works the same for the same user in all applications. It should, anyway. This is why we have APIs.
To cite your example, why visual grep through a bunch of thumbnails looking for a particular photo when you can just indicate with a few gestures the "type" of photo you are looking for? I like the iPhoto interface when I'm browsing photographs, but if I want a particular photo of the GF from a rough date taken at night, I certainly don't want to browse through 1000's of images, especially when some of them can be hard to discern at thumbnail resolutions. I certainly don't want to do this repeatedly when I'm assembling a photo album on a specific subject.
Let the computer do the grunt work of selecting a result set that matches my criteria, and then I can use my human abilities to select the object I want, or refine the search.
Most of us already keep our aggregate file types in associated groups on the filesystem already. In most cases, the tree structure of most filesystems is sufficient. All this does is extended the functionality of the filesystem so that you can choose to abstract aggregate file objects and treat them in a a myriad of different ways. In the most basic sense, you tell the OS, "look, when I have the Explorer/Finder open on this directory of MP3's, make sure you change the column view so it shows this, this and that. In icon view, make sure that mouse-over pop-ups (if enabled) display this that and that. Default sort is alphabetically by Artist's Last Name. I don't want to see the filename, as that doesn't contain any useful information."
That is, you don't have do anything special to make use of the file attributes in this way. You just tell the ultimate app that all of us use the most (the operating system's file browser) to treat certain directories in a different manner.
-- clvrmnky
.... partly..... and barfed.
I know a thing or two about filesystems: I work in the data-recovery business.
This article was written by someone who has some basic details, and has fantasized a lot of incorrect info around it. Bunches of terms are used incorrectly etc. etc.
NTFS is a very well-thought-out filesystem. It can be made to perform well, and has almost no limits. (it does have limits: For example, files are limited to about 16 Billion gigabytes. Something like that....)
I sure hope they don't throw away the good things about NTFS....
Microsoft makes very little "good" software. But the NTFS filesystem generates the impression that it's different. They probably didn't design it themselves.