The Linux Filesystem Challenge
Joe Barr writes "Mark Stone has thrown down the gauntlet for Linux filesystem developers in his thoughtful essay on Linux.com. The basic premise is that Linux must find a next-generation filesystem to keep pace with Microsoft and Apple, both of whom are promising new filesystems in a year or two. Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"
Instead, try to keep up with the demands and needs of users.
We live in a network-based universe. Local filesystems are already good - whether its just continued development in Reiser, or whatever else.
Nfs4, though - its like afs, only without the sucky stuff. AIX is now including nfs4 in its AIX5.3 release, even! With the Big Dog on board, we should realize there's wisdom in that direction ;)
... the BeOS file system? I heard that it was supposed to be the latest and greatest, the OS to end all OSes.
Nobody has come up with a compelling reason or feature to make me want to change filesystems.
Free Mac Mini Yeah, it's
Filesytems are tools that will suit different purposes. Some are good for databases, some for lots of small files, some for lots of reading, some for writing, some for networks, some for streaming.
So to develop a one handy "swiss army knife" of filesystems may not be the best route. For the most part one knows what a system will be doing and can build in the most appropriate filesystem for the job.
--
Agreed. Although Reiser4 looks very handsome, Linux needs to worry more about functionality and ease of use. Not filesystems. Joe Sixpack cares little for the filesystem even if he has a clue of what a filesystem actually is or does. He wants an easy to use, easy to get up and going machine that will play at least a couple of games, view a little porn, access his email, and perhaps get some work done on the side. If the filesystem is robust, then it's a bonus. Joe Sysadmin worries about the filesystem perks and quirks.
This has been a core feature of OpenVMS for a long time.
Hey, let's admit that Microsoft did a good thing with NTFS. Before I get roasted, let me say I've been working with FAT, NTFS, EXT2/3, Reiser, and others over the last 12 years, and I've had a chance to get a view of reliability, ease of recovery, etc., with several of these in production environments. I think the NTFS permissions model is one that the Linux world would welcome over the old and, I think, inadequate U-G-O scheme we continue to tolerate.
It's only funny until someone gets hurt. Then, it's hilarious.
Filesystems are so crucial to OS stability, that I'd say it's worth formally-verifying them to a certain extent (i.e. prove that the algorithms/code work, instead of just observing that they work in normal conditions).
P.S. The whole thing - filesystem as a DB - is complete crap. You can't do a bunch of fs operations in a single transaction and have ACID semantics on the transaction as a whole. Sure - searching is great. But database means much more than just a searching interface.
The Raven
Everyone seems to like this method. But do you also encrypt your swap partition? If not, then whenever the system swaps, unencrypted data gets stored somewhere on the swap partition.
Here's something that might terrify you: run grep on your swap partition and give it a few characters from your password. You needn't list the entire password. Scary, eh? (This won't work for everyone, but it might for you.)
Remember, if you're using loopback with crypto, you're just pissing in the wind unless you encrypt your swap, too.
Doesn't Palm OS have a database/filesystem hybrid too?
I've been using linux since the early 90's.
Always the stable version, didn't have problems until I tried reiserfs, switched back to ext2 ext3 actually, and I didn't have problems again.
The source of my problem appears to be resierfs directly or indirectly I don't know, like most users I don't really care either.
Doctor it hurts when I do this.
Then stop doing that.
Good enough for me.
This whole article is based on nonsense. Microsoft has a long way to go before it catches up with Linux in the filesystem area. There is no realistic prospect of Microsoft keeping pace with Linux filesystems in the foreseeable future.
(Before dismissing me a Linux fanboy, note that the above applies only to filesystems. When it comes to understanding of GUI issues, I'd make a similar statement but with Linux and Microsoft swapped. But that would be off-topic.)
Microsoft has a huge marketing department so they can please the people they are trying to sell to. Linux developers are their own market. By following Linux developers you are, more often then not, finding out what developers, like myself, want and need. By following Microsoft, you are finding what glitzy feature convinces someone that, maybe it's time to retire Office 97. The fact is, I have found few people, if any, who truly want a 3D desktop enviroment. I know others think diffrently, but i found XPs default taskbar seemed childish and condiscending.
frankly, i don't care if the casual consumer uses linux or not (though a larger market share would have some benefits). the people who develop for linux generally want and need the same things as my self and i'm happy.
That being said, faster file searching is definatly a useful tool. But if the registry in Windows is any indication of what the file system is going to look like in order for anything to get found, i don't want it.
On a random side note, App Rocket is a nifty program launcher for windows that finds files very fast.
The Neo-Bohemian Techno-Socialist
You're right, it wouldn't be traditional UID/GID anymore, it would be something better. We are all in this make something "better," aren't we? Or is this whole OSS thing just one big echo chamber?
Is the Linux/Unix community so "steeped in tradition" (also known as stubborness, obstinance, intolerance, and narrow-mindedness) that it willfully clings to an outdated, inferior way of doing things?
In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
Data that has not been backed up in a secure location should for all practical purposes be considered deleted.
Sorry, I'm not about to trust archived video to alpha code, or even beta code. If there's no release-worthy option on Linux, we have to stick to NTFS on Windows.
In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
Searching is all wonderful and that, but not the direction I believe would provide the most benefit.
Embed versioning into the filesystem. I believe Reiser has talked about this. Imagine being able to right-click on a file, folder or even partition and choose "roll back" or "restore" from the context menu. It then presents you with a list of snap-shot points you can restore to, starting with "last change".
Who backs up their hard drives any more? Have you thought of the problems and time involved in backing up 40, 80 or even 200 Gb of data? I'd MUCH MUCH rather have this feature than some enhanced search.
Learning HOW to think is more important than learning WHAT to think.
I'm an architect for a large corporation that is today trying to find a replacement for NFS. Our key goals are:
- Integration with a Kerberos SSO strategy
- Fast performance
- Cross-platform compatibility with Windows
- Robust Access Control mechanisms, RBAC would be nice but DACL is probably reality.
In my opinion, these are the primary goals that companies are looking for. Not a "journaling" file system, or built-in encryption. Sure those are nice, but let's get the basics first. Unfortunately, CIFS is still in quite a state of beta (even on the 2.6 Kernel) and there don't seem to be any real other alternatives.
Does anyone else find the fact that someone is comparing WinFS to the likes of reiserfs (despite my dislike for it), XFS, and hell, even ext3?
Granted, the proposed featureset of WinFS is vastly 'superior' to that of the 3 main linux contenders, but it could be argued that WinFS is neither a filesystem itself, nor is it on par with any of the linux filesystems in terms of performance or stability (if NTFS5 is to be of any forboding).
I seem to recall reading about several projects that impliment WinFS-like features. I don't recall what they were, and I don't think they were kernel-space projects, but I recall thinking, "this looks nice".
Besides, let's be honest here. What practical functionality does WinFS provide that is above and beyond the combination of 'locate', and 'file' used in conjunction? WinFS seems to me to be merely a crude hack so as to make up for the fundamental shortcomings with MS's OS design.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
Changing the filesystem to store real names is trivial actually. The hard part (and the point you seem to have missed) is that you would also have to alter almost every unix program ever written that ever made calls to setgid() or setuid() or getgid() or getuid(). That means that adapting to this new filesystem of which you speak means chucking most of the utility programs unix has right out the window and starting over again. At that point you might as well start over and make a new OS.
Don't label something "offtopic" unless you know the topic well enough to tell what's on topic.
There is a tendency to make the developers of Free Software responsible for hunting after every and any feature and idea MS and Apple care to implement to fish for users that is really annoying.
First, all Unix file systems since some decades have proved to just fit the bill quite fine. Searchable Metadata and other "features" is actually application-level stuff. 98% of the data on an average Desktop Unix system (and 99.5% of the data on a server) does not need that, because it rarely changes and is of no special interest for the user at all. And if it does, application-level data is better integrated, faster, and more flexible.
What is happening here (and in many other recent discussions) is dragging the Free Software community into an arms race it can not win. You can't make Linux/Gnome (or FreeBSD/KDE or whatever your favorite Open Source system may be) into a system that is just like Longhorn or Tiger but gratis and free. Never. What Free Software really needs, now more than ever, is to be picky about its users, its uses and its features. Better offer 10% of all users a system that is better than offer 90% of them a system that is a poor emulation of the OS they get for free with their PC anyway.
This is a point that the Free Software community has to (re)learn, better today than tomorrow.
IMHO the OSS community is underperforming in picking up new technology and good ideas.
Hardly. There are a lot of OSS projects that are leading the way with new technologies and in implementing good ideas.
But in quite a few areas it's not at all uncommon to see slow support for new tech. The community divides about how to implement the new ideas, which slows things down, but that division fosters competition and provides a base for testing out different ways of getting the new tech out the door.
Sometimes doing it well is better than doing it first.
I really don't see the point in a case-sensitive file system. Remembering case but ignoring it in comparisions makes life a lot easier. Can anyone point at some application or library or kernel part or anything that depends on the case of the filename alone to tell files apart? I can't think of any -- and besides, I'd consider it very bad behaviour.
-Lars
I, for one, would flame someone who suggested changing the semantics of pipes. Well-defined interfaces are the heart of reliable software. Changing an interface is a very big deal - especially one as entrenched as, for example, pipes.
Of course interfaces need to change sometimes. But first you need to ask how much you're going to break. If the kernel hackers break existing interfaces too much, they risk alienating the users/distros and forking the kernel.
Is this a check on innovation? Absolutely. But I'll point out that Microsoft has even larger checks on innovation - they promise far more backward compatibilty. And it has to be binary compatibility, which is harder than source compatability.
"The danger is not that a particular class is unfit to govern. Every class is unfit to govern." - Lord Acton
If the encryption is any good, it will be too random to achieve any compression.
I see a small problem with these "new" filesystems. Microsoft is creating their filesystem with their metadata format. Apple is adding metadata to their system in their own format. And now somebody wants to add metadata to Linux filesystems. But in what format?
Is it just me, or a lot of supposedly smart people missing the very obvious problem that you're not going to be able to exchange files between these different systems and keep the metadata? Before anybody ships a metadata-based system, I'd like to see some kind of standard defined for metadata interchange. Frankly, I think a standard is long overdue (please let filename extensions die!), and it's absolutely essential that a standard exist before different formats become too firmly entrenched.
You would think that people would have learned their lesson, but this is starting to look like the 80s all over again.
NTFS5 is miles ahead of most Linux filesystems (reparse points, encryption, compression,... all supported transparently), only ReiserFS 4 comes close, and it's not yet in a usable state.
I'm wondering if you even know what WinFS is, comparing it to file and locate it laughable at best.
Try finding all mp3 by Brian Adams or Withney Houston on a 200Gb disk filled with 250'000 files with file and locate, you'll get the answer 10 minutes later.
With WinFS, it will take you a whooping 2 seconds maximum.
That is without talking about the user-defined attributes, etc... that make WinFS more powerful than anything of that kind on Linux.
Maybe he isn't root and doesn't have permission to create groups.
automatically map bad blocks to good blocks, but you can monitor the current number of bad blocks through SMART. If your drive is causing errors you had better stop using it (production environment or no) as quickly as possible. Bad blocks are almost always a sign of impending failure.
Jebus, do you want to run a production system on a drive with known bad blocks? Whoever hired you must be a complete moron.
HAND.