Slashdot Mirror
The Linux Filesystem Challenge
Joe Barr writes "Mark Stone has thrown down the gauntlet for Linux filesystem developers in his thoughtful essay on Linux.com. The basic premise is that Linux must find a next-generation filesystem to keep pace with Microsoft and Apple, both of whom are promising new filesystems in a year or two. Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"
Linux must find a next-generation filesystem to keep pace
What are the winds of change saying? R..E..I..S..E..R...4...
Hans Reiser has written a white paper containing his thoughts on the design of the next major version of ReiserFS.
This is supposedly the indestructible filesystem. Compared to database driven filesystems which are still based on NTFS probably I'd preferr a slower filesystem that was just plain reliable.
Instead, try to keep up with the demands and needs of users.
...wrote "Open Sources", which you can read/buy here. He's a fairly savvy fellow...
The Army reading list
I think promising new filesystems in a year or two is a big word.
Wasn't the immediate urgency a comparable GUI to Windows or Apple?
Uselessful technology (Air-Charged
We live in a network-based universe. Local filesystems are already good - whether its just continued development in Reiser, or whatever else.
Nfs4, though - its like afs, only without the sucky stuff. AIX is now including nfs4 in its AIX5.3 release, even! With the Big Dog on board, we should realize there's wisdom in that direction ;)
I haven't RTFA, but... Apple is just adding more meta-data to that already used by HFS+. They will still be using a journaled HFS+ and the forthcoming Spotlight will just make use of the meta-data they are adding to their FS. Many decry HFS+ as outmoded and inefficient, but with the changes Apple is making, it is looking more promising. I hope M$ puts something new out because NTFS is getting a bit long in the tooth. Just my $0.02.
Help a college student
Hans Reiser has some interesting ideas about the role of a modern file system. Here's a recent USENET post describing some of the immediately visible features of reiserfs v3. Some people have said that there was corruption in the past, but I think there are no longer any problems in recent 2.4 kernels. Namesys is now developing Reiser4, which appears to be more flexible (still needs time to stabilize though). If I had to put down my money on a future filesystem though, it would be ReiserFS.
... the BeOS file system? I heard that it was supposed to be the latest and greatest, the OS to end all OSes.
I want a disk equivalent of top - something that'll tell me what processes are kicking the shit out of the disks, and by how much.
If Linux could do that - it's more a VM thing than a filesystem - I'd stick with ext3 for years to come.
Who needs a filesystem in a database when you have a database that lives on your filesystem (updatedb). Get that updating in realtime, with more things (like permissions, access times etc.) and a lot of the work is done.
john
Nobody has come up with a compelling reason or feature to make me want to change filesystems.
Free Mac Mini Yeah, it's
Are there any encrypted filesystems available for Linux? Last time I checked (quite a while ago), there wasn't.
#!/
Filesytems are tools that will suit different purposes. Some are good for databases, some for lots of small files, some for lots of reading, some for writing, some for networks, some for streaming.
So to develop a one handy "swiss army knife" of filesystems may not be the best route. For the most part one knows what a system will be doing and can build in the most appropriate filesystem for the job.
--
Gnome Storage should be a step in the right direction, and it gets it right by not reinventing the wheel, just using PostgreSQL as its database engine.
This way we can test the waters without messing with the kernel. When the concept is tried, we can decide if we make PostgreSQL a required part of a GNU/Linux system, or a Hurd translator, or whatever.
Leandro Guimarães Faria Corcete DUTRA
DA, DBA, SysAdmin, Data Modeller
GNU Project, Debian GNU/Lin
I'll use flat files and grep like god intended.
I want a new world. I think this one is broken.
Is there anything that a true database filesystem offers that something like a realtime updatedb index and maybe a background updated glimpse index of /home cant offer?
I have about 18GB of files in my main home dir, and I can search it in seconds with slocate and if I need a content search, with glimpse.
I know that this kind of database FS provides a lot of cool opportunities in terms of meta-data, but how useful is it for non-techies, who usually dont name their files coherently, let alone correct ID3 tags or other other meta-data.
Agreed. Although Reiser4 looks very handsome, Linux needs to worry more about functionality and ease of use. Not filesystems. Joe Sixpack cares little for the filesystem even if he has a clue of what a filesystem actually is or does. He wants an easy to use, easy to get up and going machine that will play at least a couple of games, view a little porn, access his email, and perhaps get some work done on the side. If the filesystem is robust, then it's a bonus. Joe Sysadmin worries about the filesystem perks and quirks.
..anyone got a good RTFA or source that explains how journalled filesystems work without.. um... doubling the data and filesystem?
thanks.
do() || do_not();
What do you want these next generation features for? Mainly features like access control, security, robustness, and above all organizing and sharing data.
Why not go higher level, use a reliable and simple underpinning such as ext3fs, with something like WebDAV (Distributed Authoring and Versioning) on top of it? Like SubVersion, it is based on HTTP, with specification for versioning and rich access controls.
Or maybe even go to the level of a Java JSR, so you could have a cross platform API for accessing files so it really doesn't matter what the back end is, KaZAA, Google or a DataSette, as long as your programs have a high level view of the information.
You might even end up with something liek the original TB-L Web, with everyone running their own Web server.
Of course, excluded from the above is performance, which would be ok for office type apps but not something that requires direct disk access, but perhaps the simpler file system would be most suitable for that.
Of course, I'm just rambling here, so would be happy to hear more developed responses to this suggestion...
Just make sure it is incompatible with all the current applications so we can rewrite everything. Add a cool feature or something too.
Business isn't willing to pay for products, innovation and careers, so we get brands, mortgage commercials and layoffs.
This explains everyting you need to know. But basically FFS is a compatibility thing. Apple still recommends its HFS+.
So Joe Barr who writes for Linux.com, submits a story to Slashdot (who owns Linux.com) written by another OSDN guy. Good job for editorial independance.
The more you know, the less you understand.
Introduction to the Mac OS X File System.
I watched C-beams glitter in the dark near the Tannhauser gate.
If,
MS can do a filesystem built on top of SQL Server then Linux should try to build one on top of MySQL.
It probably would be popular...
BTW, before somebody gets all bent outta shape the above is a joke.
Caution: Contents under pressure
Care to point out the bugs in the current NTFS that make you sure the next one will also be?
Doesn't it make you feel good to know that our freedoms are protected by politicans, lawyers and journalists.
I am using JFS on my Gentoo server, and it's just fine - fast and stable. Not that I know much about filesystems, but the fact that IBM is behind it gives it some credibility, it would seem.
12:50 - press return.
OS X has had a journaled file system since 10.2 and it is "on" by default in 10.3
As below, so above and beyond, I imagine drawn beyond the lines of reason. Push the envelope. Watch it bend.
I know that some don't like it, but we need the option of file system versioning, so that if/when you delete half the lines in your letter/program/... you can get them back from the previous copy on disk.
There is an expectation that the application should do it, that means extra code in each application and they all do it slightly differently.
OK: need an O_NOVERSION on open(2) if the app *really* doesn't want this - eg a database.
Make the core filesystem small, robust and fast. Journalling, realtime and not much else. Make add-on modules for fancy things like ACL's, quota, compression, encryption, compatability, extended attributes, etc... Put in shims for calling attributes from a database (db or SQL or whatever)
XFS comes close, ReiserFS 4 is nice, too. The most important thing is keeping the base filesystem simple and FAST. You think NTFS is fast? Try deleting a complete Cygwin install (>30K files) It takes AGES, even from the command prompt. I've deleted 15K files (That's 15 THOUSAND files) on Reiser 3 on the same machine, it took a few seconds.
DO NOT make a database driven filesystem. Some day we will have a true, document based desktop paradigm (OpenDoc anyone?) but probably not for several years, until then we need SPEED.
My Other Computer Is A Data General Nova III.
OK we have all these DB things that seem more for meta data and seach realy thats a bit secondary to a filesystem. Most filesystems are access by applications for surprise surprise files with very little user files and lots of application files. While it might make snece to mount /home as some DB is a filesystem with piles of indeed and seachable data so the users can be even more clueless to where anything is. The rest of the system needs faster all around and cluster aware from my point of view. Versioning in the FS ala VMS would be a nice thing as well. Disks are the slowest thing on your average system with Gigabit ethernet moving more data than the highest performing single disk in the real world.
No sir I dont like it.
I thought the real trick to running a search engine was the index. As a matter of fact, I thought I heard a story on NPR about Google's intent to index a whole mess of new stuff for their engine just to deep-six the competition.
"Rocky Rococo, at your cervix!"
Lets get the "this generation" filesystems working correctly, shall we?
Solid, universal support for ACLs, and while we're at it, let's fix the whole user/group namespace mess Unix has with it. Let's use an SID-style id like Windows does.
For example: my small network at home, centrally authenticated through ldap.
Now, windows knows the difference between the user "jim" on local machine A, "jim" on machine B, and "jim" the domain user. They'd be shown as MACHINEA/jim, DOMAIN/jim, etc.. The various SIDs take the domain (or workstation) SID and append the UID. So if his number is 100, his sid is "long-domain-sid" + uid. So when you pass around sid tokens, you know exactly which jim you're talking about.
Now in linux, we just have numbers for users and groups. If user 100 on machine A is "jim", user 100 could be "sally" on machine B. Moving that stuff to ldap becomes messy, now I have to reconcile the numbering schemes of all the machines I want to migrate. Ick. And you get all kinds of screwy stuff sharing folders, if you ls it on one machine it'll show wholly different ownerships.. Is the source of about a billlion and one nfs security holes.
And of course, since a file can only have one permission set - owner, user, group, it sure does make for some sucky shit. The lazy among us would just run as root all the time to avoid the whole damn mess.
I know there's a circle jerk of workarounds, patches and gotchas to avoid this, but it should never be a problem in the first place. The basic unix security model is out-of-date, and is the source of many systemic problems.
I don't need no instructions to know how to rock!!!!
While I think that Reiser will go far, I suspect that the next real inovation will occur in a distributed FS with redunancy.
I prefer the "u" in honour as it seems to be missing these days.
Honestly, EXT3 is leaps and bounds ahead of anything coming out of Microsoft or Apple these days. Journaling, extensive disk cluster customization, bit handling that's yet to be equaled... it's got it all. With continued development of EXT3 its popularity will only grow, and if the community can focus its efforts properly, it will be the choice this author is looking for.
+ Donald Gunth
+ Email: dgunth@quicktek.net
"Caffeine is the greatest lubricant ever created." -ESR
Can somebody explain me WHY should we put things like database, indexing/previewing etc. into the filesystem => KERNEL SPACE!!!! What advantage does it bring?
Any good (XFS, JFS, ext3) filesystem now has nice feature called Extended Attributes which is intented for STORING such a data (like previews etc.). And using user-space server it's much more easier to add plug-ins for various file-formats, "search" plugins etc.
Using the loopback driver's encryption options, you can encrypt any file system - be it ext2, rieserfs, or even vfat.
:).
Heck, you can even encrypt ntfs
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Current Apple systems ship with an HFS+ root filesystem, or at least the iBook I bought a few months ago did. HFS+ is journalled AFAICT, but it's pretty much the same old Mac HFS (resource forks, etc.) other than that. It's not even really case-sensitive; "touch a ; touch A" will only create one file.
So far as I can tell, the big feature of OSX 10.4 is Spotlight, which looks like a version of locatedb that knows about file contents to me. This really isn't an OS-level thing IMHO; it could be implemented in an FS-independent way in user-land without too much trouble. It seems like this would be functionality you'd want in the file browser (eg. Nautilus or Konqueror) rather than in the filesystem... although this kind of metadata indexing might be where the HFS resource forks might come in handy.
"My life's work has been to prompt others... and be forgotten." --Cyrano de Bergerac
It keep a journal file of last modifications
(ie "Replacing node 37827 with node 5279867....replaced")
One the modification is done, it erases the entry.
After a crash, the system only need to look in the journal file to know which file 'might' be corrupted and restore the old version of each...
At least, that's how I understand it...
I live in Soviet Canuckistan you insensitive clod!
I'm not moving away from EXT3 any time soon. I have tried other filesystems (including Reiser and XFS), but nothing can touch the rock solid stability of EXT. I don't mind if it's a tad slower, i can sleep easily knowing my data won't disappear overnight unless a lighning hits the HD. Directly.
I suppose one benefit of a closed source / company controled system is that they get to choose one file system and stick with it for a while. Linux on the other hand has several to choose from.
:)
I remember my first linux install, I didn't really know anything at all about it. when it came time to choose my file system, i was like, huh?
Seriously, if something like this is going to happen, there will have to be a mutual cooperation between the linux distros. Either that or a DB style file system will be so compelling that everyone will rush to use it, and a Darwin style evolution will take place
http://www.rustyrazorblade.com
both hfs+ and ntfs are both journaled. ntfs has been for a while.
Since new systems now are running with +1gig memory and with 64-bit systems over 4gig memory is possible,
why not run the entire operating system in ram instead of hd and just write differential changes in ramrootfs to disk.
It would probably slightly slow down booting, but that could be avoided by loading software to memory as needed
There are no atheists when recovering from tape backup.
Anyone? ... Bueller?
But seriously, even though he mentions Reiser, he doesn't seem to consider it's future direction, which is to allow varying degrees of structure, that could include attributes, as the user sees fit. At least that's how I understand it.
NTFS is journaled and has been for a long time, I think back to NT4 or earlier, but for sure Windows 2000.
Apple's HFS+ has been journaled since at least MacOS 10.3 if not 10.2.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
It doesn't matter who made it. It matters that it's in the product that Apple is presenting to the consumer. Just like it doesn't matter than ReiserFS was made outside of the Linux dev community if that's what filesystem I get when I install SuSE Linux.
"I've got to stop masturbating! It makes me too lazy! Stop it, Albert. Stop it." -- Albert Einstein
This has been a core feature of OpenVMS for a long time.
HFS+ is journaled, and NTFS has sort-of journaling:
Read about it here...
dtrace, due with Solaris 10 does that. So it's not quite a top equivelent, but it does laet you answear your questions ("What processes are kicking the shit out of the disk", and "By how much"), and long with the also useful "In what way" i.e. many small writes, hugh seek to read ratio, or what have you.
It is, however, expert driven, unlike top, which is simple to use. Still, I think that dtrace shows the furture of performance monitoring apps.
Note that dtrace lives partially in the kernel - it's not portable to Linux.
Actually, both have journaling filesystems.
hfs+ supports a journal (starting with macos 10.2.2 server and 10.3 panther), and ntfs5 supports a journal (starting with win2k)
All indications are that Linux, Windows, and Mac OS are moving in a common direction with filesystem innovation
Whether or not it is useful, one thing is clear: this sort of thing is not "innovation". Databases as file systems have been around for decades, as has the question of file system metadata. The UNIX choices in this area are by design, not by an accident of history, and the motivations behind those choices are as valid today as they were several decades ago.
Linux is a ways yet from having a fully attributed, database-driven, journaling filesystem. The direction of future development looks promising, though. Linux will certainly compete as the search wars come to the desktop. Linux's value to the enterprise depends on it.
There are two things one needs to keep apart: what functionality do we want to support and how do we want to support it. Search clearly is important. Metadata clearly is important. However, whether a "fully attributed, database-driven, journaling filesystem" is the best way of implementing those features is an open question. There are many possible alternative designs.
And, in fact, it seems right now as if Microsoft is, in fact, not building what the author seems to think they are building, but is choosing an implementation strategy that combines a more traditional file system with user-level databases.
It's not even really case-sensitive; "touch a ; touch A" will only create one file.
/bin/ls) it would say it wasn't found. Also if you were in the same directory as all the executable commands, sometimes it would recognize them in uppercase, sometimes it wouldn't.
Actually, i recall fiddling around with a mac once.. I seem to remember that typing "LS" would result in the same thing as "ls", but if you typed the direct path to it (/bin/LS (i think) instead of
It's really strange, and followed no reason. I don't have a mac in front of me right now to double check all that, but thenagain it may have changed since then...
do() || do_not();
I had corruption with Reiserfs.
Yes I have cheap memory, yes I have an AMD cpu.
But I know I can't trust Reiserfs on my computer.
File corruption is one of the most unacceptable errors a user can deal with.
I think the most frustrating part is I don't even know of a way to prove out my hardware so this won't happen in the future.
Linux doesn't *need* a new filesystem to keep up with windows. Windows can have it's sure-to-be-bug-ridden shiney filesystem, and I'll keep everything important that I have on something more stable with a proven track record.
Care to point out the bugs in the current NTFS that make you sure the next one will also be?
He didn't say NTFS, he said "shiney" as in new as in WinFS. Would you be in a hurry to deploy a version one MS filesystem in a production environment? While the original poster is marked "Flamebait", his point is extremely valid...
Why must some one elses change precipitate a change everywhere?
The wheel is still pretty damn good. Some people might have added rubber, or made is swivel, but the wheel is unchanged.
If the argument is that it is lacking, then I might understand. And when I say lacking, I mean, if I look at the FS by itself, not looking at its neighbors, and am wanting more.
When I say lacking, I do not mean that when compared to some other FS, it doesn't have as many bells and whistles.
Both sides accuse the other of playing catch up. Well don't play catch up, and no one can make that complaint. If an option is truly lacking, then the option will become an obvious solution to the problem space.
And neither of whom have a journaled filesystem yet, while Linux has many to choose from.
... you get the point.
What are you talking about? NTFS has had journalling for over a decade. And Unicode. And ACLs. And streams. And reparse points (these are amazingly cool). And compression. And encryption. And
Now, MS doesn't use most of this good stuff, but it's all in there. Even three-letter file extensions on Windows are obsolete, since everything on NTFS can be an OLE server. There's nothing on Linux that comes close to the capabilities of NTFS. About the only major thing NTFS is missing is versionning, which VMS has.
I've gone well past the point where my data is worth more than the total cost of a new computer, and I don't want to lose it to a HD or computer failure. I'm particularly concerned that we are digitizing our family photos and that they could poof one day. So for me the killer application for a filesytem is 100% reliability. Ideally I'd just run some sort of transparent distributed raid thing and it would just automatically copy everything to one or more of my computers. That way when one crashed I could just plug in another one and be on my way.
The relational database filesystem seems like a big boondoggle to me. We already have several free RDB products (Postgress, MYSQL, etc.) as well as stable programming interfaces to those products in numerous languages. We also have good support for small files (Reiser3 already), and we support hard links. It looks like most of that could be tried out in userland too see how well it worked without any changes to the underlying filesystems.
I'm not a fan of metadata. The WWW has shown that having extra content about the contents of your content simply does not work very well. The metadata is redundant information and is thus prone to many syncronization problems with the original data that cause it to be invalid. There is a reason that we all search the web via GOOGLE instead of some metadata scheme.
Michael
Everyone should store their data on Gmail accounts and then use google to find what you need, while letting corporations know what we are searching for so they can send us emails!!
Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"
The seamless filesystem-in-a-database was created in the Multi-Valued DB structure in the mid-60's and release as the the Pick OS. It is still sold by Raining Data and runs on Windows, Unix, and Linux.
Frankly, much of this is re-inventing the wheel. The issue isn't so much journaling or other standard features - its making the file-system more accesible for both end users and application programmers.
The filesystem in BeOS made many applications virtually obsolete. Managing MP3s, photos - even email was trivial using its standard file manager. Any mime-type could be assigned a set of default attributes which were then indexed (in real time) and could be searched on.
Searches could be saved and were updated in real time - thus the same search for any mail from a collegue over the past 5 years that might take minutes in Outlook, was instanteous merely using Be's file browser.
If the open source movement wants to look forward, it should look beyond MS and Apple to the real innovators (the once that get crushed by Microsoft before ever becoming a threat.)
+--------------------- You idiot! I told you we were facing the wrong way!
I mean I've heard about this filesystem-is-a-database concept, that it's supposed to be revolutionary and do all these supposedly nifty things, except I don't really know what those nifty things are, or any concrete reasons why adding a database to a filesystem makes it any better. My gut reaction would be that adding something like this on top of what a filesystem normally does would slow things down more than speed them up.
Anyone willing to take a stab at enightening me here?
Funny but I don't remember having to surrender my sense of humour to be able to moderate. Is this a new requirement?
If VISTA is the answer, you didn't understand the question
File system options are stupid. I hate options. In past jobs I have spent much of my time fixing other sysadmin's misconfigured filesystem options.
In each successive iteration of NFS and Sun's UFS, the two filesystems I've used consistently for over 10 years, there has been less and less need to F.W. options to get things to work, even cross-vendor NFS mounts.
BTW I like resierfs because you can grow it on the fly.
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
And I thought Windows NT 4 (Cairo) was a good release (after they got to SP4).
One line blog. I hear that they're called Twitters now.
By "Java JSR" I mean the Content Management JSR, which might be at about the right high level between "simple" filesystem and metadatabase
http://www.day.com/en/product/jsr_170.main.html
Hey, let's admit that Microsoft did a good thing with NTFS. Before I get roasted, let me say I've been working with FAT, NTFS, EXT2/3, Reiser, and others over the last 12 years, and I've had a chance to get a view of reliability, ease of recovery, etc., with several of these in production environments. I think the NTFS permissions model is one that the Linux world would welcome over the old and, I think, inadequate U-G-O scheme we continue to tolerate.
It's only funny until someone gets hurt. Then, it's hilarious.
There are some disadvantages to this approach.
First, it's minimally supported by distros. I can't just set up a Fedora system out of box, and check "use encryption" and have it do an NTFS-style decryption of the file encryption key using the password entered at login for each user to decrypt that users' files. It requires hacking around pam and maybe initscripts.
Second, if that *was* done, it would take a different filesystem per user (per key), which is a pain to maintain.
Third, it can't be enabled by users (would require root dicking around with pam and filesystems) as NTFS encryption can be.
Fourth, it can't be enabled on-the-fly (requires creating new filesystems and copying the contents over, unlike NTFS).
Fifth, it's a pain to maintain -- on NTFS, it's easy for a user to just say "I want the contents of this directory and below to be encrypted" and choose to have things encrypted on a per-directory basis. The equivalent on Linux would be having the root user be creating new filesystems (knowing the appropriate sizes in advance and wasting any excess space allocated) copying over the contents and adding mount points for every filesystem mounted.
Sixth, NTFS supports key recovery with a backup, emergency passphrase (it can maintain two copies of the encryption key, one encrypted with, say, the administrator's password). Dunno about the Linux status of this.
Having an encryption layer above the block layer is a nice idea, but it's not a drop-in substitute for encryption support in the filesystem.
It would be possible to add a layer in which an encryption layer could be *added* (preprocess file/directory contents -- if one *only* wanted encrypted files and not directories, this could already be done with an LUFS or fuse module). Space for such a layer does not currently exist in Linux.
May we never see th
Filesystems are so crucial to OS stability, that I'd say it's worth formally-verifying them to a certain extent (i.e. prove that the algorithms/code work, instead of just observing that they work in normal conditions).
P.S. The whole thing - filesystem as a DB - is complete crap. You can't do a bunch of fs operations in a single transaction and have ACID semantics on the transaction as a whole. Sure - searching is great. But database means much more than just a searching interface.
The Raven
The case-sensitivity may have changed with an update, or I modified something; my system creates 2 files with touch a; touch A
That created hell when a Dell documentation CD capitialized all file names, but the links in the HTML files used lowercase.
I have a shitty sig!
copying an idea from IBM's hugely successful OS/400
?
Exactly why hasn't Microsoft released a SQL-queryable database filesystem? They validated the architecture with their marketing years ago, after IBM proved it technologically. And its advantages are obvious. In addition to better features, it offers Microsoft the opportunity to sell its SQLServer product to serious users, with a natural upgrade path. And its an opportunity to promote the MS version of SQL across the world, raising the tide against Oracle and the rest (including MySQL). It could also make an end-run around the Samba project, which Microsoft initially helped, but now apparently fears. And of course it's a better platform on which Microsoft can offer "open yet proprietary" file formats. Is Microsoft really so incapable of actual innovation, or is there something else wrong with this picture?
--
make install -not war
With the interest in an open bios it ought to be possible to handle file systems at that level. I am sick of having to restore certain broken OSs before I can read my backups, this also sometimes happens with Linux or any other OS, so there should be a basic capability at BIOS level to dump and restore, as well as boot management, file system repair, partition resizing etc.
A bit of joined-up thinking across a number of projects could easily result in a final product that would far surpass anything that could conceivably be created in Redmond.
That's great, but is that what we (application developers) want or need? No, it is not. In fact it is extremely different from what we want.
What we want is the ability to make objects (a word processor file, etc) persist. We also want to be able to search for these persistent objects in various ways, by saying "show me all the word processing files that have so-and-so as the author, and were created on this date and contain the word 'nigritude ultramarine'". That's what we want.
A file system that allows sequential access to bytes, and creates a hierarchy of names (/your/file/is/here.txt) is really very different from that. That's why we need to have persistence tools (like JDO, XML, etc) and also search tools (Unix locate, file browsers, etc). But these are just hacks because the real problem of object persistence and retrieval isn't solved in one place.
One problem with solving it is that Linux is all C and has C mentality all through it, even at the application layer. People still scoff at object oriented design. This gets in the way of implementing cool filesystems like this.
Reiser4 does not exactly have these object oriented features, but it's much closer than anything else, and the object persistence could easily be implemented using Reiser4. I'm glad to see that Suse is using it as the default FS. I hope it becomes part of the standard Linux kernel. I also like its plug-in architecture, so we may finally get some advanced FS-layer security features in Linux.
Doesn't Palm OS have a database/filesystem hybrid too?
Uhhhmmm. Apple is NOT building a new filesystem; they made that point very clear. They are building all sorts of cool features on top of the current filesystem, and kqueue. They never mentioned kqueue by name, but come on, why do you think they put that in back a year or two ago?
I think the best thing would be to have a filesystem which didn't have any built-in organization, but let the organization be handled separately. You associate each file with a 128-bit binary handle. You can also add indirection by having handles that refer to each other. All the organization which makes this useful would be in libraries and daemons or kernel modules which associate handles with other things.
This allows arbitrary organization methods to be added later, and makes it convenient for programs to apply their own local organizations.
I've been using linux since the early 90's.
Always the stable version, didn't have problems until I tried reiserfs, switched back to ext2 ext3 actually, and I didn't have problems again.
The source of my problem appears to be resierfs directly or indirectly I don't know, like most users I don't really care either.
Doctor it hurts when I do this.
Then stop doing that.
Good enough for me.
This whole article is based on nonsense. Microsoft has a long way to go before it catches up with Linux in the filesystem area. There is no realistic prospect of Microsoft keeping pace with Linux filesystems in the foreseeable future.
(Before dismissing me a Linux fanboy, note that the above applies only to filesystems. When it comes to understanding of GUI issues, I'd make a similar statement but with Linux and Microsoft swapped. But that would be off-topic.)
Microsoft has a huge marketing department so they can please the people they are trying to sell to. Linux developers are their own market. By following Linux developers you are, more often then not, finding out what developers, like myself, want and need. By following Microsoft, you are finding what glitzy feature convinces someone that, maybe it's time to retire Office 97. The fact is, I have found few people, if any, who truly want a 3D desktop enviroment. I know others think diffrently, but i found XPs default taskbar seemed childish and condiscending.
frankly, i don't care if the casual consumer uses linux or not (though a larger market share would have some benefits). the people who develop for linux generally want and need the same things as my self and i'm happy.
That being said, faster file searching is definatly a useful tool. But if the registry in Windows is any indication of what the file system is going to look like in order for anything to get found, i don't want it.
On a random side note, App Rocket is a nifty program launcher for windows that finds files very fast.
The Neo-Bohemian Techno-Socialist
Actually, i recall fiddling around with a mac once.. I seem to remember that typing "LS" would result in the same thing as "ls", but if you typed the direct path to it (/bin/LS (i think) instead of /bin/ls) it would say it wasn't found. Also if you were in the same directory as all the executable commands, sometimes it would recognize them in uppercase, sometimes it wouldn't.
/bin/ is in my path, and ls is in /bin, so it hashes ls to /bin/ls so it knows what to do. There is no hash for LS, so it'll fail if you try to run it. However, if you type /bin/LS, then exec* will open exactly what you asked for, and execute it. The open will open the file with the case insensitive match.
It's really strange, and followed no reason.
You've got this backwards and it makes absolute perfect sense:
HFS+ does not distinguish case. open(2)/stat(2)/exec(3)/etc... would not when used on an HFS+ partition. It does, however, remember case. tcsh knows that
As far as being in the same directory, that would only apply if you have . in your path (you don't, do you?)...but the same basic rules would apply.
BTW, I like case sensitivity, but given the rules, the behavior is expected.
-- The world is watching America, and America is watching TV.
i thought there was a kernel option to get round the 4gig limit, but it turns out i was thinking of this:
64GB (HIGHMEM64G)
Select this if you have a 32-bit processor and more than 4
gigabytes of physical RAM.
64GB eh? thats in xconfig, before you compile the linux kernel. oh yes i forgot, you cant compile your kernel on windows or support 64gb of ram on windows.
I know there's a circle jerk of workarounds, patches and gotchas to avoid this, but it should never be a problem in the first place. The basic unix security model is out-of-date, and is the source of many systemic problems.
I completely agree! As will anyone with any netware security experiance. I'd love to see eDir's security model fit on top of ext3 or ReiserFS
At default, Linux EXT3 does not log access reads/fails from users (or root).
At default, Linux EXT3 does not make it easy to real-time compress folders.
At default, Linux EXT3 does not make it easy to real-time encrypt folders.
NTFS does this. Somebody fix EXT3!
Why does Linux have to come up with something "new" because M$ is? Since 96% of the users out there are unfamiliar with Linux, couldn't we just say "Hey! We have this new filesystem called ..." and put any type in there? :)
Sorry, I'm not about to trust archived video to alpha code, or even beta code. If there's no release-worthy option on Linux, we have to stick to NTFS on Windows.
In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
Both Novell and Hummingbird (aka NFS Maestro for Windows) are players in the NFSv4 working group, as is Network Appliance. NFSv4 ACLs are pretty much NTFS ACLS, not POSIX. The preamble of the NFSv4 spec announce that one of the reasons it exists is to better incorporate Windows clients and servers.
Stop spreading FUD. NFSv4 is our best hope for non-sucky network file systems, since AFS suffers from sole-vendor-ism, as well as having to bite off too much in one chunk (as in EJB coders having to 'swallow the whole elephant').
I used to be a die-hard AFS user, but now have gone laptop and CVS replaces my need for network filesystems. Disconnected activity done right!
But this whole thread is immaterial relative to the topic at hand. Network protocols != on-the-disk filesystem implementations.
HFS+ (and HFS before it, and MFS even earlier) have always been case-preserving but case-insensitive. Starting with Panther, there is the option to format volumes in a case-sensitive version of HFS+; I am unsure how well former version of the Mac OS deal with such volumes. Journalling can be enabled on either variant of HFS+ (and is now on by default).
Nae, better to do the versioning by user. If you don't want a particular daemon to do versioning, simply give it to a user with the noversion option. UID of an inode is already saved, so that makes it easier...
I'd want versioning off by default. Really it comes in most handy when I'm doing development, so in my case I'd just do dev as a user with versioning in case I make an "oops."
To take it a bit further, you could do both (enable versioning by user/file) or simply allow versioning on any file belonging to user/group X
>So how come all I can remember is the name? I can't seem to recall the OS at all.
Probably because it was renamed to Windows NT 4.0 before released.
Searching is all wonderful and that, but not the direction I believe would provide the most benefit.
Embed versioning into the filesystem. I believe Reiser has talked about this. Imagine being able to right-click on a file, folder or even partition and choose "roll back" or "restore" from the context menu. It then presents you with a list of snap-shot points you can restore to, starting with "last change".
Who backs up their hard drives any more? Have you thought of the problems and time involved in backing up 40, 80 or even 200 Gb of data? I'd MUCH MUCH rather have this feature than some enhanced search.
Learning HOW to think is more important than learning WHAT to think.
I'm an architect for a large corporation that is today trying to find a replacement for NFS. Our key goals are:
- Integration with a Kerberos SSO strategy
- Fast performance
- Cross-platform compatibility with Windows
- Robust Access Control mechanisms, RBAC would be nice but DACL is probably reality.
In my opinion, these are the primary goals that companies are looking for. Not a "journaling" file system, or built-in encryption. Sure those are nice, but let's get the basics first. Unfortunately, CIFS is still in quite a state of beta (even on the 2.6 Kernel) and there don't seem to be any real other alternatives.
Apple is simply adding functionality to HFS+. Everything you've read about Spotlight describes a services that sits above the file system. It takes advantage of HFS+, but there is NO database driven FS coming out from Apple.
Their solution is to build a service that can interact with individual files, including their native metadata (ID3 tags, pdf metadata, MS Office metadata, email headers, etc.) through metadata importers and to store the metadata indexes in a separate database. This is relatively similar to how iTunes does it's thing. The services will have lots of APIs open to apps to incorporate the functionality locally.
The obvious clue that HFS+ isn't going away is that Apple is finally pushing full HFS+ support back up to the command line utils like cp to support resource forks and whatnot in 10.4, so hopefully we can stop needing OS X specific tools like ditto.
They've been adding improvements steadily over the years, such as journaling and most recently case sensitivity. The more obvious question to me is why doesn't the Linux community just jump all over HFS+ and build off of Apple's work since they seem more than willing to give the HFS+ support back anyway?
So, write a few patches, or something. Or use another distro.
on NTFS, it's easy for a user to just say "I want the contents of this directory and below to be encrypted"
Which is silly. Why not just encrypt all partitions that store user data. /var/spool, /home, and maybe /tmp ? Or set up gpg and a public key, and install kgpg.
Get your own free personal location tracker
Does anyone else find the fact that someone is comparing WinFS to the likes of reiserfs (despite my dislike for it), XFS, and hell, even ext3?
Granted, the proposed featureset of WinFS is vastly 'superior' to that of the 3 main linux contenders, but it could be argued that WinFS is neither a filesystem itself, nor is it on par with any of the linux filesystems in terms of performance or stability (if NTFS5 is to be of any forboding).
I seem to recall reading about several projects that impliment WinFS-like features. I don't recall what they were, and I don't think they were kernel-space projects, but I recall thinking, "this looks nice".
Besides, let's be honest here. What practical functionality does WinFS provide that is above and beyond the combination of 'locate', and 'file' used in conjunction? WinFS seems to me to be merely a crude hack so as to make up for the fundamental shortcomings with MS's OS design.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
OS400 native FS is still the neatest model for DB-FS integration out there. It's unique because the File system is written to allow file-member-record-field access directly from a command line call... It's similar to what was posted above with Reiser4 and having plugins. The key to AS400 success is that the "file system driver" is pushed down into hardware controller roms so the DB like access is nearly fool-proof. Stuff like queries and SQL are just "plugins" on top of that model. The only thing I see preventing linux from adopting such a scheme is that it would require an entire dedicated HDD/partition to do properly because you need complete control of the disk structure to the FS driver...and where AS400 has very few native file types it deals in, linux would need to litteraly create a file model "plugin" for every type of Mime type..otherwise you still have a bunch of meaningless BLOBS to parse. OSS/Linux is uniquely qualified to write this because they have the "keys" to everything, but it would take enormous cooperation to implement it correctly!
From the article: However, Microsoft (with Longhorn) and Apple (with Tiger) have made it clear that they consider the filesystem of the future to be a database of information to be mined, and that client PCs will be a major part of the next chapter in the "search wars." The future of Linux may depend on whether Linux filesystems continue to innovate.
Ugh. First off, who says MS is right? Do we see people chomping at the bit to write database-like filesystem using apps? Do we have any notion that this is a good idea?
Ok, that's point one, and I can argue both sides of it. I just want to point out that it's not a foregone conclusion.
Point two: Since when did keeping up with Microsoft constitute innovation?
Reiser was the first B-tree directory-structured filesystem that performed on-par with existing filesystems. Ext3 was the first backward-compatible journaling filesystem. And of course, there are dozens of innovations that have been absorbed via JFS and XFS which were released to Linux by their owners.
I'd like to see more innovation on the sceduling, thread/process-control and inter-process-messaging fronts. I'd like to see the availability of more passive messaging (e.g. like alert(2), but for more system events than just time and user-defined events beyond just SIGUSR1 and SIGUSR2). In short I'd like the steady stream of real innovation in Linux to not get bogged down because the grass looks greener over in Redmond....
Does anyone know of a way to salvage deleted files on a Linux filesystem? This would be a really great feature, and is probably the only thing keping me from implementing a linux file server. One of the things I just love about Netware as a fileserver is how it uses the entire disk space to keep old files and overwrites them as needed for disk space. Need a deleted file back? No problem at all. A user wants a file from 2 hours and 3 versions of the same file earlier in the same day and you have no tape backup? Too easy. Want to know who deleted the file? It is part of the attributes of the deleted file now. Microsoft has tried copying this feature with the Shadow Copy service on Windows Server 2003, but their implementation is not near as slick or useful as Netware's, and i find it about useless in comparison. I really hope Novell implements this feature into Linux now that they have a large interest in Linux.
Yes, the old VMS operating system had this for its users' files by default. Doing things with file "foo" would actually create "foo;1", "foo;2", and so forth. (It's not a Unix system, so the semicolon isn't special to its shell.) Asking for file "foo" would auomatically pick the latest revision.
You quickly learned to issue the "purge old revisions" command once you smacked up against your diskspace quota repeatedly. :-) But being able to trivially look at older revisions was worth almost any hassle.
So if you google around for "linux filesystem vms" you may find something. Haven't tried looking myself.
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
It's not just Apple and Sun that are doing cool file system stuff.
. as p
http://www.eweek.com/article2/0,1759,1604449,00
the capital of egypt. whats that to do with filesystems ? wait...
This is why everytime a comparison is made of Linux and Windows is made, and Linux comes out ahead, the comparison is limited to Web Server functionality. Anything on the network requiring real, scalable access control becomes a nightmare that would scare the pants off corporate IT managers.
It's about time we took this problem seriously and recognized it is a HUGE PROBLEM TO LINUX ADOPTION!!!!
Now, MS doesn't use most of this good stuff, but it's all in there.
Which begs the question:
If they're not bothering to use most of the stuff in the current filesystem why are they creating a new one and accepting such significant conversion costs for them and their customers?
---
It's wrong that an intellectual property creator should not be rewarded for their work.
It's equally wrong that an IP creator should be rewarded too many times for the one piece of work, for exactly the same reasons.
Reform IP law and stop the M$/RIAA abuse.
I don't think the article author realized this, but the introduction of WinFS will not change the storage layer underneath, which will be the exact same version of NTFS that Windows XP/Server 2003 is using. So the filesystem PROPER won't change. However, there will be new features and services added that couple NTFS to a dedicated "Jet" instance which will discover, store, and search meta-data about these files. That part and the associated APIs is WinFS.
Considering ReiserFS is just about as featureful as NTFS is, why do we need a new file system again?
And let's not forget, even lowly ext2 supports POSIX attributes and ACLs... so really there's no excuse.
I think pushing to make sure all the modern FSs (ext2/3, JFS, ReiserFS, XFS, NFSv4) adhere to the POSIX extended attribs API in Linux is a good first step, so that you can then make Gnome/KDE take advantage of them in more cases.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
And metadata and these other features are going to make "Joe Sixpack"s life much easier. Apple's Spotlight is just one example of what metadata lets you do. There are many advantages to the end user in having a sophisticated FS.
Unix is dead. Long live the Unix.
---
Mod me down, you fucking twits. Go ahead. I dare you.
(I read with sigs off.)
There is a tendency to make the developers of Free Software responsible for hunting after every and any feature and idea MS and Apple care to implement to fish for users that is really annoying.
First, all Unix file systems since some decades have proved to just fit the bill quite fine. Searchable Metadata and other "features" is actually application-level stuff. 98% of the data on an average Desktop Unix system (and 99.5% of the data on a server) does not need that, because it rarely changes and is of no special interest for the user at all. And if it does, application-level data is better integrated, faster, and more flexible.
What is happening here (and in many other recent discussions) is dragging the Free Software community into an arms race it can not win. You can't make Linux/Gnome (or FreeBSD/KDE or whatever your favorite Open Source system may be) into a system that is just like Longhorn or Tiger but gratis and free. Never. What Free Software really needs, now more than ever, is to be picky about its users, its uses and its features. Better offer 10% of all users a system that is better than offer 90% of them a system that is a poor emulation of the OS they get for free with their PC anyway.
This is a point that the Free Software community has to (re)learn, better today than tomorrow.
Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"
Okay, okay, we get it, you hate "M$." But WinFS is coming. The API is already implemented in Longhorn builds. Not sure what you're trying to say here--that it won't ever come out? It's coming out. That it's a rip-off? Yeah, you say that on the website that supports a UNIX clone which runs a Windows clone on top of it. Do we really want to start listing off rip-offs? Because I guarantee OSS would have the longer list.
Maybe the point was that you just had to get that quota of "M$" bashing off of your chest because you knew it would get you posted...
As to the original poster's comments aobut HFS+ not being case sensitive, it's actually case-preserving, but case insensitive, Eg:
Most unix types (myself included) look at that and say "that's just wrong". That was until after I read a long discussion about the reasoning behind it. Some things to consider:
- From a programming standpoint, it is much easier to make
a case insentive or non-case preserving filesystem than it is to do what HFS does (preserve the case, but not use it for matching). So it is not a case of Apple developers being lazy, they actually wanted the current HFS behavior.
- If someone told you to go get a file named "Taxes 2004"
from a regular (physical) filing cabinet, and all you found was a file labeled "TAXES 2004" would you think it was the correct file?
- If you created two files with the above labels, would you expect anyone else to recognize they were different?
- Because HFS is case preserving, most of the things you would normall use cases for still work. [Eg, many people name all directories with a capital letter to help differentiate them from regular files when looking at a directory listing. This works fine in HFS since 'ls' will show the original case.]
Though I'm still emotionally attached to the unix implementation of case sensitivity, I'm increasingly of the opinion that the HFS behavior is the more "correct" one from a user point of view.IMHO the OSS community is underperforming in picking up new technology and good ideas.
Hardly. There are a lot of OSS projects that are leading the way with new technologies and in implementing good ideas.
But in quite a few areas it's not at all uncommon to see slow support for new tech. The community divides about how to implement the new ideas, which slows things down, but that division fosters competition and provides a base for testing out different ways of getting the new tech out the door.
Sometimes doing it well is better than doing it first.
Since my posting was moderated as a flamebait.
Following, to be mor precise.
Reiser4 offers a clean all tree based structure, with everthing handled as files, every file can store metadata, add to that an indexing mechanism and you are way beyound WinFS, which still basically is an NTFS with a btree index for metadata added.
I see nothing in this article which justifies another filesystem development, because Reiser4 is the most advanced filesystem currently on commodity level.
Another issue is how fast the tools and desktops can integrate those features.
This goes pretty much for everything except DRM in longhorn, there is already a solution in linux available or will be within this year.
As such, if it ain't broke, why fix it?
Oh yeah - it is broke. =/
Rather than trying to overtake MS on the filesystem market (something that isn't going to make/break anything, see the "shiny thing" bit above), let's just develop something that works correctly. Quickly would be cool as well, but I'm patient.
This sig no verb.
Agreed - I don't know that much about OS/400 but I know enough to know the author doesn't know sh*t about it - he seems to have picked up that OS/400 has some magic database filesystem - get over it, IT DOESN'T.
What the OS/400 has is a lot of flat files that to me look like csv - you have delimited fields in flat physical files that can be turned into fields if you look at the file using strsql (ie SQL interface to the filesystem) rather than dsppfm. BIG DEAL Now grant it you can have logical files which are to OS/400 what views are to real databases, but this isn't the file system itself as much as the different ways you can look at it - it's like saying that Windows has a "word-processor" based file system because they have tools that can open files and make them look like preprocessed text!! OK grant it, you can use strsql on an AS/400 box to get stuff to look like fields and tables, but like word files if you open them with notepad, they're not fields and tables at all, but files and seperated values.... the only difference is the TOOLS YOU USE TO OPEN THEM
Try NetBSD... safe,straightforward,useful.
LRC, the best-read libertarian site on the web
I think this whole "tree" filesystem is fundamentally flawed. Why have one root (\) when you can have... many.
I think having one "root" drive for your main hard drive would be a good idea. And as to stop user confusion, get rid of this "\" and name it a "see" drive. Extending on this philosophy, we can group other similar devices around it. A "Deigh" drive for a cd-rom, and an "eh?" drive for a floppy disk for example. The possibilities are endless.
Oh, and I also think its pivotal to have encorporated a graphical assistant built into the very core of the device drivers. One that, when you at least expect it, pops up and asks you if you need help to "view your see drive", or "delete off your eh? drive".
Well actually they use most of those features, expcept the reparse point. Which is there was for posix subsystem.(But also works under win32)
They use streams on the server OS, when you have afs enabled. They use encryption, and be enabled on any folder. They have been using journaling since win2k. And anyone using windows is using ACLs, and unicode...
I think it would great to have the ability to roll back to a version of a filesystem (cvs) and also have security based on XACML policy schema (need to define resources, etc.) but you get the idea.
1. A completely version-controlled file system, so I can backtrace through changes to files and so forth, like a continuous backup.
/usr/bin without upsetting other users /usr/bin.
2. Reiser4 style file/directory interchangability
3. A Plan 9:ish (i think, I might be confusing things) system of unifying the namespace, where users can install stuff into their
Give me these three things in a filesystem and I'll be a happy camper for quite a while, until I come up with new cool features.
...ceterum censeo Carthaginem esse delendam.
Okay lets see what would be nice:
handles 1,000,000's of files
Fast searching
Quick to recover
FAST backup interface at the FS level in a nice all over standard way that backup software vendors can easily hook into (Hmm NDMP comes to mind). (Imagine backing up filesystems that take HOURS to do nothing more than a simple find.)
How about spliting up the data from the file metadata and the locking so the whole lot can be improved separately or even (see lustre) spread over many machines for added resilience / performance.
The list of nice fs features goes on and on....
Novell's NSS for Linux should be avalible in Q4, I know it will take care of my of my issues.
Why is adaption of new filesystems is so slow? Well, you can alter either which metadata that is collected or the way data is stored.
New metadata: well, what happens when you send that file to someone else, running another filesystem? Where are you supposed to store that extra information? Wrap the file in something else? Can you say 'binhex'? :-)
Sure, internal metadata can always be gathered, like last accesstime, usage frequency, etc. But even that require that applications can use it, something which will probably take quite some time (especially when there are alternative filesystems on the same platform!).
Think about it. Apple tried to add more meta-data to the filesystem: ok, it works on the Mac but everybody else goes ballistic if you so much as mention 'fork'.
Data storage: just as with metadata, this will require that applications are written to use the new features. If the new filesystem isn't available on all platforms (like all versions of Windows) adaptation will probably be slow. If you want to make your program a success, are you sure you want require something that not everybody has?
New features as atomic writes, new kind of files, plugins, whatever: sure, but do you know how many features there are in NTFS that practically noone use? (like a stream coupled to a standard file - gives a whole new dimension to (meta)data storage).
Every new filesystem that is introduced, be it on Windows or some Unix-variant, will have a really hard time to get any marketshare to speak of. Especially if the filesystem in question is too different to the old ones.
Computer scientists have tried for decades to evolve filesystems, with very little success. If you think people are conservative and stays with the given standard solution (Windows), do you think they will stuff their valuable data in some new filesystem thingy?
You can access EXT3 partitions via some utilites in Windows; through it's not suported directly.
Try a search on http://www.download.com/ ; there were a few there but i can't recall their names.
It seemed promising.
= 9J =
At work here, the previous admin installed a number of machines with EXT3 FS on the drives. These machines (RH 8.0 - EL3) crash sporadically often giving indications the FS was at fault.
While I personally believe Redhat is known to push "unstable" releases, I was suprized that from 8.0 - EL3 the EXT3 fs was still crashing and Redhat was still offering this as default on an install.
Anyone else had better expierences with EXT3? I am curious if anyone has more information on why this FS seems so damn unstable.
For test purposes we run "out of the box" installs, so there should be no kernel tweaking or any other "anamolous" things going on with the install's or the boxes.
If we don't make light of everything, we are just stumbling in the dark - Blank
I really don't see the point in a case-sensitive file system. Remembering case but ignoring it in comparisions makes life a lot easier. Can anyone point at some application or library or kernel part or anything that depends on the case of the filename alone to tell files apart? I can't think of any -- and besides, I'd consider it very bad behaviour.
-Lars
The first feature I would like to see added would provide the capabilities of a Partition Data Set from the IBM TSO days. We would not copy what IBM did of course because they did some stupid things meant to sell more hardware. One thing was deleted files were not really deleted and you needed to "re-generate" the PDS when it filled up. Nice trick!
Suppose you had an infinit number of loop back devices and these were hidden and used internally by the file system and when you started an application you could "mount" what for many intents and purposes looks like a TARBALL and the application in question and ONLY the application in question got to see all the files in this TARBALL. Well, the files inside a "TARBALL" of this nature would probably not be compressed, but, they could be if desired... Well, that is the concept of a Partition Data Set.
In the case of a user logging in, when the shell is started a mount could take place against the user's private data set. By doing this on a shared machine, file security can be guaranteed. For export and import the system could mount a "shared" dataset.
This sort of secutiry is far superior to ACL's and anything present file systems offer for the very simple reason that normal people including systems administrators would not normally see any of the files inside one of these datasets. Consider the advantages of running an apache server where you KNOW all associated files needed by that release of apache are in a single dataset. There IS not easy way to lose a file or clobber it or accidently delete it and so forth. Next consider that when that copy of apache starts up it _could_ simply mount a set of files each of which contains the whole website for a given domain.
Upgrading to a new copy of apache would be as simple as copying in a new dataset and mounting it against the web datasets. If a glitch is found, simply restart the old copy.
Backing up a set of files becomes a simple copy operation. Replication can be accomodated as well.
Systems Administration in those old IBM mainframes was MUCH easier than with UNIX systems and this is in large part because of the way the system handled partition datasets.
------------
Now, with this we would want to be able to mark certain files as being external sort of like opening up a window, and through this window we could for instance access certain files which might be the executables and supporting scripts.
Of course people will point out we can accomplish some of this with a loop back mount. The problem with the loopback mount is that it populates the directory tree and this is what I really want to avoid. Frankly there really *IS* no reason for even a sysadmin to be able to see 90% of the files that say consitute a web server, or say PostFix, or PostgreSQL. We accomplish a lot if the executable which needs access to its supporting files has a "private" loopback and only this executable by default gets to see the mounted dataset.
--------------
Next idea is versioning the way Digital Equipment Corporation did it in the VAX. We simply append a version number and what the delete does is append a version number. With disk drive capacities heading into the stratoshere there is no reason to be conservative.
And this leads to the next idea which has been mentioned before... that is replication - across machines.
I can buy for $20 bux a machine (P1 200mHz) that can run a 20 GB hard drive and in fact I think they can run 80 GB hard drives as well. Rsync is useful, but a full replicating filesystem at the kernel level, or at least at the level of a deamon close to the kernel would mean that a machine can be backed up to another machine in perhaps another building automatically and with little effort.
Well, I'm sure other people have other things they might like to add. This is my wish list.
So, write a few patches, or something. Or use another distro.
/var/spool, /home, and maybe /tmp ?
[shrug] I've set it up, and I've had a friend set it up, and neither wanted to bother with ramming everything through pam. It isn't worth the effort.
I don't know of any major distros that allow this, though I wouldn't be surprised if one did.
Which is silly. Why not just encrypt all partitions that store user data.
A couple reasons. First, encryption has a fair amount of overhead associated with it -- generally, there are a few things that I want encrypted, but most things I really don't care about, and would prefer to not have encrypted.
Second, I use a single partition, not a set of partitions, as is the case for many home workstations.
Or set up gpg and a public key, and install kgpg.
I'm not aware of any system that does this. It doesn't mean that one couldn't be made, but the only thing gpg buys me is file-level encryption (which, FWIW, I do use), not loopback encryption.
May we never see th
... I'm still interested in the idea of using a directory metaphor for metadata.. I wonder if it's possible to adapt files (kind of like how Apple does .app directories) so you could cd into them and change metadata within files in that 'directory'..
Also, if you're gonna start trying to add DB stuff to filesystems, why not think about mounting a DB _as_ a filesystem.. Multiple systems could mount the fs in write with row-level-locking, you could use proven DB tools for robustness and recoverability, etc.. I recall Oracle doing something like this, but their costs were overkill.. Maybe with postgresql as a backend? It'd be easy to add further capabilities like ACLs, filetype metadata, etc.. new functionality with tools and ALTER TABLE...
Reiser4 has a compression plugin coming. We got gzip to work, but it consumes too much cpu, so now we are doing lzo which can compress at disk drive speed. The lzo plugin has a bug, maybe next week....
Hans
(You can email edward@namesys.com for details).
As much as this kind of thing usually results in flamewars about what a computing environment should be, it still comes down to the filing system.
/01/01 - Open Office.org resides here. /01/02 - Mozilla resides here /02-00/01a/ - root's textual docs live here /02/01a/01 - the first user's textual docs live here /02/01a/02 - the second user's textual docs live here /00/mbox - root's mail is here /03/500/mbox - the first user's mail is here /03/501/mbox - the second user's mail is here
;p
It's come to my attention that people are always complaining about not being able to find their files on most OSes. But Unix seems to confound people to no end, which makes absolutely no sense since it is VERY WELL logically structured. Unlike Windows which is just a fucking mess. The Unix philosophy behind filing your data is all about standard locations for the different types of programs, configs and data you might have on your systems. If adhered to, you can go to any Unix system and easily locate files. However, I will argue that it needs some reorganizing for today's applications. And NO "My Documents" is a stupid fucking idea for morons with the intelligence of a slime mold!!! That will NEVER happen on any systems I administer. So now, I give you how I lay out my systems these days:
First level designations in a path:
00 in the first level of the path is the root user's personal directory
01 in the first level of the path is the designation for the "applications" directory
02 in the first level of the path is the "documents" directory
03 in the first level of the path is the "users" directory (equivalent to the stupid "home" designation used by troglodytes)
Note: in directories of type 03 in the first level of the path, the second directory is just the user's id number
Second level designations in a path:
01 in the second level of the path is always the "OpenOffice.org" application directory
02 in the second level of the path is always the "Mozilla" application director
01a in the second level of the path is always "Textual" documents
This is a much easier setup once you get used to it as it makes it VERY easy to find stuff. It also makes scripting possible for searching for files and working with files in a character based setting. I don't allow my users to use anything other than three character numeric file names. I haven't heard a word out of them since I implemented this system. In general, they seem to be pleased since they no longer have to think hard about where their docs are. The docs are always within easy reach, and as an operator, my life has become considerably easier. More time to play Nethack... So get it though your heads you fucking idiots!!! Long file names based on alpha characters are nothing but a big pain in the ass. Get used to the coming paradigm of numeric filenames. It's WAY easier. Fucking idiot amateurs.
(In a Daffy Duck voice): Woohoo! Whoohoo!! Wooohooo!!!! Whoohooo!!!!
Un-news
I'd like to have an object oriented database, like Lambda MOO, but accessible from a multitude of languages, instead of just one.
What's supposed to happen in your case-insensitive filesystem when one types a ß ?(german sz character) The capitalization of this character is SS (two characters)
. . . . . . .
may u!sh 2 sm!le at dz!z bad nn.!m!tat!ion
It's not about actually making good applications for computing, it's about a losing penis competition with Microsoft and Apple.
Microsoft is the big penis. It got big through buying lots of natural herbs, getting its dick sucked by lesser folk, and having an expensive personal trainer make them do cock pushups everyday.
Apple is the small penis, but it works hard to make itself look bigger by trimming the pubes around the base of its shaft. Commonly, the pubes are trimmed into artistic mutton chops, and you'll see cockrings and piercings.
Linux is the extremely tiny penis, desperately trying to extend itself with a very painful penis pump. It's also desperately downing lots of penis enlargment pills and furiously masturbating itself in any attempt whatsoever to compete with the other guys.
The other two penises accept their size and work with it. The Linux penis tries to be the other penises.
Be your own penis, Linux!
It sounds like all you really need is a linux port of huffYUV. That'll give you nearly 2:1 on standard NTSC (dirty) analog caps. On clean HD you might even go 3:1 or more. MythTV offers this as a codec, so most of the work may already be done.
We could add a nice feature to things like the graphical file manager so that when the user clicks on "properties" they can elect to "encrypt" . We can just grab the user's password and create a cert that is used to strongly encrypt the file.
We could use this technique without telling the user and at the same time implement it in such as way that only the GUI interface can deal with it. We would not want to touch the CLI interface because we would want to preserve backward compatibility for all those old apps that don't know about our nifty new encryption scheme.
Next, when the user loses their password and the systems adminstrator has to assign a new password we of course would not tell the user about all the files the system has lost access to. This would only create fear in the mind of the user's anyway and it is much better to say nothing.
We can add some silly permission bits as well that again only the GUI interface knows about. This way the user can lose access to his own files and never know about it. When we do this, we need to set the system up so it says you NEED to be an "adminstrator" in order to access the files, and then set it up so an adminstrator actually cannot access the file, but can only change some of the permissions. We should issure no message when the adminstrator TRIES to access the file and not tell him that the system didn't let him do anything, because if we issue a message then the administrator might get the idea something is wrong with the way we implemented this.
Another thing we can do is remove all error messages that the file system can issue and instead point the finger at the application and claim there is an "Unavoidable Application Error".
We can set it up so that if for instance the user powers down say an external disk drive while it is being accessed that no message is sent to the application other than "normal END OF FILE". This is pretty obvious anyways because if the drive get powered off clearly access to the file has ended, right?
Also, we should stream line the I/O so that when an application asks for data to be written to the disk, then EVERYTHING else in the computer stops until this has been completed. We can justify this because hardware is getting cheaper and faster and writing data is the single most important activity we can do when it is requested.
Also, benchmarks are typically run "stand alone" so as to get the highest numbers and if we program the machine so that there are no "background" tasks then the benchmarks will yeild higher marks.
RIGHT?
While we are at it, why don't we get rid of the concept of "users" and just make everyone root?
Barring the fact that WinFS isn't a filesystem replacement anyway, but a service running in the background, let's see you do some brute-force searching in 2006 on the new 200GB hard drives that will be out by then.
When Longhorn can do a search for e-mail contacts who like the same MP3 playlists you like, or all the Quake 4 pictures on your hard drive, and it can return the results in 2 seconds, let me know if you still feel that vanilla ext3 is something we can stick with for years to come.
I don't think EXT3 is that great... it is painfully slow when compared to something like ReiserFS. ReiserFS4 is the future.
Meh.
And novell's NSS filesystem even blows NTFS away. NSS even has X-pass transparent file shredding at the NOS level!
I see a small problem with these "new" filesystems. Microsoft is creating their filesystem with their metadata format. Apple is adding metadata to their system in their own format. And now somebody wants to add metadata to Linux filesystems. But in what format?
Is it just me, or a lot of supposedly smart people missing the very obvious problem that you're not going to be able to exchange files between these different systems and keep the metadata? Before anybody ships a metadata-based system, I'd like to see some kind of standard defined for metadata interchange. Frankly, I think a standard is long overdue (please let filename extensions die!), and it's absolutely essential that a standard exist before different formats become too firmly entrenched.
You would think that people would have learned their lesson, but this is starting to look like the 80s all over again.
I bet it will be released with Darwin. Why reinvent the wheel?
NTFS5 is miles ahead of most Linux filesystems (reparse points, encryption, compression,... all supported transparently), only ReiserFS 4 comes close, and it's not yet in a usable state.
I'm wondering if you even know what WinFS is, comparing it to file and locate it laughable at best.
Try finding all mp3 by Brian Adams or Withney Houston on a 200Gb disk filled with 250'000 files with file and locate, you'll get the answer 10 minutes later.
With WinFS, it will take you a whooping 2 seconds maximum.
That is without talking about the user-defined attributes, etc... that make WinFS more powerful than anything of that kind on Linux.
It seems like having gzip support would be more exciting though, since presumably one could combine it with a tar plugin, and that would allow you to cd into a gzipped file like a directory to access the uncompressed data, and then cd into the uncompressed data like a directory to get at the files themselves. Then there would be a totally transparent way to access tarballs that fits into the model that, from my understanding, Reiser4 follows.
Of course it's nice to have it implemented at fs level for speed and compatibility reasons
Don't think of it as a flame---it's more like an argument that does 3d6 fire damage
'Magic database' it may not be, but don't write the author's words off that quickly. You say the files are little more than CSVs? Perhaps you failed to notice the fact that every one of these "CSV"s are rigidly defined and structured, with field descriptions?
LOL. Rigidly defined and structured? You mean that records have a set number of characters - oh yes you can take 150 characters and divide it up into 5 fields of 2 characters and the rest as spacers, but get over it, someone is someday going to deliver the same phyical file with 10 fields of 15 characters and no spacers. It's gonna break all the other cls, querys, pgms you have on that box, and you'll have to replace the logical files that use it too. Not as rigid as you make out, the field names only matter till you move them... The number of characters per "record" ain't gonna change, someone has just decided to carve the spacers up... Forward thinking decades ago, it most certainly was, and to this day remains useful, but it isn't a magic SQL filesystem like many make out, it's just a filesystem built from the ground up with the idea that it should be made easy to represent the data in whatever way suits.
Try NetBSD... safe,straightforward,useful.
s/god/ken/
i speak for myself and those who like what i say.
Reality collapses, of course, Right?
developing a local file system is nice, and i'm glad people are working on it. but what's much more interesting to me - and i think more generally useful - would be a better protocol for talking to said file systems. VFS is better than the more direct stuff that predates it, but it's still a nightmare. 9P, the file system protocol in Plan 9, is wonderful to work with. if we could get a better way to talk to file systems, doing work on file systems would become much, much easier.
the likely side effect of user-level file systems would be a huge win for the system as a whole. loopback NFS? ick.
9P patches have existed for BSD at various points. it's not hard, and it'd be a big win.
i speak for myself and those who like what i say.
Linux is a penis
i don't even want to imagine beowolf cluster of these...
I, for one, would like a filesystem with unrestricted metadata and easy access to it. Say you have a folder full of camera pictures, store a thumbnail, comment, date etc but in the filesystem where it doesn't get in the way of stupid little utils that barf on comment headers, something like Mac's resource forks. In essence each file would have a matching metadata chunk, handled by the kernel.
-Billco, Fnarg.com
It would be nice maybe if we could use OpenBeOS's file system... I have a feeling it might be under a strange licence. I.e, more strange than the GPL or LGPL :(
Giving IE users a taste of their own medicine since 2005 - http://pods.-is-a-geek.net/
I agree with the need for compression, but it would have to be something lossless and quick, at the expense of compression ratio.
I've always wanted to see the Reed-Solomon routines get integrated into the filesystem. RAID is great, an PAR2 has opened up a lot of possibilities, but I still think it would be a good option to provide in the filesystem itself.
Encryption would be another great option, but it would have to be a method that actually deserves my trust, not ike the ntfs encryption.
Hopefully Reiser4 and plugins will make all that a reality in the near future.
Do i remember Cairo? How could I forget it. It was the first time I'd ever laid eyes on you, and my thoughts have been consumed by that moment ever since.
It was hot - the sun in the sky, and the Egyptian counterintelligence on the ground. Very hot indeed. Dust filled the bazaar, crowded with people and the scent of spices. The cameras around my neck (for show, of course) seemed to grow heavier throughout the day, and the perspiration threatened to show through my white linen suit.
I stopped by a stall and pretended to peruse the knives for sale, but under the horn-rimmed sunglasses, my eyes scanned the bazaar. I was worried - you were supposed to be here an hour ago.
As I turned back into the crowd, I saw you. I will never forget that moment. Your face was mostly covered by the traditional dress of the region, but your eyes were glittering blue jewels that mesmerized me.
I stood unable to move for several seconds. Coming to my senses, I gave the pre-arranged signal: taking the hat from my head, I wiped my forehead and the back of my neck with a handkerchief, exposing the black widow spider design stitched into its corner.
Immediately, you relaxed, smiled with your eyes, turned, and walked into a dark, dusty doorway, and I followed after you.
And so I have followed you ever since. I tried to remain professional, but I was merely denying to myself what I already knew on that first day: I was doomed. It was a classic case of giving way to my heart, and in this business, that is the first step to giving away your life.
In the previous cell - the torture cell - there was nothing but blackness, and agony. Here, there is at least a crack in the ceiling. Light sometimes filters through the perpetual dust, and it falls across me during certain times of the day. In the dustbeams, I still see you. Even after everything, I see you every day, not knowing what thoughts hold court behind that beautiful, mysterious face.
The dysentery has left me quite weak, so I don't often move. There is no water here, and no food. But mostly it is the lack of water. That's what will get me in the end. I've been put here to die. In these final days, I don't think of the apple orchard in Indiana, or of my mother, or of the ice storm that trapped me in the mountains so many years ago. I think only of you, and the bazaar, and that moment I first met you. I think of... CAIRO.
pr0n - keeping monitor glass spotless since 1981.
Sounds like you organisation could really benefit from this feature. It also sounds like it would save you money. So why not use some of the potentical money savings, and offer to pay one of the Linux file system developers to develop it for you.
It sounds like you've benefited from the Linux development community, why not, by paying for further development of one of the existing file systems, recipricate.
The Internet's nature is peer to peer - 20050301_cs_profs.pdf
What are you talking about? NTFS has had journalling for over a decade. And Unicode. And ACLs. And streams. And reparse points (these are amazingly cool). And compression. And encryption. And ... you get the point.
Now, MS doesn't use most of this good stuff, but it's all in there. Even three-letter file extensions on Windows are obsolete, since everything on NTFS can be an OLE server. There's nothing on Linux that comes close to the capabilities of NTFS. About the only major thing NTFS is missing is versionning, which VMS has.
While it is true that NTFS is a journalling file system its implementation is *very feature poor*. NTFS can save you from MFT corruption which would be fatal to something like FAT but little more. NTFS' current journalling capabilites cannot provide enough data to roll back incremental changes (WinFS is supposed to have this) like Reiser and all other journalled FSes that Linux can use. NTFS' crappy journaling implementation is the main reason everyone on a Windows box STILL has to sit through those LONG volume scans if a disk isn't unmounted cleanly. In more robust FS like ReiserFS, Ext3, XFS, etc. the OS can just replay the log and redo any transactions that didn't go as planned. You cannot do this with NTFS as I understand it.
This feature alone makes your claim that "There's nothing on Linux that comes close to the capabilities of NTFS" ring hollow in my ears at least.
Also, if a feature is not implemented it might as well not exist because the user cannot derive any benefit from it. If I can't use a feature who on earth cares that it exists?
G. Washington on Government "it is force. Like fire, it is a dangerous servant and a fearful master."
> Hummingbird (aka NFS Maestro for Windows) are
> players in the NFSv4 working group
Maestro with NFSv4 support has been available
for two years.
HFS+ is journaled, and NTFS has sort-of journaling HFS+ and NTFS use exactly the same kind of journaling.
Before coming up with some sort of DBFS, some extra thoughts on security are needed.
So far the biggest reason to crack a system is to get access to its network resource, as a spam relay, a springboard/anonymizer, or a bigger pen1s. (bragging rights) Back to spam for a moment, about the only data harvested so far on those cracked systems is the address book. That's because the address book is the easiest to find - the most integrated.
Enter DBFS. Now ALL of your data becomes easy to find, *and mine*. I presume/hope there will be encryption of sensitive data, but there will always be weak passwords, weak encryption, etc. It's easy to imagine your own system cracking away all night trying to break into your own data, on behalf of someone else.
DBFS makes it easier. Granted it's currently possible to scan normal places for Quicken or MS Money, and see if any account numbers or such are there. But it's the *integration* that's the fear, because you can start doing *more* with DBFS and integrated applications. But those 'nice' features increase your vulnerability.
Not that we shouldn't do such a thing, but security needs to be a fundamental part of the architecture, not a bolt-on afterward.
The living have better things to do than to continue hating the dead.
I was able to encode realtime 640x480 NTSC video (using a 16 bit YUV source) on a 450MHz PII and that was WHILE I surfed the web. That was back when I needed to - with an XP1600 I can rip through PAL sized video (720x576) at 100fps or more - and that's in windows with a conversion filter or two in between the "in" and the "out."
Are you using RGB24? If you are using RGB you might try keeping everything RGB32 - you'll find it much faster, and that empty alpha channel won't take up much extra space :)
I don't know enough about the implementation issues to know whether these belong in the FS or should remain in some layer above it, but it always seemed odd to me that "rm" in *nix is unrecoverable unless you custom build something that makes rm an alias for something that isn't rm.
Why not have rm mark the file as being in the special "deleted" directory and have all such files fully recoverable until the moment their space is needed. (If you don't WANT a file to be recoverable, use something like rm --scrub and the file system would intentionally make it unrecoverable.)
Likewise for file renaming. If you rename a batch of files and then realize that you screwed up (perhaps sorted alphabetically instead of numerically, so they're renamed in the wrong order), you just issue an undo command of some sort and the the damage is undone.
Yes, I know this sort of thing can be a feature of your GUI, but I'm wondering why it couldn't just be a standard feature at the command line so even on servers you would have the ability to undo things and wouldn't need that very unhelpful and primitive "Are you sure...?" confirmation hack for so many operations.
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
uhhhh... what?
if the filesystem does the compression, the apps (or you) can't see it happen. that's the POINT. your suggestion, above, is ridiculous. If you had a tar.gz file, you could extract it to the FS, but it would actually be equally compressed (cause it's a gzip compressed FS), and then you could play with the files to your heart's content, without worrying about the compression, cause it's transparent. You wouldn't need or want some kinda plugin or something...
Unless the FS wasn't compressed, and you wanted a transparent way to access tar.gz files. That idea would make sense.
"that 3 years from now, the next version of Linux will wash my clothes, write my reports for me, and wipe my ass, all while I'm asleep."
That last thing can already be taken care of, now, before this week is over: For "only" about $900! and the Japanese already were using it before Linux (or Windows or MacOS for that matter) even existed, so I don't think it uses Linux...
But think about it, it's a machine that not only wipes your ass, but also keeps those ugly skid-tracks out of your underwear! No, I don't work for these guys (imagine working for an actual butt-wipe company), but ever since I got to use one for a couple of days, I know the Toto Washlet on my wishlist!
--- Hindsight is 20/20, but walking backwards is not the answer.
IIRC it should be something like that (foo should be a directory containing your stuff):
When you want to use the compressed file, use something likeYou can also use automount, by adding a line like this to your automount config file (on my machine it isTo avoid running out of loop devices that these filesystems use, you may want to add
to yourOf course, huffyuv etc. should be more suitable in your case, as other posters have said, if your software can support it.
I am working in multimedia production, including video. For lossless video compression (yuy, yuv or rgb) you can use HUFFYUV. The original site have vanished but some goggling will lend you to this.
I used it for lossless outputs, lossless hi-res projections etc... stable and all, it think it's free software.
Tomorrow is another day...
I realize that; my point was that we have more of a need to work on network filesystems than we do to work on local filesystems. Reiserfs et al. are just fine. MS isn't putting out some new thing there that's going to blow everything away.
But if I search for "Brian Adams" will I also get songs by Bryan Adams? http://www.bryanadams.com/
As far as the metadata side of the linux.com article is concerned,
many of the things talked about have been implemented and working
nicely in libferris for
quite a while now
Most filesystems these days support EA (ext3, xfs, Reiser etc) the
trick is having the system do something interesting for you with that
EA capability automatically. This is another area where libferris
helps, making EA that is trapped in files available aswell and
providing indexing over your EA. (A catch-22 is that EA for files is
supported in many kernel level filesystems but can be very slow
relative to even file access, you have to index EA in some way in
order to search for your files based on queries over your metadata).
Supporting ACID at the libferris level is a pending challenge.
While it is true that NTFS is a journalling file system its implementation is *very feature poor*. NTFS can save you from MFT corruption which would be fatal to something like FAT but little more. NTFS' current journalling capabilites cannot provide enough data to roll back incremental changes (WinFS is supposed to have this) like Reiser and all other journalled FSes that Linux can use. NTFS' crappy journaling implementation is the main reason everyone on a Windows box STILL has to sit through those LONG volume scans if a disk isn't unmounted cleanly. In more robust FS like ReiserFS, Ext3, XFS, etc. the OS can just replay the log and redo any transactions that didn't go as planned. You cannot do this with NTFS as I understand it.
I don't even have to respond. You know nothing about NTFS and even journalling in nature. Go pretend your smart somewhere people don't know how dumb you sound.
What about an open spec so that you can read and write it on non-Windows systems without having to reverse engineer? Can NTFS be used on a digital camera or Palm organizer? Not reliably.
Technically the FAT and other file systems used on these devices are not open either. They are reverse engineered, and technically illegal.
NTFS could be reverse engineered for these devices, but Win98,WinME users would not have access to the content, so providers have not used NTFS for this.
Just about everything in Linux which looks at an ext2 filesystem relies upon case matching exactly.
Contribute to civilization: ari.aynrand.org/donate
You just found the point where they will breach your data security.
It's still something that a lot of organizations will want to have before they deploy something. The IT guy is nearly always a security hole anyway -- he's setting up the computer and saying "you can trust this". You might as well trust him with recovery duties as well.
May we never see th
do you really trust the Microsoft encryption to do anything else than making your systems slower?
I'm not sure that I'd want to directly trust either the Windows or the Linux code with my life WRT the presence of zero flaws.
On the other hand, I'd rely on both of them to be pretty solid, and be comfortable using either for data encryption of sensitive, classified, or incriminating data.
May we never see th
I really thought when Veritas was making a push for vxfs, that was going to be the real dominent commerical filesystem with a real industrial volume manager. I guess they couldn't make this free to any extent, so it went no where.
I assume you're finding those particular MP3s so you can delete them. Badoom tish.
checksums are good. let's have self repairing filesystems. end bitrot forever.
WinFS exists and already runs, as opposed to your "more open, more robust, more Free and better" solutions which currently don't do much.
:)
As for Briand Adams & Whitney, well, I decline responsibility, I swear, I don't have a single CD or mp3 of these people
automatically map bad blocks to good blocks, but you can monitor the current number of bad blocks through SMART. If your drive is causing errors you had better stop using it (production environment or no) as quickly as possible. Bad blocks are almost always a sign of impending failure.
Jebus, do you want to run a production system on a drive with known bad blocks? Whoever hired you must be a complete moron.
HAND.
I wrote a longer reply which got lost (bloody crashy browser), but essentially is boils down to: The semantics are unclear without expanding the traditional UNIX system calls to include more semantic details about the context of the various operations (including allowing for real transactions). Adding this would make this unreasonably slow, because most of the time it's not actually needed (only user operations on files need this support, and it's completely wasted on anything else).
For user files you can also approximate this by just using tla on your home directory (with appropriate embedded ID tags to prevent mv's from screwing up the file "tracking").
HAND.
Set up subversion via apache, then use the path as a WebDAV store. Then all writes to the files should be versioned [I think] - that's what I've understood from reading the documentation, though I haven't tried setting it up myself.
The ability to undo after the fact would also complicate free space allocation (which is already quite nontrivial in file systems like Reiser4) because you'd have to find an (arbitrary) balance between: "Do we remove this file permanently" and "do we allocate some other free (but clearly suboptimal) blocks?". I.e.: There are lots of policy decisions which are usually best left to userspace.
HAND.
Add Plan9 and 9P to Linux. This allows interpretation of most filenames to be done by user level programs and by remote services. This gives you encryption and compression and lots of other stuff that some people think are "filesystem" things. This also gives you the ability to put the entire "database" in user space, ie "/search/foo=bar" would be a directory listing all the files in the system that match the "foo=bar" criteria, dynamically created.
Add Plan9 union mounts to Linux.
Move all of the "ioslaves" that are in KDE/Gnome and move them to using 9P. The program "cat" should be able to open any string you can type into the fanciest KDE file browser and copy the contents. Instead of colon use slash. See 9P for details.
Hit all the idiots with a clue by 4 until they figure out that a "filename" is a STREAM OF BYTES. It is not "text". This means no "canocolization" or "convert to wide characters" or "make sure it is legal UTF-8" or any of the other crap that we have been putting up with for years and years that has killed all progress in this field. Microsoft is of course the worst offender here, but new Unix designs are not so hot either, note that with locales I can easily create a file that I cannot display in KDE, that is wrong.
"Metadata" should be required to be a cache of information figured out from the file. Do you want the postagestamp image? Look for the metadata, but if it is not there, run a program that will create it. Although physically not prevented, metadata that cannot be recreated from file data should be considerd a bad idea because it will be lost if the file is moved. This gets rid of all need to transmit metadata. Also it allows the metadata to be stored on a different system than the file, which is very useful if the file is remote or you don't have permission to write it.
Using ReiserFS or EXT3 or NTFS WinFS is all trivial issues compared to this stuff.
I'd go for / and just look through the 10 photos that are stored there (possibly named after the people in the photos, but I'm lazy so I usually just leave them with names like IMGXXXX.jpg), but I digress.
My question (and I believe, the original poster's) is this: If people cannot be relied upon to even provide a file name that's meaningful, what makes you think that they'll be inclined to even provide a list of the people in photos, locations, etc.? Of course, one could just be obnoxious(sp?) and require that the user provide at least 3 pieces of metadata to identify a file. Not sure how well that would work, though.
My current naming scheme is quite simple: Make sure that the path to your data contains all the metadata that you will want to search for. Then you can trivially use "locate" to find what you're looking for. Free text search in path names is actually quite powerful once you start naming files reasonably and putting the files into directory names which contain all the metadata necessary to retrieve them.
(I would love system-wide virtual folders (backed by metadata) though,
The only place I see people relatively consistently using metadata is music, but that's only because they actually don't have to: Somebody's already done it for them and put the results on FreeDB or Musicbrainz...
In short: People are generally too lazy to make this work reasonably for most people. That's not to say I won't be using it. In fact, I'm quite looking forward to it.
HAND.
"Server crash blitzes Florida's e-voting records"o tes/
http://www.theregister.co.uk/2004/07/28/florida_v
one wonders why a server crash allows for a possibility to loose its data. Apparenlty it was e-voting records which were lost. Also very very strange. Why?
From the old days, using norton disk doctor (msdos based) and nowaways using the crash recovery kit (linux based), recovery was plain simple: use the basic rules for filesystems on disk sectors, and to loose all your data became an impossibility, except when having real physical disk damage.
With the move from a filesystem to a database driven file and document storage a couple of things happen. 1st Knowing the basic rules of filesystems and disc sectors will become a useless skill. 2nd We come dependant to database recovery skills, which a market study shows are way more expensive per hour (CALL e.g. 0800-ORACLE) as filesystem/disk recovery services.
This will have as major impact. If someone's Longhorn crashes and cannot recover itself anymore, your neighbour just might loose all its data. Why is it, I suddenly smell a dirty rat here? Do Ballmer & Co want to reshape the grand public perception that if your Windows dies, then also your data dies with it? Did he maybe get a clue from the RIAA/MPAA how to proceed further? Like how can we nail those kiddo's with all them 120Gig and bigger disks full mp3 and mpeg copyrighted multimedia content? Only the future will tell. Sofar i'm happy with running Linux on my desktop and laptop.
I guess the perfect perception goal for Ballmer might be this future quote, meaning someone lost all his windows data : "Has your PC been Longhorned?"
Now why linux.com/Novell also want Linux users to believe this, and advocate for the fast introduction of a linux database driven journalled filesystem, i don't know.
Robert
I read your post and the responses and I can't agree with you. I tested it out on a file server I have:
Tyan mobo, dual 333MHz PIII, 512M of RAM and 4 x 160G WD JB drives connected to a 3Ware 6400 in a RAID 5 configuration.
The total file system space is 480GB and it is about half full (239GB). There are about 400,000 files on the system.
I did various searches using 'locate' and 'grep'. On average the searches took 0.6 seconds... which is three times faster than WinFS. Some searches took as long as 1.8 seconds when combined with mutiple filters but that was the max. Shortest was 0.43 seconds. I didn't only search for MP3s (which are spread out all over the place) but also RPMs, tarballs, text files, video files, source files... you name it. Same results.
Now it is obvious you don't understand what 'locate' is or does. You may even know something about WinFS. You claimed somewhere that you work for MS and have seen WinFS in action. I know nothing about WinFS. I will just have to take your word for WinFS being as much as three times slower than Linux/ext3 filesystem when doing searches for filenames (shrug). This is slashdot and there are all kinds of under-informed claims made here. You might want to be a bit more careful with your employer though... lest they be embarassed when their new WinFS doesn't actually stack up.
-DU-...etc...
"Don't sweat the technique."
Reiser4 has a compression plugin coming.
What about encryption plug-in? That would be nice for those easy-to-lose USB keys.
Phillip.
Property for sale in Nice, France
Try finding all mp3 by Brian Adams or Withney Houston on a 200Gb disk filled with 250'000 files with file and locate, you'll get the answer 10 minutes later.
.mp3 | mp3info -p "%a" | grep "Brian Adams". Personally I think on any decent spec machine it should only take a few seconds.
Would it? Can someone try on their 200GB collection and post the results of: locate
Phillip.
Property for sale in Nice, France
Laugh a little. :) It wasn't a troll, sheesh.
Actually, the plugin does encryption, authentication and compression at once (only the ones that you want, of course).
one word: no
This whole thread would be slightly more useful if you actually said what else WinFS is useful for.
Particularly for you, actually. What does it give *you* that other filesystems don't?