Slashdot Mirror
The Linux Filesystem Challenge
Joe Barr writes "Mark Stone has thrown down the gauntlet for Linux filesystem developers in his thoughtful essay on Linux.com. The basic premise is that Linux must find a next-generation filesystem to keep pace with Microsoft and Apple, both of whom are promising new filesystems in a year or two. Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"
Linux must find a next-generation filesystem to keep pace
What are the winds of change saying? R..E..I..S..E..R...4...
Hans Reiser has written a white paper containing his thoughts on the design of the next major version of ReiserFS.
This is supposedly the indestructible filesystem. Compared to database driven filesystems which are still based on NTFS probably I'd preferr a slower filesystem that was just plain reliable.
Instead, try to keep up with the demands and needs of users.
...wrote "Open Sources", which you can read/buy here. He's a fairly savvy fellow...
The Army reading list
I think promising new filesystems in a year or two is a big word.
Wasn't the immediate urgency a comparable GUI to Windows or Apple?
Uselessful technology (Air-Charged
Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now.
Oh, so creating a new filesystem because everyone else is isn't posing at all, is it?
We live in a network-based universe. Local filesystems are already good - whether its just continued development in Reiser, or whatever else.
Nfs4, though - its like afs, only without the sucky stuff. AIX is now including nfs4 in its AIX5.3 release, even! With the Big Dog on board, we should realize there's wisdom in that direction ;)
Linux doesn't *need* a new filesystem to keep up with windows. Windows can have it's sure-to-be-bug-ridden shiney filesystem, and I'll keep everything important that I have on something more stable with a proven track record.
I haven't RTFA, but... Apple is just adding more meta-data to that already used by HFS+. They will still be using a journaled HFS+ and the forthcoming Spotlight will just make use of the meta-data they are adding to their FS. Many decry HFS+ as outmoded and inefficient, but with the changes Apple is making, it is looking more promising. I hope M$ puts something new out because NTFS is getting a bit long in the tooth. Just my $0.02.
Help a college student
Hans Reiser has some interesting ideas about the role of a modern file system. Here's a recent USENET post describing some of the immediately visible features of reiserfs v3. Some people have said that there was corruption in the past, but I think there are no longer any problems in recent 2.4 kernels. Namesys is now developing Reiser4, which appears to be more flexible (still needs time to stabilize though). If I had to put down my money on a future filesystem though, it would be ReiserFS.
... the BeOS file system? I heard that it was supposed to be the latest and greatest, the OS to end all OSes.
I want a disk equivalent of top - something that'll tell me what processes are kicking the shit out of the disks, and by how much.
If Linux could do that - it's more a VM thing than a filesystem - I'd stick with ext3 for years to come.
Who needs a filesystem in a database when you have a database that lives on your filesystem (updatedb). Get that updating in realtime, with more things (like permissions, access times etc.) and a lot of the work is done.
john
Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"
Yes, because if Microsoft hasn't done it yet, it's just not worth doing.
The basic premise is that Linux must find a next-generation filesystem to keep pace with Microsoft and Apple
Is Apple's new (or current) Filesystem really Apple's?
I thought it was a neutered Berkely FFS from the Darwin/Freebsd/Netbsd code they've taken.
do() || do_not();
I thought Linux did well in Egypt.
A ruler wears a crown while the rest of us wear hats. But which would you rather have when it's raining?
...this?
13-4=54/6
Nobody has come up with a compelling reason or feature to make me want to change filesystems.
Free Mac Mini Yeah, it's
Are there any encrypted filesystems available for Linux? Last time I checked (quite a while ago), there wasn't.
#!/
Filesytems are tools that will suit different purposes. Some are good for databases, some for lots of small files, some for lots of reading, some for writing, some for networks, some for streaming.
So to develop a one handy "swiss army knife" of filesystems may not be the best route. For the most part one knows what a system will be doing and can build in the most appropriate filesystem for the job.
--
I'm on a diet so I don't get FAT32.
--
Are you a Chipotle Fan?
Gnome Storage should be a step in the right direction, and it gets it right by not reinventing the wheel, just using PostgreSQL as its database engine.
This way we can test the waters without messing with the kernel. When the concept is tried, we can decide if we make PostgreSQL a required part of a GNU/Linux system, or a Hurd translator, or whatever.
Leandro Guimarães Faria Corcete DUTRA
DA, DBA, SysAdmin, Data Modeller
GNU Project, Debian GNU/Lin
Is there anything that a true database filesystem offers that something like a realtime updatedb index and maybe a background updated glimpse index of /home cant offer?
I have about 18GB of files in my main home dir, and I can search it in seconds with slocate and if I need a content search, with glimpse.
I know that this kind of database FS provides a lot of cool opportunities in terms of meta-data, but how useful is it for non-techies, who usually dont name their files coherently, let alone correct ID3 tags or other other meta-data.
..anyone got a good RTFA or source that explains how journalled filesystems work without.. um... doubling the data and filesystem?
thanks.
do() || do_not();
What do you want these next generation features for? Mainly features like access control, security, robustness, and above all organizing and sharing data.
Why not go higher level, use a reliable and simple underpinning such as ext3fs, with something like WebDAV (Distributed Authoring and Versioning) on top of it? Like SubVersion, it is based on HTTP, with specification for versioning and rich access controls.
Or maybe even go to the level of a Java JSR, so you could have a cross platform API for accessing files so it really doesn't matter what the back end is, KaZAA, Google or a DataSette, as long as your programs have a high level view of the information.
You might even end up with something liek the original TB-L Web, with everyone running their own Web server.
Of course, excluded from the above is performance, which would be ok for office type apps but not something that requires direct disk access, but perhaps the simpler file system would be most suitable for that.
Of course, I'm just rambling here, so would be happy to hear more developed responses to this suggestion...
Just make sure it is incompatible with all the current applications so we can rewrite everything. Add a cool feature or something too.
Business isn't willing to pay for products, innovation and careers, so we get brands, mortgage commercials and layoffs.
You Linux users are just jealous that us Windows users don't have a 4GB file limitation. :D
So Joe Barr who writes for Linux.com, submits a story to Slashdot (who owns Linux.com) written by another OSDN guy. Good job for editorial independance.
The more you know, the less you understand.
If,
MS can do a filesystem built on top of SQL Server then Linux should try to build one on top of MySQL.
It probably would be popular...
BTW, before somebody gets all bent outta shape the above is a joke.
Caution: Contents under pressure
they're not resolving naughty or nice queries for the reindeer chimneys presents man. Not until our new filesystem overlords are properly welcomed. ...in Japan.
Yeah, I remember them. They were just another 80's power-band along the lines of Asia and Europe...
10 MD
I hope not!
The good is that everything UNIX calls a file is an object in OS/400, and has it's own meta data and infinitely better security model than UNIX's user/group/public garbage.
The bad is that each object needs it's own set of commands to manipulate. And object names are limited to 10 characters. Blurgh! The OS/400 main file system is not hierarchical either, all object reside is one of a number of flat libraries (directories).
I am using JFS on my Gentoo server, and it's just fine - fast and stable. Not that I know much about filesystems, but the fact that IBM is behind it gives it some credibility, it would seem.
12:50 - press return.
OS X has had a journaled file system since 10.2 and it is "on" by default in 10.3
As below, so above and beyond, I imagine drawn beyond the lines of reason. Push the envelope. Watch it bend.
Maybe. Look into cryptoloop. This is a kernel option I keep saying no to each time I see it, but it seems to be an encrypted loopback filesystem.
So with cryptoloop, which seems to be parallel to loopback, you get a big chunk of disk space, mount it as a block device using cryptoloop, then format it as any filesystem you want. Cryptoloop (en|de)crypts the blocks as they pass through it. This way, you have an encrypted filesystem of whatever design you want.
Actually I just made the last paragraph up, but if I were writing a program like that, I would call it cryptoloop, so maybe cryptoloop is like that.
I know that some don't like it, but we need the option of file system versioning, so that if/when you delete half the lines in your letter/program/... you can get them back from the previous copy on disk.
There is an expectation that the application should do it, that means extra code in each application and they all do it slightly differently.
OK: need an O_NOVERSION on open(2) if the app *really* doesn't want this - eg a database.
Make the core filesystem small, robust and fast. Journalling, realtime and not much else. Make add-on modules for fancy things like ACL's, quota, compression, encryption, compatability, extended attributes, etc... Put in shims for calling attributes from a database (db or SQL or whatever)
XFS comes close, ReiserFS 4 is nice, too. The most important thing is keeping the base filesystem simple and FAST. You think NTFS is fast? Try deleting a complete Cygwin install (>30K files) It takes AGES, even from the command prompt. I've deleted 15K files (That's 15 THOUSAND files) on Reiser 3 on the same machine, it took a few seconds.
DO NOT make a database driven filesystem. Some day we will have a true, document based desktop paradigm (OpenDoc anyone?) but probably not for several years, until then we need SPEED.
My Other Computer Is A Data General Nova III.
OK we have all these DB things that seem more for meta data and seach realy thats a bit secondary to a filesystem. Most filesystems are access by applications for surprise surprise files with very little user files and lots of application files. While it might make snece to mount /home as some DB is a filesystem with piles of indeed and seachable data so the users can be even more clueless to where anything is. The rest of the system needs faster all around and cluster aware from my point of view. Versioning in the FS ala VMS would be a nice thing as well. Disks are the slowest thing on your average system with Gigabit ethernet moving more data than the highest performing single disk in the real world.
No sir I dont like it.
I thought the real trick to running a search engine was the index. As a matter of fact, I thought I heard a story on NPR about Google's intent to index a whole mess of new stuff for their engine just to deep-six the competition.
"Rocky Rococo, at your cervix!"
Wasn't Reiser4 going to do what M$ promised WinFS will do in 5 years when schLonghorn comes out? Isn't it supposed to be the same thing?
Lets get the "this generation" filesystems working correctly, shall we?
Solid, universal support for ACLs, and while we're at it, let's fix the whole user/group namespace mess Unix has with it. Let's use an SID-style id like Windows does.
For example: my small network at home, centrally authenticated through ldap.
Now, windows knows the difference between the user "jim" on local machine A, "jim" on machine B, and "jim" the domain user. They'd be shown as MACHINEA/jim, DOMAIN/jim, etc.. The various SIDs take the domain (or workstation) SID and append the UID. So if his number is 100, his sid is "long-domain-sid" + uid. So when you pass around sid tokens, you know exactly which jim you're talking about.
Now in linux, we just have numbers for users and groups. If user 100 on machine A is "jim", user 100 could be "sally" on machine B. Moving that stuff to ldap becomes messy, now I have to reconcile the numbering schemes of all the machines I want to migrate. Ick. And you get all kinds of screwy stuff sharing folders, if you ls it on one machine it'll show wholly different ownerships.. Is the source of about a billlion and one nfs security holes.
And of course, since a file can only have one permission set - owner, user, group, it sure does make for some sucky shit. The lazy among us would just run as root all the time to avoid the whole damn mess.
I know there's a circle jerk of workarounds, patches and gotchas to avoid this, but it should never be a problem in the first place. The basic unix security model is out-of-date, and is the source of many systemic problems.
I don't need no instructions to know how to rock!!!!
While I think that Reiser will go far, I suspect that the next real inovation will occur in a distributed FS with redunancy.
I prefer the "u" in honour as it seems to be missing these days.
Honestly, EXT3 is leaps and bounds ahead of anything coming out of Microsoft or Apple these days. Journaling, extensive disk cluster customization, bit handling that's yet to be equaled... it's got it all. With continued development of EXT3 its popularity will only grow, and if the community can focus its efforts properly, it will be the choice this author is looking for.
+ Donald Gunth
+ Email: dgunth@quicktek.net
"Caffeine is the greatest lubricant ever created." -ESR
Can somebody explain me WHY should we put things like database, indexing/previewing etc. into the filesystem => KERNEL SPACE!!!! What advantage does it bring?
Any good (XFS, JFS, ext3) filesystem now has nice feature called Extended Attributes which is intented for STORING such a data (like previews etc.). And using user-space server it's much more easier to add plug-ins for various file-formats, "search" plugins etc.
Using the loopback driver's encryption options, you can encrypt any file system - be it ext2, rieserfs, or even vfat.
:).
Heck, you can even encrypt ntfs
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
It keep a journal file of last modifications
(ie "Replacing node 37827 with node 5279867....replaced")
One the modification is done, it erases the entry.
After a crash, the system only need to look in the journal file to know which file 'might' be corrupted and restore the old version of each...
At least, that's how I understand it...
I live in Soviet Canuckistan you insensitive clod!
I'm not moving away from EXT3 any time soon. I have tried other filesystems (including Reiser and XFS), but nothing can touch the rock solid stability of EXT. I don't mind if it's a tad slower, i can sleep easily knowing my data won't disappear overnight unless a lighning hits the HD. Directly.
I suppose one benefit of a closed source / company controled system is that they get to choose one file system and stick with it for a while. Linux on the other hand has several to choose from.
:)
I remember my first linux install, I didn't really know anything at all about it. when it came time to choose my file system, i was like, huh?
Seriously, if something like this is going to happen, there will have to be a mutual cooperation between the linux distros. Either that or a DB style file system will be so compelling that everyone will rush to use it, and a Darwin style evolution will take place
http://www.rustyrazorblade.com
both hfs+ and ntfs are both journaled. ntfs has been for a while.
lets look at history. BeOS has a db-like filesystem, but it wasn't compelling enough for people to switch. Just because MS is make their file system db-like, doesn't mean it is a good idea. Keeping the file system focused on the basic tasks and doing them efficiently and reliably should be the goal. Adding a bunch of marketure to the file system sounds like running around in circles.
Since new systems now are running with +1gig memory and with 64-bit systems over 4gig memory is possible,
why not run the entire operating system in ram instead of hd and just write differential changes in ramrootfs to disk.
It would probably slightly slow down booting, but that could be avoided by loading software to memory as needed
There are no atheists when recovering from tape backup.
NTFS is journalled. I'm not sure of the extent or design of its journalling, but it does have it. For some anecdotal evidence, I've never lost data from non-graceful shutdowns on NTFS, ext3, Reiser, or XFS. I have lost data of FAT32 and ext2 before though. I have no clue about HFS+ (MacOS), but I bet it is journalled too.
Anyone? ... Bueller?
But seriously, even though he mentions Reiser, he doesn't seem to consider it's future direction, which is to allow varying degrees of structure, that could include attributes, as the user sees fit. At least that's how I understand it.
NTFS is journaled and has been for a long time, I think back to NT4 or earlier, but for sure Windows 2000.
Apple's HFS+ has been journaled since at least MacOS 10.3 if not 10.2.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
> to keep pace with Microsoft and Apple
AFAIK, the only things needed to get a feature like Spotlight are:
- good indexing methods
- a way to detect filesystem changes without polling
Mac OS 9 already had #1, Mac OS X has #2, but it is not used yet.
Apart from that, it will be a lot of hard work to get things working smoothly, esp. in a multi-user system where removable volumes are not exceptions (where do you store your indexes? How do you prevent user x from learning about user's y's files through the search engine?)
It doesn't matter who made it. It matters that it's in the product that Apple is presenting to the consumer. Just like it doesn't matter than ReiserFS was made outside of the Linux dev community if that's what filesystem I get when I install SuSE Linux.
"I've got to stop masturbating! It makes me too lazy! Stop it, Albert. Stop it." -- Albert Einstein
This has been a core feature of OpenVMS for a long time.
Anybody remember Cairo?
Hmmmm. I seem to remember that name. Yes, yeah. Killer OS from Microsoft. The Next Big Thing. The Mother of All OSes.
So how come all I can remember is the name? I can't seem to recall the OS at all.
Must have been the first OS to do such a good job you didn't evn notice it. Yeah, that must be it!
HFS+ is journaled, and NTFS has sort-of journaling:
Read about it here...
dtrace, due with Solaris 10 does that. So it's not quite a top equivelent, but it does laet you answear your questions ("What processes are kicking the shit out of the disk", and "By how much"), and long with the also useful "In what way" i.e. many small writes, hugh seek to read ratio, or what have you.
It is, however, expert driven, unlike top, which is simple to use. Still, I think that dtrace shows the furture of performance monitoring apps.
Note that dtrace lives partially in the kernel - it's not portable to Linux.
Actually, both have journaling filesystems.
hfs+ supports a journal (starting with macos 10.2.2 server and 10.3 panther), and ntfs5 supports a journal (starting with win2k)
All indications are that Linux, Windows, and Mac OS are moving in a common direction with filesystem innovation
Whether or not it is useful, one thing is clear: this sort of thing is not "innovation". Databases as file systems have been around for decades, as has the question of file system metadata. The UNIX choices in this area are by design, not by an accident of history, and the motivations behind those choices are as valid today as they were several decades ago.
Linux is a ways yet from having a fully attributed, database-driven, journaling filesystem. The direction of future development looks promising, though. Linux will certainly compete as the search wars come to the desktop. Linux's value to the enterprise depends on it.
There are two things one needs to keep apart: what functionality do we want to support and how do we want to support it. Search clearly is important. Metadata clearly is important. However, whether a "fully attributed, database-driven, journaling filesystem" is the best way of implementing those features is an open question. There are many possible alternative designs.
And, in fact, it seems right now as if Microsoft is, in fact, not building what the author seems to think they are building, but is choosing an implementation strategy that combines a more traditional file system with user-level databases.
I fail to see how bloating the file system would make it faster, more stable or inherently easier to search. The only way a database file system would be able to speed up search, is to index the files. That means when you save the file, it has to index the file and update the index. For large word documents, how is that suppose to be faster and more reliable? If anything, it could require 2x the amount of space to save the same file in NTFS. Those buying a new machine wouldn't care, but those with older hardware would have serious issues.
that makes me question the value of longhorn. If all I do is check email and surf the web, why the hell should I have to buy a new pentium4 4ghz and fork over 1K? that doesn't sound like progress to me. But it does sound like silly marketing hype.
that 3 years from now, the next version of Linux will wash my clothes, write my reports for me, and wipe my ass, all while I'm asleep. This will all happen three years from now. Well, maybe four years from now, depending on when I can coordinate release dates with the distributors. Windows and Mac have some SERIOUS catching up to do!
I had corruption with Reiserfs.
Yes I have cheap memory, yes I have an AMD cpu.
But I know I can't trust Reiserfs on my computer.
File corruption is one of the most unacceptable errors a user can deal with.
I think the most frustrating part is I don't even know of a way to prove out my hardware so this won't happen in the future.
Why must some one elses change precipitate a change everywhere?
The wheel is still pretty damn good. Some people might have added rubber, or made is swivel, but the wheel is unchanged.
If the argument is that it is lacking, then I might understand. And when I say lacking, I mean, if I look at the FS by itself, not looking at its neighbors, and am wanting more.
When I say lacking, I do not mean that when compared to some other FS, it doesn't have as many bells and whistles.
Both sides accuse the other of playing catch up. Well don't play catch up, and no one can make that complaint. If an option is truly lacking, then the option will become an obvious solution to the problem space.
All of the current filesystems STILL have overlooked the absolutely miserable approach to filesystem recovery via lost+found. Yes, I know, journals are supposed to help avoid that. Sorry, but it just isn't true. Absolutely nobody considers what happens when you lose the journal as well.
/lost+found and see this for yourself.
This is critical as we move more commonly beyond the Terabyte limit.
I deal with Terabyte systems all the time, and let me tell you the utter disaster which happens when you have filesystem damage, AND you lose the journal (yes, is is rare - but yes, this does happen). In a nutshell, you can end up with so many files that all normal Linux utilities start to break down. tar, find, ls, etc. None are designed for this type of scenario. Stick 12 Million files and directories into
Unfortunately we're still dealing with the same approach to lost+found as was used nearly 30 years ago: stick everything in there using very limited organization. What is needed is some simple method for separating things out.
Sadly though, I expect this issue to continue to be ignored, and the importance of it realized only when more people start gaining experience with real live multi-terabyte filesystems, and end up losing their journal. Only then will the problem of this antique approach become apparant.
And neither of whom have a journaled filesystem yet, while Linux has many to choose from.
... you get the point.
What are you talking about? NTFS has had journalling for over a decade. And Unicode. And ACLs. And streams. And reparse points (these are amazingly cool). And compression. And encryption. And
Now, MS doesn't use most of this good stuff, but it's all in there. Even three-letter file extensions on Windows are obsolete, since everything on NTFS can be an OLE server. There's nothing on Linux that comes close to the capabilities of NTFS. About the only major thing NTFS is missing is versionning, which VMS has.
I've gone well past the point where my data is worth more than the total cost of a new computer, and I don't want to lose it to a HD or computer failure. I'm particularly concerned that we are digitizing our family photos and that they could poof one day. So for me the killer application for a filesytem is 100% reliability. Ideally I'd just run some sort of transparent distributed raid thing and it would just automatically copy everything to one or more of my computers. That way when one crashed I could just plug in another one and be on my way.
The relational database filesystem seems like a big boondoggle to me. We already have several free RDB products (Postgress, MYSQL, etc.) as well as stable programming interfaces to those products in numerous languages. We also have good support for small files (Reiser3 already), and we support hard links. It looks like most of that could be tried out in userland too see how well it worked without any changes to the underlying filesystems.
I'm not a fan of metadata. The WWW has shown that having extra content about the contents of your content simply does not work very well. The metadata is redundant information and is thus prone to many syncronization problems with the original data that cause it to be invalid. There is a reason that we all search the web via GOOGLE instead of some metadata scheme.
Michael
No, and I don't recall Bob either.
Thank You Electroshock Treatments.
Everyone should store their data on Gmail accounts and then use google to find what you need, while letting corporations know what we are searching for so they can send us emails!!
Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"
The seamless filesystem-in-a-database was created in the Multi-Valued DB structure in the mid-60's and release as the the Pick OS. It is still sold by Raining Data and runs on Windows, Unix, and Linux.
Frankly, much of this is re-inventing the wheel. The issue isn't so much journaling or other standard features - its making the file-system more accesible for both end users and application programmers.
The filesystem in BeOS made many applications virtually obsolete. Managing MP3s, photos - even email was trivial using its standard file manager. Any mime-type could be assigned a set of default attributes which were then indexed (in real time) and could be searched on.
Searches could be saved and were updated in real time - thus the same search for any mail from a collegue over the past 5 years that might take minutes in Outlook, was instanteous merely using Be's file browser.
If the open source movement wants to look forward, it should look beyond MS and Apple to the real innovators (the once that get crushed by Microsoft before ever becoming a threat.)
+--------------------- You idiot! I told you we were facing the wrong way!
I mean I've heard about this filesystem-is-a-database concept, that it's supposed to be revolutionary and do all these supposedly nifty things, except I don't really know what those nifty things are, or any concrete reasons why adding a database to a filesystem makes it any better. My gut reaction would be that adding something like this on top of what a filesystem normally does would slow things down more than speed them up.
Anyone willing to take a stab at enightening me here?
Funny but I don't remember having to surrender my sense of humour to be able to moderate. Is this a new requirement?
If VISTA is the answer, you didn't understand the question
HEIL REISER!
Maybe it's a stupid question, but... What happens if I put a HD formatted in ext3 into a windows system? Will windows recognize it, or no?
(The reason I ask is because I would like to migrate one of my older storage drives into my windows machine, and throw a new bigger drive into the linux server...)
Thanks!
AFS has the advantage over NFS4 on this point alone.
No network filesystem is any good if it doesn't run on a variety of platforms and is easy to use. The popularity of Samba has shown this.
File system options are stupid. I hate options. In past jobs I have spent much of my time fixing other sysadmin's misconfigured filesystem options.
In each successive iteration of NFS and Sun's UFS, the two filesystems I've used consistently for over 10 years, there has been less and less need to F.W. options to get things to work, even cross-vendor NFS mounts.
BTW I like resierfs because you can grow it on the fly.
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
And I thought Windows NT 4 (Cairo) was a good release (after they got to SP4).
One line blog. I hear that they're called Twitters now.
By "Java JSR" I mean the Content Management JSR, which might be at about the right high level between "simple" filesystem and metadatabase
http://www.day.com/en/product/jsr_170.main.html
Hey, let's admit that Microsoft did a good thing with NTFS. Before I get roasted, let me say I've been working with FAT, NTFS, EXT2/3, Reiser, and others over the last 12 years, and I've had a chance to get a view of reliability, ease of recovery, etc., with several of these in production environments. I think the NTFS permissions model is one that the Linux world would welcome over the old and, I think, inadequate U-G-O scheme we continue to tolerate.
It's only funny until someone gets hurt. Then, it's hilarious.
Ok, I remember reading about ext3 for the first time here on slashdot, it was better & faster then ext2, also read it on redhats website.. great i thougt, so now i'm using it on my box.. but, now i hear you are all taling about Reiser.. and on their website, what do i see but ext2 is faster then ext3 :).. ohwell, now i haveto migrate to Reiser4. damn. it looks great on paper tho, all this kinda reminds me of OFS/FFS/AFS ;-))
There are some disadvantages to this approach.
First, it's minimally supported by distros. I can't just set up a Fedora system out of box, and check "use encryption" and have it do an NTFS-style decryption of the file encryption key using the password entered at login for each user to decrypt that users' files. It requires hacking around pam and maybe initscripts.
Second, if that *was* done, it would take a different filesystem per user (per key), which is a pain to maintain.
Third, it can't be enabled by users (would require root dicking around with pam and filesystems) as NTFS encryption can be.
Fourth, it can't be enabled on-the-fly (requires creating new filesystems and copying the contents over, unlike NTFS).
Fifth, it's a pain to maintain -- on NTFS, it's easy for a user to just say "I want the contents of this directory and below to be encrypted" and choose to have things encrypted on a per-directory basis. The equivalent on Linux would be having the root user be creating new filesystems (knowing the appropriate sizes in advance and wasting any excess space allocated) copying over the contents and adding mount points for every filesystem mounted.
Sixth, NTFS supports key recovery with a backup, emergency passphrase (it can maintain two copies of the encryption key, one encrypted with, say, the administrator's password). Dunno about the Linux status of this.
Having an encryption layer above the block layer is a nice idea, but it's not a drop-in substitute for encryption support in the filesystem.
It would be possible to add a layer in which an encryption layer could be *added* (preprocess file/directory contents -- if one *only* wanted encrypted files and not directories, this could already be done with an LUFS or fuse module). Space for such a layer does not currently exist in Linux.
May we never see th
Filesystems are so crucial to OS stability, that I'd say it's worth formally-verifying them to a certain extent (i.e. prove that the algorithms/code work, instead of just observing that they work in normal conditions).
P.S. The whole thing - filesystem as a DB - is complete crap. You can't do a bunch of fs operations in a single transaction and have ACID semantics on the transaction as a whole. Sure - searching is great. But database means much more than just a searching interface.
The Raven
copying an idea from IBM's hugely successful OS/400
?
Exactly why hasn't Microsoft released a SQL-queryable database filesystem? They validated the architecture with their marketing years ago, after IBM proved it technologically. And its advantages are obvious. In addition to better features, it offers Microsoft the opportunity to sell its SQLServer product to serious users, with a natural upgrade path. And its an opportunity to promote the MS version of SQL across the world, raising the tide against Oracle and the rest (including MySQL). It could also make an end-run around the Samba project, which Microsoft initially helped, but now apparently fears. And of course it's a better platform on which Microsoft can offer "open yet proprietary" file formats. Is Microsoft really so incapable of actual innovation, or is there something else wrong with this picture?
--
make install -not war
With the interest in an open bios it ought to be possible to handle file systems at that level. I am sick of having to restore certain broken OSs before I can read my backups, this also sometimes happens with Linux or any other OS, so there should be a basic capability at BIOS level to dump and restore, as well as boot management, file system repair, partition resizing etc.
A bit of joined-up thinking across a number of projects could easily result in a final product that would far surpass anything that could conceivably be created in Redmond.
That's great, but is that what we (application developers) want or need? No, it is not. In fact it is extremely different from what we want.
What we want is the ability to make objects (a word processor file, etc) persist. We also want to be able to search for these persistent objects in various ways, by saying "show me all the word processing files that have so-and-so as the author, and were created on this date and contain the word 'nigritude ultramarine'". That's what we want.
A file system that allows sequential access to bytes, and creates a hierarchy of names (/your/file/is/here.txt) is really very different from that. That's why we need to have persistence tools (like JDO, XML, etc) and also search tools (Unix locate, file browsers, etc). But these are just hacks because the real problem of object persistence and retrieval isn't solved in one place.
One problem with solving it is that Linux is all C and has C mentality all through it, even at the application layer. People still scoff at object oriented design. This gets in the way of implementing cool filesystems like this.
Reiser4 does not exactly have these object oriented features, but it's much closer than anything else, and the object persistence could easily be implemented using Reiser4. I'm glad to see that Suse is using it as the default FS. I hope it becomes part of the standard Linux kernel. I also like its plug-in architecture, so we may finally get some advanced FS-layer security features in Linux.
Doesn't Palm OS have a database/filesystem hybrid too?
Uhhhmmm. Apple is NOT building a new filesystem; they made that point very clear. They are building all sorts of cool features on top of the current filesystem, and kqueue. They never mentioned kqueue by name, but come on, why do you think they put that in back a year or two ago?
I think the best thing would be to have a filesystem which didn't have any built-in organization, but let the organization be handled separately. You associate each file with a 128-bit binary handle. You can also add indirection by having handles that refer to each other. All the organization which makes this useful would be in libraries and daemons or kernel modules which associate handles with other things.
This allows arbitrary organization methods to be added later, and makes it convenient for programs to apply their own local organizations.
I've been using linux since the early 90's.
Always the stable version, didn't have problems until I tried reiserfs, switched back to ext2 ext3 actually, and I didn't have problems again.
The source of my problem appears to be resierfs directly or indirectly I don't know, like most users I don't really care either.
Doctor it hurts when I do this.
Then stop doing that.
Good enough for me.
This whole article is based on nonsense. Microsoft has a long way to go before it catches up with Linux in the filesystem area. There is no realistic prospect of Microsoft keeping pace with Linux filesystems in the foreseeable future.
(Before dismissing me a Linux fanboy, note that the above applies only to filesystems. When it comes to understanding of GUI issues, I'd make a similar statement but with Linux and Microsoft swapped. But that would be off-topic.)
Microsoft has a huge marketing department so they can please the people they are trying to sell to. Linux developers are their own market. By following Linux developers you are, more often then not, finding out what developers, like myself, want and need. By following Microsoft, you are finding what glitzy feature convinces someone that, maybe it's time to retire Office 97. The fact is, I have found few people, if any, who truly want a 3D desktop enviroment. I know others think diffrently, but i found XPs default taskbar seemed childish and condiscending.
frankly, i don't care if the casual consumer uses linux or not (though a larger market share would have some benefits). the people who develop for linux generally want and need the same things as my self and i'm happy.
That being said, faster file searching is definatly a useful tool. But if the registry in Windows is any indication of what the file system is going to look like in order for anything to get found, i don't want it.
On a random side note, App Rocket is a nifty program launcher for windows that finds files very fast.
The Neo-Bohemian Techno-Socialist
You do realize that Apple hired some of the developers of the BeFS system to work on the FS in tiger...which will be the first "mainstream" database-like system released!
so, why fix it?
I, for one, will stick with ext3.
I know there's a circle jerk of workarounds, patches and gotchas to avoid this, but it should never be a problem in the first place. The basic unix security model is out-of-date, and is the source of many systemic problems.
I completely agree! As will anyone with any netware security experiance. I'd love to see eDir's security model fit on top of ext3 or ReiserFS
At default, Linux EXT3 does not log access reads/fails from users (or root).
At default, Linux EXT3 does not make it easy to real-time compress folders.
At default, Linux EXT3 does not make it easy to real-time encrypt folders.
NTFS does this. Somebody fix EXT3!
"* Reiser4 uses dancing trees, which obsolete the balanced tree algorithms used in databases (see farther down). "
Any databases using "dancing trees"?
"* Reiser4 is based on plugins, which means that it will attract many outside contributors, and you'll be able to upgrade to their innovations without reformatting your disk. If you like to code, you'll really like plugins...."
What language will they be written in?
Why does Linux have to come up with something "new" because M$ is? Since 96% of the users out there are unfamiliar with Linux, couldn't we just say "Hey! We have this new filesystem called ..." and put any type in there? :)
Sorry, I'm not about to trust archived video to alpha code, or even beta code. If there's no release-worthy option on Linux, we have to stick to NTFS on Windows.
In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
"http://www.gnome.org/projects/beagle/"
I do believe there's an RPM for a beagle-epiphany plugin
HFS+ (and HFS before it, and MFS even earlier) have always been case-preserving but case-insensitive. Starting with Panther, there is the option to format volumes in a case-sensitive version of HFS+; I am unsure how well former version of the Mac OS deal with such volumes. Journalling can be enabled on either variant of HFS+ (and is now on by default).
Nae, better to do the versioning by user. If you don't want a particular daemon to do versioning, simply give it to a user with the noversion option. UID of an inode is already saved, so that makes it easier...
I'd want versioning off by default. Really it comes in most handy when I'm doing development, so in my case I'd just do dev as a user with versioning in case I make an "oops."
To take it a bit further, you could do both (enable versioning by user/file) or simply allow versioning on any file belonging to user/group X
*plink*
Searching is all wonderful and that, but not the direction I believe would provide the most benefit.
Embed versioning into the filesystem. I believe Reiser has talked about this. Imagine being able to right-click on a file, folder or even partition and choose "roll back" or "restore" from the context menu. It then presents you with a list of snap-shot points you can restore to, starting with "last change".
Who backs up their hard drives any more? Have you thought of the problems and time involved in backing up 40, 80 or even 200 Gb of data? I'd MUCH MUCH rather have this feature than some enhanced search.
Learning HOW to think is more important than learning WHAT to think.
I'm an architect for a large corporation that is today trying to find a replacement for NFS. Our key goals are:
- Integration with a Kerberos SSO strategy
- Fast performance
- Cross-platform compatibility with Windows
- Robust Access Control mechanisms, RBAC would be nice but DACL is probably reality.
In my opinion, these are the primary goals that companies are looking for. Not a "journaling" file system, or built-in encryption. Sure those are nice, but let's get the basics first. Unfortunately, CIFS is still in quite a state of beta (even on the 2.6 Kernel) and there don't seem to be any real other alternatives.
Apple is simply adding functionality to HFS+. Everything you've read about Spotlight describes a services that sits above the file system. It takes advantage of HFS+, but there is NO database driven FS coming out from Apple.
Their solution is to build a service that can interact with individual files, including their native metadata (ID3 tags, pdf metadata, MS Office metadata, email headers, etc.) through metadata importers and to store the metadata indexes in a separate database. This is relatively similar to how iTunes does it's thing. The services will have lots of APIs open to apps to incorporate the functionality locally.
The obvious clue that HFS+ isn't going away is that Apple is finally pushing full HFS+ support back up to the command line utils like cp to support resource forks and whatnot in 10.4, so hopefully we can stop needing OS X specific tools like ditto.
They've been adding improvements steadily over the years, such as journaling and most recently case sensitivity. The more obvious question to me is why doesn't the Linux community just jump all over HFS+ and build off of Apple's work since they seem more than willing to give the HFS+ support back anyway?
So, write a few patches, or something. Or use another distro.
on NTFS, it's easy for a user to just say "I want the contents of this directory and below to be encrypted"
Which is silly. Why not just encrypt all partitions that store user data. /var/spool, /home, and maybe /tmp ? Or set up gpg and a public key, and install kgpg.
Get your own free personal location tracker
Does anyone else find the fact that someone is comparing WinFS to the likes of reiserfs (despite my dislike for it), XFS, and hell, even ext3?
Granted, the proposed featureset of WinFS is vastly 'superior' to that of the 3 main linux contenders, but it could be argued that WinFS is neither a filesystem itself, nor is it on par with any of the linux filesystems in terms of performance or stability (if NTFS5 is to be of any forboding).
I seem to recall reading about several projects that impliment WinFS-like features. I don't recall what they were, and I don't think they were kernel-space projects, but I recall thinking, "this looks nice".
Besides, let's be honest here. What practical functionality does WinFS provide that is above and beyond the combination of 'locate', and 'file' used in conjunction? WinFS seems to me to be merely a crude hack so as to make up for the fundamental shortcomings with MS's OS design.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
OS400 native FS is still the neatest model for DB-FS integration out there. It's unique because the File system is written to allow file-member-record-field access directly from a command line call... It's similar to what was posted above with Reiser4 and having plugins. The key to AS400 success is that the "file system driver" is pushed down into hardware controller roms so the DB like access is nearly fool-proof. Stuff like queries and SQL are just "plugins" on top of that model. The only thing I see preventing linux from adopting such a scheme is that it would require an entire dedicated HDD/partition to do properly because you need complete control of the disk structure to the FS driver...and where AS400 has very few native file types it deals in, linux would need to litteraly create a file model "plugin" for every type of Mime type..otherwise you still have a bunch of meaningless BLOBS to parse. OSS/Linux is uniquely qualified to write this because they have the "keys" to everything, but it would take enormous cooperation to implement it correctly!
From the article: However, Microsoft (with Longhorn) and Apple (with Tiger) have made it clear that they consider the filesystem of the future to be a database of information to be mined, and that client PCs will be a major part of the next chapter in the "search wars." The future of Linux may depend on whether Linux filesystems continue to innovate.
Ugh. First off, who says MS is right? Do we see people chomping at the bit to write database-like filesystem using apps? Do we have any notion that this is a good idea?
Ok, that's point one, and I can argue both sides of it. I just want to point out that it's not a foregone conclusion.
Point two: Since when did keeping up with Microsoft constitute innovation?
Reiser was the first B-tree directory-structured filesystem that performed on-par with existing filesystems. Ext3 was the first backward-compatible journaling filesystem. And of course, there are dozens of innovations that have been absorbed via JFS and XFS which were released to Linux by their owners.
I'd like to see more innovation on the sceduling, thread/process-control and inter-process-messaging fronts. I'd like to see the availability of more passive messaging (e.g. like alert(2), but for more system events than just time and user-defined events beyond just SIGUSR1 and SIGUSR2). In short I'd like the steady stream of real innovation in Linux to not get bogged down because the grass looks greener over in Redmond....
Does anyone know of a way to salvage deleted files on a Linux filesystem? This would be a really great feature, and is probably the only thing keping me from implementing a linux file server. One of the things I just love about Netware as a fileserver is how it uses the entire disk space to keep old files and overwrites them as needed for disk space. Need a deleted file back? No problem at all. A user wants a file from 2 hours and 3 versions of the same file earlier in the same day and you have no tape backup? Too easy. Want to know who deleted the file? It is part of the attributes of the deleted file now. Microsoft has tried copying this feature with the Shadow Copy service on Windows Server 2003, but their implementation is not near as slick or useful as Netware's, and i find it about useless in comparison. I really hope Novell implements this feature into Linux now that they have a large interest in Linux.
Yes, the old VMS operating system had this for its users' files by default. Doing things with file "foo" would actually create "foo;1", "foo;2", and so forth. (It's not a Unix system, so the semicolon isn't special to its shell.) Asking for file "foo" would auomatically pick the latest revision.
You quickly learned to issue the "purge old revisions" command once you smacked up against your diskspace quota repeatedly. :-) But being able to trivially look at older revisions was worth almost any hassle.
So if you google around for "linux filesystem vms" you may find something. Haven't tried looking myself.
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
I've suggested this months back, but I would have UID/GID as the base, and ACL's on top of that, so you'd have the best of course-grain/ fine-grain security. Backwards compatiable, because the software would only pay attention to what it understands.
It's not just Apple and Sun that are doing cool file system stuff.
. as p
http://www.eweek.com/article2/0,1759,1604449,00
the capital of egypt. whats that to do with filesystems ? wait...
This is why everytime a comparison is made of Linux and Windows is made, and Linux comes out ahead, the comparison is limited to Web Server functionality. Anything on the network requiring real, scalable access control becomes a nightmare that would scare the pants off corporate IT managers.
It's about time we took this problem seriously and recognized it is a HUGE PROBLEM TO LINUX ADOPTION!!!!
Now, MS doesn't use most of this good stuff, but it's all in there.
Which begs the question:
If they're not bothering to use most of the stuff in the current filesystem why are they creating a new one and accepting such significant conversion costs for them and their customers?
---
It's wrong that an intellectual property creator should not be rewarded for their work.
It's equally wrong that an IP creator should be rewarded too many times for the one piece of work, for exactly the same reasons.
Reform IP law and stop the M$/RIAA abuse.
I don't think the article author realized this, but the introduction of WinFS will not change the storage layer underneath, which will be the exact same version of NTFS that Windows XP/Server 2003 is using. So the filesystem PROPER won't change. However, there will be new features and services added that couple NTFS to a dedicated "Jet" instance which will discover, store, and search meta-data about these files. That part and the associated APIs is WinFS.
Considering ReiserFS is just about as featureful as NTFS is, why do we need a new file system again?
And let's not forget, even lowly ext2 supports POSIX attributes and ACLs... so really there's no excuse.
I think pushing to make sure all the modern FSs (ext2/3, JFS, ReiserFS, XFS, NFSv4) adhere to the POSIX extended attribs API in Linux is a good first step, so that you can then make Gnome/KDE take advantage of them in more cases.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Unix is dead. Long live the Unix.
---
Mod me down, you fucking twits. Go ahead. I dare you.
(I read with sigs off.)
It doesn't exist yet, therefore, doesn't work on windows.
Hummingbird is non-free. Novell may or may not make the windows client free.
"No, the entire OSS community is not an echo chamber, but the kernel development community is. I've seen flamefests caused by some poor soul suggesting very minor changes to, for example, the semantics of pipes. Unix isn't just written in stone, it's laminated and stored in an evacuated nuclear-blastproof case 500 meters underground."
Of course. One breaking things for the sake of breaking them is bad. Two remember people use computers to get work done. Not to show how geeky they are. Three unfortunately a lot of the ideas tossed around on the kernel list, are grounded in ignorance of the problem space. Fourth I would recommend you give a hand to one of the other OSes out there. Not for the reasons you think, but to illustrate in the most concrete manner possible, an OS is an excercise in the art and science of compromises. Of hardware, software, and people.
There is a tendency to make the developers of Free Software responsible for hunting after every and any feature and idea MS and Apple care to implement to fish for users that is really annoying.
First, all Unix file systems since some decades have proved to just fit the bill quite fine. Searchable Metadata and other "features" is actually application-level stuff. 98% of the data on an average Desktop Unix system (and 99.5% of the data on a server) does not need that, because it rarely changes and is of no special interest for the user at all. And if it does, application-level data is better integrated, faster, and more flexible.
What is happening here (and in many other recent discussions) is dragging the Free Software community into an arms race it can not win. You can't make Linux/Gnome (or FreeBSD/KDE or whatever your favorite Open Source system may be) into a system that is just like Longhorn or Tiger but gratis and free. Never. What Free Software really needs, now more than ever, is to be picky about its users, its uses and its features. Better offer 10% of all users a system that is better than offer 90% of them a system that is a poor emulation of the OS they get for free with their PC anyway.
This is a point that the Free Software community has to (re)learn, better today than tomorrow.
Never mind that Microsoft has been promising its "innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) for more than ten years now. Anybody remember Cairo?"
Okay, okay, we get it, you hate "M$." But WinFS is coming. The API is already implemented in Longhorn builds. Not sure what you're trying to say here--that it won't ever come out? It's coming out. That it's a rip-off? Yeah, you say that on the website that supports a UNIX clone which runs a Windows clone on top of it. Do we really want to start listing off rip-offs? Because I guarantee OSS would have the longer list.
Maybe the point was that you just had to get that quota of "M$" bashing off of your chest because you knew it would get you posted...
You must be new here.
IMHO the OSS community is underperforming in picking up new technology and good ideas.
Hardly. There are a lot of OSS projects that are leading the way with new technologies and in implementing good ideas.
But in quite a few areas it's not at all uncommon to see slow support for new tech. The community divides about how to implement the new ideas, which slows things down, but that division fosters competition and provides a base for testing out different ways of getting the new tech out the door.
Sometimes doing it well is better than doing it first.
Since my posting was moderated as a flamebait.
Following, to be mor precise.
Reiser4 offers a clean all tree based structure, with everthing handled as files, every file can store metadata, add to that an indexing mechanism and you are way beyound WinFS, which still basically is an NTFS with a btree index for metadata added.
I see nothing in this article which justifies another filesystem development, because Reiser4 is the most advanced filesystem currently on commodity level.
Another issue is how fast the tools and desktops can integrate those features.
This goes pretty much for everything except DRM in longhorn, there is already a solution in linux available or will be within this year.
As such, if it ain't broke, why fix it?
Oh yeah - it is broke. =/
Rather than trying to overtake MS on the filesystem market (something that isn't going to make/break anything, see the "shiny thing" bit above), let's just develop something that works correctly. Quickly would be cool as well, but I'm patient.
This sig no verb.
I think this whole "tree" filesystem is fundamentally flawed. Why have one root (\) when you can have... many.
I think having one "root" drive for your main hard drive would be a good idea. And as to stop user confusion, get rid of this "\" and name it a "see" drive. Extending on this philosophy, we can group other similar devices around it. A "Deigh" drive for a cd-rom, and an "eh?" drive for a floppy disk for example. The possibilities are endless.
Oh, and I also think its pivotal to have encorporated a graphical assistant built into the very core of the device drivers. One that, when you at least expect it, pops up and asks you if you need help to "view your see drive", or "delete off your eh? drive".
Well actually they use most of those features, expcept the reparse point. Which is there was for posix subsystem.(But also works under win32)
They use streams on the server OS, when you have afs enabled. They use encryption, and be enabled on any folder. They have been using journaling since win2k. And anyone using windows is using ACLs, and unicode...
I think it would great to have the ability to roll back to a version of a filesystem (cvs) and also have security based on XACML policy schema (need to define resources, etc.) but you get the idea.
1. A completely version-controlled file system, so I can backtrace through changes to files and so forth, like a continuous backup.
/usr/bin without upsetting other users /usr/bin.
2. Reiser4 style file/directory interchangability
3. A Plan 9:ish (i think, I might be confusing things) system of unifying the namespace, where users can install stuff into their
Give me these three things in a filesystem and I'll be a happy camper for quite a while, until I come up with new cool features.
...ceterum censeo Carthaginem esse delendam.
Okay lets see what would be nice:
handles 1,000,000's of files
Fast searching
Quick to recover
FAST backup interface at the FS level in a nice all over standard way that backup software vendors can easily hook into (Hmm NDMP comes to mind). (Imagine backing up filesystems that take HOURS to do nothing more than a simple find.)
How about spliting up the data from the file metadata and the locking so the whole lot can be improved separately or even (see lustre) spread over many machines for added resilience / performance.
The list of nice fs features goes on and on....
Novell's NSS for Linux should be avalible in Q4, I know it will take care of my of my issues.
Why is adaption of new filesystems is so slow? Well, you can alter either which metadata that is collected or the way data is stored.
New metadata: well, what happens when you send that file to someone else, running another filesystem? Where are you supposed to store that extra information? Wrap the file in something else? Can you say 'binhex'? :-)
Sure, internal metadata can always be gathered, like last accesstime, usage frequency, etc. But even that require that applications can use it, something which will probably take quite some time (especially when there are alternative filesystems on the same platform!).
Think about it. Apple tried to add more meta-data to the filesystem: ok, it works on the Mac but everybody else goes ballistic if you so much as mention 'fork'.
Data storage: just as with metadata, this will require that applications are written to use the new features. If the new filesystem isn't available on all platforms (like all versions of Windows) adaptation will probably be slow. If you want to make your program a success, are you sure you want require something that not everybody has?
New features as atomic writes, new kind of files, plugins, whatever: sure, but do you know how many features there are in NTFS that practically noone use? (like a stream coupled to a standard file - gives a whole new dimension to (meta)data storage).
Every new filesystem that is introduced, be it on Windows or some Unix-variant, will have a really hard time to get any marketshare to speak of. Especially if the filesystem in question is too different to the old ones.
Computer scientists have tried for decades to evolve filesystems, with very little success. If you think people are conservative and stays with the given standard solution (Windows), do you think they will stuff their valuable data in some new filesystem thingy?
It seemed promising.
= 9J =
At work here, the previous admin installed a number of machines with EXT3 FS on the drives. These machines (RH 8.0 - EL3) crash sporadically often giving indications the FS was at fault.
While I personally believe Redhat is known to push "unstable" releases, I was suprized that from 8.0 - EL3 the EXT3 fs was still crashing and Redhat was still offering this as default on an install.
Anyone else had better expierences with EXT3? I am curious if anyone has more information on why this FS seems so damn unstable.
For test purposes we run "out of the box" installs, so there should be no kernel tweaking or any other "anamolous" things going on with the install's or the boxes.
If we don't make light of everything, we are just stumbling in the dark - Blank
I really don't see the point in a case-sensitive file system. Remembering case but ignoring it in comparisions makes life a lot easier. Can anyone point at some application or library or kernel part or anything that depends on the case of the filename alone to tell files apart? I can't think of any -- and besides, I'd consider it very bad behaviour.
-Lars
The first feature I would like to see added would provide the capabilities of a Partition Data Set from the IBM TSO days. We would not copy what IBM did of course because they did some stupid things meant to sell more hardware. One thing was deleted files were not really deleted and you needed to "re-generate" the PDS when it filled up. Nice trick!
Suppose you had an infinit number of loop back devices and these were hidden and used internally by the file system and when you started an application you could "mount" what for many intents and purposes looks like a TARBALL and the application in question and ONLY the application in question got to see all the files in this TARBALL. Well, the files inside a "TARBALL" of this nature would probably not be compressed, but, they could be if desired... Well, that is the concept of a Partition Data Set.
In the case of a user logging in, when the shell is started a mount could take place against the user's private data set. By doing this on a shared machine, file security can be guaranteed. For export and import the system could mount a "shared" dataset.
This sort of secutiry is far superior to ACL's and anything present file systems offer for the very simple reason that normal people including systems administrators would not normally see any of the files inside one of these datasets. Consider the advantages of running an apache server where you KNOW all associated files needed by that release of apache are in a single dataset. There IS not easy way to lose a file or clobber it or accidently delete it and so forth. Next consider that when that copy of apache starts up it _could_ simply mount a set of files each of which contains the whole website for a given domain.
Upgrading to a new copy of apache would be as simple as copying in a new dataset and mounting it against the web datasets. If a glitch is found, simply restart the old copy.
Backing up a set of files becomes a simple copy operation. Replication can be accomodated as well.
Systems Administration in those old IBM mainframes was MUCH easier than with UNIX systems and this is in large part because of the way the system handled partition datasets.
------------
Now, with this we would want to be able to mark certain files as being external sort of like opening up a window, and through this window we could for instance access certain files which might be the executables and supporting scripts.
Of course people will point out we can accomplish some of this with a loop back mount. The problem with the loopback mount is that it populates the directory tree and this is what I really want to avoid. Frankly there really *IS* no reason for even a sysadmin to be able to see 90% of the files that say consitute a web server, or say PostFix, or PostgreSQL. We accomplish a lot if the executable which needs access to its supporting files has a "private" loopback and only this executable by default gets to see the mounted dataset.
--------------
Next idea is versioning the way Digital Equipment Corporation did it in the VAX. We simply append a version number and what the delete does is append a version number. With disk drive capacities heading into the stratoshere there is no reason to be conservative.
And this leads to the next idea which has been mentioned before... that is replication - across machines.
I can buy for $20 bux a machine (P1 200mHz) that can run a 20 GB hard drive and in fact I think they can run 80 GB hard drives as well. Rsync is useful, but a full replicating filesystem at the kernel level, or at least at the level of a deamon close to the kernel would mean that a machine can be backed up to another machine in perhaps another building automatically and with little effort.
Well, I'm sure other people have other things they might like to add. This is my wish list.
So, write a few patches, or something. Or use another distro.
/var/spool, /home, and maybe /tmp ?
[shrug] I've set it up, and I've had a friend set it up, and neither wanted to bother with ramming everything through pam. It isn't worth the effort.
I don't know of any major distros that allow this, though I wouldn't be surprised if one did.
Which is silly. Why not just encrypt all partitions that store user data.
A couple reasons. First, encryption has a fair amount of overhead associated with it -- generally, there are a few things that I want encrypted, but most things I really don't care about, and would prefer to not have encrypted.
Second, I use a single partition, not a set of partitions, as is the case for many home workstations.
Or set up gpg and a public key, and install kgpg.
I'm not aware of any system that does this. It doesn't mean that one couldn't be made, but the only thing gpg buys me is file-level encryption (which, FWIW, I do use), not loopback encryption.
May we never see th
... I'm still interested in the idea of using a directory metaphor for metadata.. I wonder if it's possible to adapt files (kind of like how Apple does .app directories) so you could cd into them and change metadata within files in that 'directory'..
Also, if you're gonna start trying to add DB stuff to filesystems, why not think about mounting a DB _as_ a filesystem.. Multiple systems could mount the fs in write with row-level-locking, you could use proven DB tools for robustness and recoverability, etc.. I recall Oracle doing something like this, but their costs were overkill.. Maybe with postgresql as a backend? It'd be easy to add further capabilities like ACLs, filetype metadata, etc.. new functionality with tools and ALTER TABLE...
Reiser4 has a compression plugin coming. We got gzip to work, but it consumes too much cpu, so now we are doing lzo which can compress at disk drive speed. The lzo plugin has a bug, maybe next week....
Hans
(You can email edward@namesys.com for details).
As much as this kind of thing usually results in flamewars about what a computing environment should be, it still comes down to the filing system.
/01/01 - Open Office.org resides here. /01/02 - Mozilla resides here /02-00/01a/ - root's textual docs live here /02/01a/01 - the first user's textual docs live here /02/01a/02 - the second user's textual docs live here /00/mbox - root's mail is here /03/500/mbox - the first user's mail is here /03/501/mbox - the second user's mail is here
;p
It's come to my attention that people are always complaining about not being able to find their files on most OSes. But Unix seems to confound people to no end, which makes absolutely no sense since it is VERY WELL logically structured. Unlike Windows which is just a fucking mess. The Unix philosophy behind filing your data is all about standard locations for the different types of programs, configs and data you might have on your systems. If adhered to, you can go to any Unix system and easily locate files. However, I will argue that it needs some reorganizing for today's applications. And NO "My Documents" is a stupid fucking idea for morons with the intelligence of a slime mold!!! That will NEVER happen on any systems I administer. So now, I give you how I lay out my systems these days:
First level designations in a path:
00 in the first level of the path is the root user's personal directory
01 in the first level of the path is the designation for the "applications" directory
02 in the first level of the path is the "documents" directory
03 in the first level of the path is the "users" directory (equivalent to the stupid "home" designation used by troglodytes)
Note: in directories of type 03 in the first level of the path, the second directory is just the user's id number
Second level designations in a path:
01 in the second level of the path is always the "OpenOffice.org" application directory
02 in the second level of the path is always the "Mozilla" application director
01a in the second level of the path is always "Textual" documents
This is a much easier setup once you get used to it as it makes it VERY easy to find stuff. It also makes scripting possible for searching for files and working with files in a character based setting. I don't allow my users to use anything other than three character numeric file names. I haven't heard a word out of them since I implemented this system. In general, they seem to be pleased since they no longer have to think hard about where their docs are. The docs are always within easy reach, and as an operator, my life has become considerably easier. More time to play Nethack... So get it though your heads you fucking idiots!!! Long file names based on alpha characters are nothing but a big pain in the ass. Get used to the coming paradigm of numeric filenames. It's WAY easier. Fucking idiot amateurs.
(In a Daffy Duck voice): Woohoo! Whoohoo!! Wooohooo!!!! Whoohooo!!!!
Un-news
I'd like to have an object oriented database, like Lambda MOO, but accessible from a multitude of languages, instead of just one.
What's supposed to happen in your case-insensitive filesystem when one types a ß ?(german sz character) The capitalization of this character is SS (two characters)
. . . . . . .
may u!sh 2 sm!le at dz!z bad nn.!m!tat!ion
It's not about actually making good applications for computing, it's about a losing penis competition with Microsoft and Apple.
Microsoft is the big penis. It got big through buying lots of natural herbs, getting its dick sucked by lesser folk, and having an expensive personal trainer make them do cock pushups everyday.
Apple is the small penis, but it works hard to make itself look bigger by trimming the pubes around the base of its shaft. Commonly, the pubes are trimmed into artistic mutton chops, and you'll see cockrings and piercings.
Linux is the extremely tiny penis, desperately trying to extend itself with a very painful penis pump. It's also desperately downing lots of penis enlargment pills and furiously masturbating itself in any attempt whatsoever to compete with the other guys.
The other two penises accept their size and work with it. The Linux penis tries to be the other penises.
Be your own penis, Linux!
It sounds like all you really need is a linux port of huffYUV. That'll give you nearly 2:1 on standard NTSC (dirty) analog caps. On clean HD you might even go 3:1 or more. MythTV offers this as a codec, so most of the work may already be done.
We could add a nice feature to things like the graphical file manager so that when the user clicks on "properties" they can elect to "encrypt" . We can just grab the user's password and create a cert that is used to strongly encrypt the file.
We could use this technique without telling the user and at the same time implement it in such as way that only the GUI interface can deal with it. We would not want to touch the CLI interface because we would want to preserve backward compatibility for all those old apps that don't know about our nifty new encryption scheme.
Next, when the user loses their password and the systems adminstrator has to assign a new password we of course would not tell the user about all the files the system has lost access to. This would only create fear in the mind of the user's anyway and it is much better to say nothing.
We can add some silly permission bits as well that again only the GUI interface knows about. This way the user can lose access to his own files and never know about it. When we do this, we need to set the system up so it says you NEED to be an "adminstrator" in order to access the files, and then set it up so an adminstrator actually cannot access the file, but can only change some of the permissions. We should issure no message when the adminstrator TRIES to access the file and not tell him that the system didn't let him do anything, because if we issue a message then the administrator might get the idea something is wrong with the way we implemented this.
Another thing we can do is remove all error messages that the file system can issue and instead point the finger at the application and claim there is an "Unavoidable Application Error".
We can set it up so that if for instance the user powers down say an external disk drive while it is being accessed that no message is sent to the application other than "normal END OF FILE". This is pretty obvious anyways because if the drive get powered off clearly access to the file has ended, right?
Also, we should stream line the I/O so that when an application asks for data to be written to the disk, then EVERYTHING else in the computer stops until this has been completed. We can justify this because hardware is getting cheaper and faster and writing data is the single most important activity we can do when it is requested.
Also, benchmarks are typically run "stand alone" so as to get the highest numbers and if we program the machine so that there are no "background" tasks then the benchmarks will yeild higher marks.
RIGHT?
While we are at it, why don't we get rid of the concept of "users" and just make everyone root?
Barring the fact that WinFS isn't a filesystem replacement anyway, but a service running in the background, let's see you do some brute-force searching in 2006 on the new 200GB hard drives that will be out by then.
When Longhorn can do a search for e-mail contacts who like the same MP3 playlists you like, or all the Quake 4 pictures on your hard drive, and it can return the results in 2 seconds, let me know if you still feel that vanilla ext3 is something we can stick with for years to come.
I don't think EXT3 is that great... it is painfully slow when compared to something like ReiserFS. ReiserFS4 is the future.
Meh.
And novell's NSS filesystem even blows NTFS away. NSS even has X-pass transparent file shredding at the NOS level!
I see a small problem with these "new" filesystems. Microsoft is creating their filesystem with their metadata format. Apple is adding metadata to their system in their own format. And now somebody wants to add metadata to Linux filesystems. But in what format?
Is it just me, or a lot of supposedly smart people missing the very obvious problem that you're not going to be able to exchange files between these different systems and keep the metadata? Before anybody ships a metadata-based system, I'd like to see some kind of standard defined for metadata interchange. Frankly, I think a standard is long overdue (please let filename extensions die!), and it's absolutely essential that a standard exist before different formats become too firmly entrenched.
You would think that people would have learned their lesson, but this is starting to look like the 80s all over again.
I bet it will be released with Darwin. Why reinvent the wheel?
NTFS5 is miles ahead of most Linux filesystems (reparse points, encryption, compression,... all supported transparently), only ReiserFS 4 comes close, and it's not yet in a usable state.
I'm wondering if you even know what WinFS is, comparing it to file and locate it laughable at best.
Try finding all mp3 by Brian Adams or Withney Houston on a 200Gb disk filled with 250'000 files with file and locate, you'll get the answer 10 minutes later.
With WinFS, it will take you a whooping 2 seconds maximum.
That is without talking about the user-defined attributes, etc... that make WinFS more powerful than anything of that kind on Linux.
It seems like having gzip support would be more exciting though, since presumably one could combine it with a tar plugin, and that would allow you to cd into a gzipped file like a directory to access the uncompressed data, and then cd into the uncompressed data like a directory to get at the files themselves. Then there would be a totally transparent way to access tarballs that fits into the model that, from my understanding, Reiser4 follows.
Of course it's nice to have it implemented at fs level for speed and compatibility reasons
Don't think of it as a flame---it's more like an argument that does 3d6 fire damage
Most open source programmers can write a file system that will rival the millions of dollars that Microsoft and Apple are developing.
Sure they have day jobs, but as soon as they hear that someone named Mark Stone has thrown down the gauntlet, they will drop their Lord of the Rings DVDs and crank out a futuristic file system. Afterwards they can throw together a Mars rover from old SCA armour they made from coffee cans. They can paint Sailor moon characters on the side.
I look forward to it.
Reality collapses, of course, Right?
developing a local file system is nice, and i'm glad people are working on it. but what's much more interesting to me - and i think more generally useful - would be a better protocol for talking to said file systems. VFS is better than the more direct stuff that predates it, but it's still a nightmare. 9P, the file system protocol in Plan 9, is wonderful to work with. if we could get a better way to talk to file systems, doing work on file systems would become much, much easier.
the likely side effect of user-level file systems would be a huge win for the system as a whole. loopback NFS? ick.
9P patches have existed for BSD at various points. it's not hard, and it'd be a big win.
i speak for myself and those who like what i say.
Linux is a penis
i don't even want to imagine beowolf cluster of these...
I, for one, would like a filesystem with unrestricted metadata and easy access to it. Say you have a folder full of camera pictures, store a thumbnail, comment, date etc but in the filesystem where it doesn't get in the way of stupid little utils that barf on comment headers, something like Mac's resource forks. In essence each file would have a matching metadata chunk, handled by the kernel.
-Billco, Fnarg.com
It would be nice maybe if we could use OpenBeOS's file system... I have a feeling it might be under a strange licence. I.e, more strange than the GPL or LGPL :(
Giving IE users a taste of their own medicine since 2005 - http://pods.-is-a-geek.net/
Anyone find it interesting that "Cairo" as a homonym for the greek letters Chi (x) and Rho (p)?
I agree with the need for compression, but it would have to be something lossless and quick, at the expense of compression ratio.
I've always wanted to see the Reed-Solomon routines get integrated into the filesystem. RAID is great, an PAR2 has opened up a lot of possibilities, but I still think it would be a good option to provide in the filesystem itself.
Encryption would be another great option, but it would have to be a method that actually deserves my trust, not ike the ntfs encryption.
Hopefully Reiser4 and plugins will make all that a reality in the near future.
This is at least a 60 year old problem -- how do you file things?
Do you file your income taxe receipts under 'I' for income tax, 'R' for receipts, 'M' for money, 'F' for finances, 'T' for taxes, 'A' for accounts payable, 'G' for government, 'D' for debts, 'E' for expense, or 'U' for i give UP?
Some people are great at filing and organizing. Some are slobs. I've yet been able to find some system that can organize a 'creative mind'. I don't think that WinFS, even if it lives up to it's hype, can help us poor creative slobs. There are just too many ways to look at the same thing.
Do i remember Cairo? How could I forget it. It was the first time I'd ever laid eyes on you, and my thoughts have been consumed by that moment ever since.
It was hot - the sun in the sky, and the Egyptian counterintelligence on the ground. Very hot indeed. Dust filled the bazaar, crowded with people and the scent of spices. The cameras around my neck (for show, of course) seemed to grow heavier throughout the day, and the perspiration threatened to show through my white linen suit.
I stopped by a stall and pretended to peruse the knives for sale, but under the horn-rimmed sunglasses, my eyes scanned the bazaar. I was worried - you were supposed to be here an hour ago.
As I turned back into the crowd, I saw you. I will never forget that moment. Your face was mostly covered by the traditional dress of the region, but your eyes were glittering blue jewels that mesmerized me.
I stood unable to move for several seconds. Coming to my senses, I gave the pre-arranged signal: taking the hat from my head, I wiped my forehead and the back of my neck with a handkerchief, exposing the black widow spider design stitched into its corner.
Immediately, you relaxed, smiled with your eyes, turned, and walked into a dark, dusty doorway, and I followed after you.
And so I have followed you ever since. I tried to remain professional, but I was merely denying to myself what I already knew on that first day: I was doomed. It was a classic case of giving way to my heart, and in this business, that is the first step to giving away your life.
In the previous cell - the torture cell - there was nothing but blackness, and agony. Here, there is at least a crack in the ceiling. Light sometimes filters through the perpetual dust, and it falls across me during certain times of the day. In the dustbeams, I still see you. Even after everything, I see you every day, not knowing what thoughts hold court behind that beautiful, mysterious face.
The dysentery has left me quite weak, so I don't often move. There is no water here, and no food. But mostly it is the lack of water. That's what will get me in the end. I've been put here to die. In these final days, I don't think of the apple orchard in Indiana, or of my mother, or of the ice storm that trapped me in the mountains so many years ago. I think only of you, and the bazaar, and that moment I first met you. I think of... CAIRO.
pr0n - keeping monitor glass spotless since 1981.
Sounds like you organisation could really benefit from this feature. It also sounds like it would save you money. So why not use some of the potentical money savings, and offer to pay one of the Linux file system developers to develop it for you.
It sounds like you've benefited from the Linux development community, why not, by paying for further development of one of the existing file systems, recipricate.
The Internet's nature is peer to peer - 20050301_cs_profs.pdf
What are you talking about? NTFS has had journalling for over a decade. And Unicode. And ACLs. And streams. And reparse points (these are amazingly cool). And compression. And encryption. And ... you get the point.
Now, MS doesn't use most of this good stuff, but it's all in there. Even three-letter file extensions on Windows are obsolete, since everything on NTFS can be an OLE server. There's nothing on Linux that comes close to the capabilities of NTFS. About the only major thing NTFS is missing is versionning, which VMS has.
While it is true that NTFS is a journalling file system its implementation is *very feature poor*. NTFS can save you from MFT corruption which would be fatal to something like FAT but little more. NTFS' current journalling capabilites cannot provide enough data to roll back incremental changes (WinFS is supposed to have this) like Reiser and all other journalled FSes that Linux can use. NTFS' crappy journaling implementation is the main reason everyone on a Windows box STILL has to sit through those LONG volume scans if a disk isn't unmounted cleanly. In more robust FS like ReiserFS, Ext3, XFS, etc. the OS can just replay the log and redo any transactions that didn't go as planned. You cannot do this with NTFS as I understand it.
This feature alone makes your claim that "There's nothing on Linux that comes close to the capabilities of NTFS" ring hollow in my ears at least.
Also, if a feature is not implemented it might as well not exist because the user cannot derive any benefit from it. If I can't use a feature who on earth cares that it exists?
G. Washington on Government "it is force. Like fire, it is a dangerous servant and a fearful master."
>Probably because it was renamed to Windows NT 4.0 before released.
Most of it was actually renamed 'XP' before it was released, and people didn't even get the joke.
The Greek letter 'X' is a capital 'chi' (pronounced as in 'kite'), and the Greek letter 'P' is a capital 'rho' (as in 'row').
Cairo
chi rho
XP
You didn't really think they'd let all of it go to waste, did you?
HFS+ is journaled, and NTFS has sort-of journaling HFS+ and NTFS use exactly the same kind of journaling.
Before coming up with some sort of DBFS, some extra thoughts on security are needed.
So far the biggest reason to crack a system is to get access to its network resource, as a spam relay, a springboard/anonymizer, or a bigger pen1s. (bragging rights) Back to spam for a moment, about the only data harvested so far on those cracked systems is the address book. That's because the address book is the easiest to find - the most integrated.
Enter DBFS. Now ALL of your data becomes easy to find, *and mine*. I presume/hope there will be encryption of sensitive data, but there will always be weak passwords, weak encryption, etc. It's easy to imagine your own system cracking away all night trying to break into your own data, on behalf of someone else.
DBFS makes it easier. Granted it's currently possible to scan normal places for Quicken or MS Money, and see if any account numbers or such are there. But it's the *integration* that's the fear, because you can start doing *more* with DBFS and integrated applications. But those 'nice' features increase your vulnerability.
Not that we shouldn't do such a thing, but security needs to be a fundamental part of the architecture, not a bolt-on afterward.
The living have better things to do than to continue hating the dead.
I was able to encode realtime 640x480 NTSC video (using a 16 bit YUV source) on a 450MHz PII and that was WHILE I surfed the web. That was back when I needed to - with an XP1600 I can rip through PAL sized video (720x576) at 100fps or more - and that's in windows with a conversion filter or two in between the "in" and the "out."
Are you using RGB24? If you are using RGB you might try keeping everything RGB32 - you'll find it much faster, and that empty alpha channel won't take up much extra space :)
What about an open spec so that you can read and write it on non-Windows systems without having to reverse engineer? Can NTFS be used on a digital camera or Palm organizer? Not reliably.
I don't know enough about the implementation issues to know whether these belong in the FS or should remain in some layer above it, but it always seemed odd to me that "rm" in *nix is unrecoverable unless you custom build something that makes rm an alias for something that isn't rm.
Why not have rm mark the file as being in the special "deleted" directory and have all such files fully recoverable until the moment their space is needed. (If you don't WANT a file to be recoverable, use something like rm --scrub and the file system would intentionally make it unrecoverable.)
Likewise for file renaming. If you rename a batch of files and then realize that you screwed up (perhaps sorted alphabetically instead of numerically, so they're renamed in the wrong order), you just issue an undo command of some sort and the the damage is undone.
Yes, I know this sort of thing can be a feature of your GUI, but I'm wondering why it couldn't just be a standard feature at the command line so even on servers you would have the ability to undo things and wouldn't need that very unhelpful and primitive "Are you sure...?" confirmation hack for so many operations.
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
uhhhh... what?
if the filesystem does the compression, the apps (or you) can't see it happen. that's the POINT. your suggestion, above, is ridiculous. If you had a tar.gz file, you could extract it to the FS, but it would actually be equally compressed (cause it's a gzip compressed FS), and then you could play with the files to your heart's content, without worrying about the compression, cause it's transparent. You wouldn't need or want some kinda plugin or something...
Unless the FS wasn't compressed, and you wanted a transparent way to access tar.gz files. That idea would make sense.
What I read is that ntfs only journals the metadata, (or was it everything except the metadata), anyways that's exactly what I would expect from Microsoft: Technically we can call it a duck, but please don't expect it to really quack and waggle.
Besides that there is Linux support for user-controlled encryption of loop-mounted filesystems: do you really trust the Microsoft encryption to do anything else than making your systems slower?
"Sixth, NTFS supports key recovery with a backup, emergency passphrase (it can maintain two copies of the encryption key, one encrypted with, say, the administrator's password)."
You just found the point where they will breach your data security.
IIRC it should be something like that (foo should be a directory containing your stuff):
When you want to use the compressed file, use something likeYou can also use automount, by adding a line like this to your automount config file (on my machine it isTo avoid running out of loop devices that these filesystems use, you may want to add
to yourOf course, huffyuv etc. should be more suitable in your case, as other posters have said, if your software can support it.
I am working in multimedia production, including video. For lossless video compression (yuy, yuv or rgb) you can use HUFFYUV. The original site have vanished but some goggling will lend you to this.
I used it for lossless outputs, lossless hi-res projections etc... stable and all, it think it's free software.
Tomorrow is another day...
If you're going to hassle with "user defined attributes", you may as well store that information in an actual RDBMS and do it *right*.
But then I don't know that the Windows "community" could accomplish something like that without waiting for Microsoft to do it for them.
Microsoft has been *trying* to do it for 10 years, and they have already made major compromises in their promises about it in the next Windows release. Does the word "Cairo" ring a bell?
Have fun waiting.
While you do that, we'll be building a more open, more robust, more Free, and better system on our own.
And seriously... Brian Adams? Whitney Houston? Do they play those artists to get the crowd fired up at MS Developer conferences or something?
"innovative" native database/filesystem (copying an idea from IBM's hugely successful OS/400) Perhaps we need to clarify to the Unixesque/mainframe world the difference betweem an idea, and how well that idea is implemented. OS400 was an idea. The implementation, on the other hand wasnt so great.
Hi
Just a thought:
Why is it not possible for a program to do IO on files without using the OS's cache?
Tasks like writing a CD or taking a backup usually destroys the cache and swap - after the task is complete, applications are swapped and the cache is full of a file that's usually just deleted.
What if you could e.g. specify a "d" in the fopen parameter, like fopen(filename,"rd") and that read (or write) would not use the cache?
I think it would be greate.A programmer often knows more about data than the OS.
But if I search for "Brian Adams" will I also get songs by Bryan Adams? http://www.bryanadams.com/
As far as the metadata side of the linux.com article is concerned,
many of the things talked about have been implemented and working
nicely in libferris for
quite a while now
Most filesystems these days support EA (ext3, xfs, Reiser etc) the
trick is having the system do something interesting for you with that
EA capability automatically. This is another area where libferris
helps, making EA that is trapped in files available aswell and
providing indexing over your EA. (A catch-22 is that EA for files is
supported in many kernel level filesystems but can be very slow
relative to even file access, you have to index EA in some way in
order to search for your files based on queries over your metadata).
Supporting ACID at the libferris level is a pending challenge.
While it is true that NTFS is a journalling file system its implementation is *very feature poor*. NTFS can save you from MFT corruption which would be fatal to something like FAT but little more. NTFS' current journalling capabilites cannot provide enough data to roll back incremental changes (WinFS is supposed to have this) like Reiser and all other journalled FSes that Linux can use. NTFS' crappy journaling implementation is the main reason everyone on a Windows box STILL has to sit through those LONG volume scans if a disk isn't unmounted cleanly. In more robust FS like ReiserFS, Ext3, XFS, etc. the OS can just replay the log and redo any transactions that didn't go as planned. You cannot do this with NTFS as I understand it.
I don't even have to respond. You know nothing about NTFS and even journalling in nature. Go pretend your smart somewhere people don't know how dumb you sound.
What about an open spec so that you can read and write it on non-Windows systems without having to reverse engineer? Can NTFS be used on a digital camera or Palm organizer? Not reliably.
Technically the FAT and other file systems used on these devices are not open either. They are reverse engineered, and technically illegal.
NTFS could be reverse engineered for these devices, but Win98,WinME users would not have access to the content, so providers have not used NTFS for this.
Just about everything in Linux which looks at an ext2 filesystem relies upon case matching exactly.
Contribute to civilization: ari.aynrand.org/donate
You just found the point where they will breach your data security.
It's still something that a lot of organizations will want to have before they deploy something. The IT guy is nearly always a security hole anyway -- he's setting up the computer and saying "you can trust this". You might as well trust him with recovery duties as well.
May we never see th
do you really trust the Microsoft encryption to do anything else than making your systems slower?
I'm not sure that I'd want to directly trust either the Windows or the Linux code with my life WRT the presence of zero flaws.
On the other hand, I'd rely on both of them to be pretty solid, and be comfortable using either for data encryption of sensitive, classified, or incriminating data.
May we never see th
I really thought when Veritas was making a push for vxfs, that was going to be the real dominent commerical filesystem with a real industrial volume manager. I guess they couldn't make this free to any extent, so it went no where.
I assume you're finding those particular MP3s so you can delete them. Badoom tish.
checksums are good. let's have self repairing filesystems. end bitrot forever.
Certain software excercises certain code that other similar code may not.
For instance, distcc seems beat on some kernel code more than just make -j 5 and bring out some potential kernel bugs.
So, it may not be reiserfs/other_software = bad, kernel gods = good
It might just be that certain combinations of software bring out the best/worst in each other.
automatically map bad blocks to good blocks, but you can monitor the current number of bad blocks through SMART. If your drive is causing errors you had better stop using it (production environment or no) as quickly as possible. Bad blocks are almost always a sign of impending failure.
Jebus, do you want to run a production system on a drive with known bad blocks? Whoever hired you must be a complete moron.
HAND.
I wrote a longer reply which got lost (bloody crashy browser), but essentially is boils down to: The semantics are unclear without expanding the traditional UNIX system calls to include more semantic details about the context of the various operations (including allowing for real transactions). Adding this would make this unreasonably slow, because most of the time it's not actually needed (only user operations on files need this support, and it's completely wasted on anything else).
For user files you can also approximate this by just using tla on your home directory (with appropriate embedded ID tags to prevent mv's from screwing up the file "tracking").
HAND.
Set up subversion via apache, then use the path as a WebDAV store. Then all writes to the files should be versioned [I think] - that's what I've understood from reading the documentation, though I haven't tried setting it up myself.
The ability to undo after the fact would also complicate free space allocation (which is already quite nontrivial in file systems like Reiser4) because you'd have to find an (arbitrary) balance between: "Do we remove this file permanently" and "do we allocate some other free (but clearly suboptimal) blocks?". I.e.: There are lots of policy decisions which are usually best left to userspace.
HAND.
Too late. Someone already did, and they even gave it a name: bukkake.
Hear recorded Slashdot headlines on your phone! New service beta testing. Just call (248) 434-5508
Add Plan9 and 9P to Linux. This allows interpretation of most filenames to be done by user level programs and by remote services. This gives you encryption and compression and lots of other stuff that some people think are "filesystem" things. This also gives you the ability to put the entire "database" in user space, ie "/search/foo=bar" would be a directory listing all the files in the system that match the "foo=bar" criteria, dynamically created.
Add Plan9 union mounts to Linux.
Move all of the "ioslaves" that are in KDE/Gnome and move them to using 9P. The program "cat" should be able to open any string you can type into the fanciest KDE file browser and copy the contents. Instead of colon use slash. See 9P for details.
Hit all the idiots with a clue by 4 until they figure out that a "filename" is a STREAM OF BYTES. It is not "text". This means no "canocolization" or "convert to wide characters" or "make sure it is legal UTF-8" or any of the other crap that we have been putting up with for years and years that has killed all progress in this field. Microsoft is of course the worst offender here, but new Unix designs are not so hot either, note that with locales I can easily create a file that I cannot display in KDE, that is wrong.
"Metadata" should be required to be a cache of information figured out from the file. Do you want the postagestamp image? Look for the metadata, but if it is not there, run a program that will create it. Although physically not prevented, metadata that cannot be recreated from file data should be considerd a bad idea because it will be lost if the file is moved. This gets rid of all need to transmit metadata. Also it allows the metadata to be stored on a different system than the file, which is very useful if the file is remote or you don't have permission to write it.
Using ReiserFS or EXT3 or NTFS WinFS is all trivial issues compared to this stuff.
I'd go for / and just look through the 10 photos that are stored there (possibly named after the people in the photos, but I'm lazy so I usually just leave them with names like IMGXXXX.jpg), but I digress.
My question (and I believe, the original poster's) is this: If people cannot be relied upon to even provide a file name that's meaningful, what makes you think that they'll be inclined to even provide a list of the people in photos, locations, etc.? Of course, one could just be obnoxious(sp?) and require that the user provide at least 3 pieces of metadata to identify a file. Not sure how well that would work, though.
My current naming scheme is quite simple: Make sure that the path to your data contains all the metadata that you will want to search for. Then you can trivially use "locate" to find what you're looking for. Free text search in path names is actually quite powerful once you start naming files reasonably and putting the files into directory names which contain all the metadata necessary to retrieve them.
(I would love system-wide virtual folders (backed by metadata) though,
The only place I see people relatively consistently using metadata is music, but that's only because they actually don't have to: Somebody's already done it for them and put the results on FreeDB or Musicbrainz...
In short: People are generally too lazy to make this work reasonably for most people. That's not to say I won't be using it. In fact, I'm quite looking forward to it.
HAND.
"Server crash blitzes Florida's e-voting records"o tes/
http://www.theregister.co.uk/2004/07/28/florida_v
one wonders why a server crash allows for a possibility to loose its data. Apparenlty it was e-voting records which were lost. Also very very strange. Why?
From the old days, using norton disk doctor (msdos based) and nowaways using the crash recovery kit (linux based), recovery was plain simple: use the basic rules for filesystems on disk sectors, and to loose all your data became an impossibility, except when having real physical disk damage.
With the move from a filesystem to a database driven file and document storage a couple of things happen. 1st Knowing the basic rules of filesystems and disc sectors will become a useless skill. 2nd We come dependant to database recovery skills, which a market study shows are way more expensive per hour (CALL e.g. 0800-ORACLE) as filesystem/disk recovery services.
This will have as major impact. If someone's Longhorn crashes and cannot recover itself anymore, your neighbour just might loose all its data. Why is it, I suddenly smell a dirty rat here? Do Ballmer & Co want to reshape the grand public perception that if your Windows dies, then also your data dies with it? Did he maybe get a clue from the RIAA/MPAA how to proceed further? Like how can we nail those kiddo's with all them 120Gig and bigger disks full mp3 and mpeg copyrighted multimedia content? Only the future will tell. Sofar i'm happy with running Linux on my desktop and laptop.
I guess the perfect perception goal for Ballmer might be this future quote, meaning someone lost all his windows data : "Has your PC been Longhorned?"
Now why linux.com/Novell also want Linux users to believe this, and advocate for the fast introduction of a linux database driven journalled filesystem, i don't know.
Robert
Apple isn't bringing a new file system with Tiger. It is just doing searching right. Folks, this can be done on current file systems.
I read your post and the responses and I can't agree with you. I tested it out on a file server I have:
Tyan mobo, dual 333MHz PIII, 512M of RAM and 4 x 160G WD JB drives connected to a 3Ware 6400 in a RAID 5 configuration.
The total file system space is 480GB and it is about half full (239GB). There are about 400,000 files on the system.
I did various searches using 'locate' and 'grep'. On average the searches took 0.6 seconds... which is three times faster than WinFS. Some searches took as long as 1.8 seconds when combined with mutiple filters but that was the max. Shortest was 0.43 seconds. I didn't only search for MP3s (which are spread out all over the place) but also RPMs, tarballs, text files, video files, source files... you name it. Same results.
Now it is obvious you don't understand what 'locate' is or does. You may even know something about WinFS. You claimed somewhere that you work for MS and have seen WinFS in action. I know nothing about WinFS. I will just have to take your word for WinFS being as much as three times slower than Linux/ext3 filesystem when doing searches for filenames (shrug). This is slashdot and there are all kinds of under-informed claims made here. You might want to be a bit more careful with your employer though... lest they be embarassed when their new WinFS doesn't actually stack up.
-DU-...etc...
"Don't sweat the technique."
Reiser4 has a compression plugin coming.
What about encryption plug-in? That would be nice for those easy-to-lose USB keys.
Phillip.
Property for sale in Nice, France
Try finding all mp3 by Brian Adams or Withney Houston on a 200Gb disk filled with 250'000 files with file and locate, you'll get the answer 10 minutes later.
.mp3 | mp3info -p "%a" | grep "Brian Adams". Personally I think on any decent spec machine it should only take a few seconds.
Would it? Can someone try on their 200GB collection and post the results of: locate
Phillip.
Property for sale in Nice, France
Did you use recent zlib library source? It's become significantly faster...
I don't know what the winds of change are saying, but the good people at Debian-legal say GPL-incompatible. Here is a controversial part of a text that the people at Debian interpreted as the licence (Read the entire license here)
Finally, nothing in this license shall be interpreted to allow you to fail to fairly credit me, or to remove my credits such as by creating a front end that hides my credits from the user or renaming mkreiser4 to mkyourcompanyfs or even just make_filesystem, without my permission, unless you are an end user not redistributing to others. If you have doubts about how to properly do that, or about what is fair, ask. (Last I spoke with him Richard was contemplating how best to address the fair crediting issue in the next GPL version.)
Is there a fundamental difference between XFree86 and reiser4?
Laugh a little. :) It wasn't a troll, sheesh.
Actually, the plugin does encryption, authentication and compression at once (only the ones that you want, of course).
What's supposed to happen in your case-insensitive filesystem when one types a ß ?(german sz character) The capitalization of this character is SS (two characters)
So? It gets stored as a ß. If a user types 'cat ß...' then it matches that file. If a user types 'cat SS...' then it matches that file. Case-preserving-yet-insensitive filesystems are nothing new; NTFS has done this for years.
Interesting... Could this feature be done as a plugin for Reiser4?
This whole thread would be slightly more useful if you actually said what else WinFS is useful for.
Particularly for you, actually. What does it give *you* that other filesystems don't?
NTFS has had journalling for over a decade. And Unicode. And ACLs. And streams. And reparse points (these are amazingly cool). And compression. And encryption. And ... you get the point.
Now, MS doesn't use most of this good stuff, but it's all in there.
Er, sorry? MS most certainly does use the journalling, unicode, ACLs, streams, compression, and encryption.
The only feature you mention which isn't exposed through the GUI is reparse points, which are trivially created with third-party utilities, and which I use every day.
So which is the "most of this good stuff" you're saying they don't use, given that they use all but one of the features you mention?