Slashdot Mirror
Merits Of The Different Journaling Filesystems?
a2800276 asks: "The story that XFS has gone beta raised some questions in my mind. There are now four journaling filesystems available under various OSS licenses and being actively developed for Linux, there being (in estimated order of maturity): SuSE/Namesys's reiserfs, SGI's XFS, IBM's JFS and Tweedie/Redhat's ext3fs. Avoiding the obvious question of why can't the effort going into four different projects be channeled into one, I think a discussion of the particular merits of the different fs's would be interesting."
Microsoft would already be doing that.
Is this a good thing.
I mean one should always assume the worst case scenario and deal with it in the code. If you can't deal with it, you should not allow the user to use the feature at all. That is, you should not allow mounting ext3 as ext2 if you can't guarantee that it will be unmounted cleanly.
People say that Linux is inferior because it needs long fscks when it's shut down. What they don't realise is that neither NT nor 9x / me actually has a Journal File System. Microsoft's filesystems are more like silent data-loss. Please do not confuse a real industrial-grade JFS with Microsoft's inferior filesystem. And when you see that fsck, be proud - your Linux has a better filesystem.
NTFS *IS* a journalling FS
This may be a corrupt sector containing metadata (maybe even for the "/" directory or "/kernel", if you were writing a new kernel at the time of the crash), or it may be other corrupt data which became corrupted in a cascade failure that resulted in the crash after one or more corrupted blocks were written to disk.
Soft updates simply can't recover from this.
If, on the other hand, it were a kernel panic that didn't result in corrupt data being written to disk, then there's no danger of a corrupt sector from a DC failure, and there is no danger of other corrupt data needing fsck'ing, so you would be in the situation where the only thing that would be out of date is the cylinder group bitmaps; you could clean this in the background by "locking" access on a cylinder group by cylinder group basis for a short period of time, to clear bits in the bitmap that said an unallocated sector was allocated. This might be seen as brief stalls by an especially observant user or program (say someone is doing profiling of code at the time), but could be accomplished in the background.
The problem is that you can not know the reason for the crash, until after the recovery.
If there were available CMOS, you could write a "power failure" value into it at boot time, and then write a "safe panic" or an "unsafe panic" code into it at crash time (a power failure would leave the "power failure" code there).
The only valid background case would be for a "safe panic", if you could really guarantee such a thing.
The worst possible failure resulting in a reboot is a hardware failure of the disk; I would really be loathe to try cleaning in the background after a track failure or even a sector failure (sector failures are identical to sector format corruption after a DC failure during a write, FWIW).
Look, soft updates are a good thing, but they aren't a panacea for all problems. Let's laud them for what they do right, but not misrepresent them as doing something they can't.
This story could not have come at a better time, here is my experience with ReiserFS, that ended yesterday.
/home, so I nuked it and partitioned/mkreiserfs on that. I only have one user account, and 2 explicit samba shares on that filesystem.
/home, Not a valid block device it said. Uh oh. I tried to run reiserfsck on it, since it wasn't mounted this was ok. No dice.
/proc/filesystems).
I downloaded the kernel patch for 2.2.17 (v3.5.26), patched compiled, and built the utilities. I have a 8gb hard disk that I use for
The next day I built a new kernel without ipmasq support (I no longer needed it) and also removed ppp. Upon reboot (a proper reboot, shutdown -r now), I noticed my ftpd had freaked out, and that only / was mounted. Uh oh. I tried to mount
I fired up fdisk. There was no partition found on that disk. Gone. I have no idea what happened, my kernel obviously still had reiserfs support (in
I realize that ReiserFS is still BETA but DAMN ! So I fdisk'd, and mke2fs and now it works again, I can reboot without losing my partitions.
I'll come back when Linus and Alan allow it into the standard kernel tree, until then, I'll be fscking with ext2.
The average linux desktop requires X11 _by definition_. You can't strip it from it. In addition to that, if you need to have a nice blended environment where everything (including apps) fits into everything else, your choices basically reduce to GNOME or KDE. Sure, you can run twm only, and run a "light" system, but you need all the other stuff. Toolbars, Pagers, Drag and drop, ...
From what I've seen of the BeOS, it's WAY more efficient at doing this. And the reason is "architecture". BeOS is a complete re-engineering of a lot of desktop concepts, whereas linux follows the work started with X11 to the extreme. Personally, I don't hold that much of a grudge against X11, I think it's pretty cool for what it does and when it was written, but it IS bloated. Luckily, they're working right now on reducing X11's size for embedded systems, so we may one day (hopefully soon) see a slim and trim X11. But X11's biggest problem to me will always be backwards compatibility. It carries so much stuff with it that's useless nowadays, and because of it no needed stuff gets added. Like usefull universal drag and drop, anti-aliased text (another two things BeOS and Windows have, and linux doesn't), and so on.
X fits the needs of today pretty well, regarding it's age. But everyone can see X was designed a long time ago.
LOL, bitter BeOS zealot!
Actually, Linux has much potential to become a mainstream OS. I bet if you had a line graph for each of the main OSes out there (MacOS, Windows, Linux, BeOS, etc) you would find that Linux is the OS that is gaining much more support than all the mentioned OSes. And the cool thing about Linux (or the part you hate) is that it is the public's OS, which gives people incentive to make it work. You see, Linux will be a success because it's everyone's baby! You're missing the point about Linux, and free software in general.
Perhaps you used RedHat 4.0 with twm a few years ago and wrote Linux off, but I can assure you it has advanced by leaps and bounds. It has advanced enough for me to put my users on it, and replace servers running Linux/x86.
In the time it took you to write that anti-Linux post, 8 new updates on Freshmeat were annouced, Miguel had another crazy Helix idea, KDE had another bug worked out, and another IBM released the source for one of their commercial products in the name of free software.
I'm using it on 2 Production Webservers with hardware-mirrored 9GB-disks where 5GB are used with 1.500.000 files on the filesystems. I don't even wnt to try the time e2fsck would need after a crash.
Both systems have an uptime of 180 days now (reboot was done because of systems moving).
I like ReiserFS.
No.
ext3 is a journalling filesystem based on ext2, with (almost) the same format on disk and which is backwards-compatible.
tux2 is a phase-tree filesystem based on ext2, which has a different method of keeping the disk as sane as possible, and which isn't trivially backwards compatible.
They share more code between them than either do with the other filesystems, because they're both based on ext2. They're still different, though.
To be fair, there's a development path here. minix was a base filesystem, from which xiafs and ext appeared. ext2 was developed to address the shortcomings of ext, and was developed after the other two and taking advantage of what they'd discovered. So it was never a straight choice between four filesystems - they didn't start off on an equal footing.
With what we're talking of here - reiserfs, ext3, xfs and jfs - it would seem that only reiserfs is taking advantage of past experience and showing innovation. xfs and jfs are ports, and ext3 is intended as a simple upgrade path using time-tested code. That said, some of the innovation in reiserfs would seem to be directed towards odd and specific things - indexing of files, directories of objects and other strangeness - which most people don't want out of their Unix filesystem, and is as likely to find a niche as become a winner in the fight to be the single general-purpose filesystem.
We accually use both reiserfs and ext3 in production on sourceforge. Both work equally well. It's all a matter of what you want.. when migrating servers that have been up and running for month I use ext3 because it's a simple remount, but when building new servers that have a need for speed and/or lots of small files (ie CVS) I use reiserfs..
- U
Mozilla.
What is with Mozilla? It takes the new honors over Emacs/XEmacs as the most bloated free software project ever. It's only going to be through efforts like Galeon and others that Mozilla will be redeemed. Many people pined for simply Web browsing from Netscape; the community responed by adding IRC and the ability to build just about any kind of app you could ever hope to build on it.
Stating on Slashdot that I like cheese since 1997.
...should be flogged.
:^) Oh wait; if you're working on Slash code, you can't stick to keeping the stories at the level you expect. Ah well; looks like Real Life(TM) kicked you in the ass again. :^)
Whine, whine, whine. If it bothers you to see this kind of thing more than once, consider getting A Real Life(TM). It pains me to no end that so many people have to bitch and moan about the quality of stories, the supposed repetition of stories, and so on. If you think you can do a better job of putting together a site that interests you, do so.
Heck, why not try to get a job at Andover. I hear Slashdot is trying to work through some serious holes in the Slash code.
Stating on Slashdot that I like cheese since 1997.
I've personally only had minor problems; I've never put a UPS on my Linux box because I've never had major problems. :^) I suppose it depends on how hardcore you use your machine.
Stating on Slashdot that I like cheese since 1997.
Hrm...I've shut down BeOS the proper way, and lost data. How strange.
Stating on Slashdot that I like cheese since 1997.
--
--
RiserFS is actualy FASTER than EXT2 on My Pentium 200 64 meg box under Mandrake 7.1
--= Isn't it surprising how badly I spell ?
yes. I am.
I put that disclaimer in my sig to force people who disagree with me to attack the substance of my coment rather than spelling.
--= Isn't it surprising how badly I spell ?
Don't sweat it. This place is so impersonal these days that it couldn't have been rejected just because you submitted it.
(0)diverge@/net/tmp242% rpm -q -a | grep e2 /home/ben/bigfile
e2fsprogs-1.18-5
(0)diverge@/net/tmp242% rpm -q glibc
glibc-2.1.3-15
(0)diverge@/net/tmp242% uname -a
Linux diverge 2.4.0-test5 #14 Wed Sep 27 17:05:35 EDT 2000 i686 unknown
(0)diverge@/net/tmp242% ll ~/bigfile
-rw-r--r-- 1 ben ben 3480223744 Oct 2 14:10
...and it was still quite happily going. I don't know about seeking, though. gzip said the file was too large, and emacs said "file not readable". Tar works beeeautifully, though.
"The biggest problem with communication is the illusion that it has taken place."
That's xfs. You wouldn't say 'LS' was the command to list files would you?
Baz
It is stuck at a 32 Meg limit. If you check the BeOS developer websites you will see that there is no workaround short of fixing the flaw in the BeOS kernel. Sadly, this has been ignored by Be Inc. They are completely unwilling to fix this flaw.
;)
Well, how many handhelds/Internet appliances need more than that?
You see, the problem with closed-source software is that when product planning directions change, _your_ feature may be dropped or your needs may be ignored.. If BeOS was OSS, this would have been fixed by now...
(The irony is, some ex-cow-orker would always feed us this line about how commercial software had a requirement to satisfy its customers, while OSS was planned on the 'whim' of its developers, without customer regard.. Sucka! He called himself a Solaris sysadmin, but the fucker never installed tcpd and left vanilla inetd.confs wherever I cleaned up after him.. Shit, he never even rem'd out a single init script.. At least he was too incompetent to leave a backdoor
Your Working Boy,
If evolution is at work on software, why do we have KDE, Gnome, and now Xfce?
Because evolution is slow. Why do you think? Even then, natural selection only works when some choice has a particular advantage over any others in the environment in which the majority of interaction is performed. And, even then, the losers may still hang around in specialised niches, becoming more and more specialised over time.
In any case, diversity is good, from an economic sense and in terms of micro-evolutionary development. You can't figure out the best approach to something without trying all of them out, can you? Give it time.
Canthros
It's been up 18 days now, which is also impressing the techies in our M$ shop
'nuff said!
Just junk food for thought...
If you've got all those hard drives, why not invest a little more cash, go on Ebay and get yourself a cheap RAID controller? 'Cause I'm cheap and those hard drives broke the budget :)
Better yet, why not use RAID-5?
Because unless I'm mistaken (great opportunity for ReiserFS hackers to jump in here) ReiserFS and RAID-5 do not get along at all.
Read my stuff.
I didn't notice any increase in speed of file accessing Want to see a real performance difference? Pile something on the order of 100,000 files (size is insignificant) in a single directory on both a ReiserFS filesystem and an ext2fs filesystem. Do a rm -Rf /path/to/test/directory. If you're doing this to the ext2fs directory, you'd best go get lunch, have a beer, go home for the night, take the rest of the week off, etc.
On the ReiserFS filesystem, you barely have time to pick your nose :)
One of the most dramatic performance improvements in ReiserFS over ext2fs is how fast it can manipulate huge numbers of files. It also handles gigantic files much more quickly than ext2fs can. There are other improvements too, such as quicker access time to a file (slightly-to-moderate in most cases) and SIGNIFICANTLY faster recovery time when the system goes down ungracefully).
Read my stuff.
I'm pretty sure he meant an in-place filesystem conversion utility. You can convert an ext2 filesystem into an ext3 filesystem in-place. The same cannot be said for ext2 to any of the other filesystems.
--Joe--
Program Intellivision!
The only FS that uses an append-only log structure for both data and metadata is Margo Setzer's defunct LFS developed under 4.3 BSD. [Xj]fs are journalling, not logging, filesystems. The "j" kind of gives it away.
You mean |= ' ';
' ' = 0x20 = 32 = 0100000.
Course, I'm just nit picking.
Chris
Surfing the net and other cliches...
Surfing the net and other cliches...
(Who Meta-Meta-Moderates the Meta-Moderators?)
ok, you fucking goober...HERE'S the harsh truth.
i'd say probably between 90-95% of those who run linux cannot afford the hardware necessary to run irix or aix, especially not with massive i/o on a couple hundred gb of disk, so why should a community that is centered around an os that has historically been built for low-mid end hardware be discussing crap that needs a $15k+ server to run on?
had you just ended your rant at "All 4 choices are beta of varying degrees; they are not ready for prime time" i'd have lauded you as a wise man, for you do have a point...and yes, the rest of your rant has a good point too. for ip-based web servers with 15k+ sites on them, we run solaris...not linux...mostly because it wouldn't handle that nearly as well. but the problem is that the tone, and your obvious dislike of the linux community, turned what could have been a halfway decent exposition of the relative merits of beta software vs. stable software into a mindless rant that made you look just as dumb as those "uninformed propaganda" slingers you rail against.
next time, check your attitude at the door or go lurk on bofh.*
fwiw...people who run beta software get what they deserve (buggy software and frequent patching), it's just a shame that jerkoffs like you usually don't get what they deserve.
-dk
-dk
Dream with the feathers of angels stuffed beneath your head.
Not here it doesn't. I'm running 2.4.0-test9-pre7 with the reiserfs patches, and I've just happily built a six gig tar file on a reiserfs partition.
"I want to use software that doesn't suck." - ESR
"All software that isn't free sucks." - RMS
Oh gosh... you mean that be-fan is going to take time out to regale us with the merits of his favorite OS? Geez, that would be amazing. I mean, be-fan sure doesn't do this in EVERY OTHER POST HE MAKES. Seriously, come on, all you do is rank on Linux for it's inadequacies and talk about how wonderful BeOS is. Well guess what - a niche operating system where every bloody useful tool is expensive, isn't going to last very long. BeOS is powerful and pretty, but it is lightyears behind anything - including Linux - in terms of the basic requisite application support.
Can't fill a niche in the BeOS software line? Then go to slashdot.com and whine about every other operating system until people use BeOS out of pity or just to shut you up!
Know ye not that ye are Gods???
Unfortunately BeOS has a flawed kernel that prevents real world industrial strength apps from being ported to it.
Thats irrelevant to his post or this story. He was talking about the BFS, not BeOS in general.
Oh for heavens sake ! :-(
How is this informative ?
It has been pointed out on more than one occasion that the OS/2 version of JFS is actually newer than the AIX version. You make it sound as if it were somehow inferior.
If you go and actually grab the JFS code you will notice that case sensitivity was on the todo list, and is fixed as of something like drop 13.
Elsewhere, somebody posted a link to an article from the Linux Gazette that gives a carefully reasearched comparison. Your characterization of JFS implies you know little about it
Tim
What I wonder is, is there any filesystem that makes it easier for filemanagers to track changes in the filesystem? For example, to show when a new file is created, without polling all directories all teh time. Or that can save metadata such what icon to use, or where the user last dropped the icon/file (locationwise on the screen). I think Nautilius needs to have such information, but as no filesystem supports it, they store it in an XML file somewhere.
Will 2.4 ship with one of these?
No.
Do any or all of them break the 2GB filesize limit?
2.4 does this as long as you have a glibc with support for LFS. 2.1 can do this with patches - 2.2 will have it as standard.
Certainly for ReiserFS or EXT3, but I'm not sure about the others.
DevFS is maintained by Richard Gooch, and is most certainly not part of ReiserFS. Its website is here.
I put that disclaimer in my sig to force people who disagree with me to attack the substance of my coment rather than spelling.
This is not intended as a flame, but bad spelling detracts from what you are trying to say. One thing I have never understood is why some computer people are not able speak or write a human language correctly yet they are able to code in a computer language flawlessly.
Most computer people would not write C or Perl code rife with grammar/spelling errors because the code would not compile/interpet correctly. Maybe you should think of a spelling/grammar checker as a debugger for your English source code...
--
You think being a MIB is all voodoo mind control? You should see the paperwork!
A man who wants nothing is invincible
I think he accidentally typed a 'c' instead of an 'h'.
The one problem that I've had is that I upgraded my primary machine to be SMP, and I'm running now two PIII-933mhz processors. Sure enough, as soon as I did this and did a stress test (our application is completely unforgiving on the file system, with enormous amounts of multithreaded O_SYNC read/write calls and network access), I exposed the "IPI synch" bug on 2.2.15.
My only problem with reiserFS is that it exposes this bug, which means that I pretty much have to upgrade my OS by patching it or moving to 2.4.0, which is easier said than done once you decide to make the silly decision to install mandrake in "high security" mode, which gives you the silly mdksecure kernel which is virtually impossible to reinstall cleanly over.
But ReiserFS rocks!
Either they're easily impressed or profoundly incompetent if they can't keep an NT box going more than 18 days. A couple of servers (here and here) at work have been up a maximum of about seven weeks since we began tracking them with uptimes.net, and they had some longer uptimes before that. (I don't think they ever went as long as the 187 days (and counting) that our mail server, which runs SuSE Linux 6.3, has been up, though...I'm not claiming that NT is an über-stable OS on the same level as Linux, but decent admins should be able to get more than 18 days out of it. Hell, I've gotten more uptime out of Win9x than that!)
_/_
/ v \
(IIGS( Scott Alfter (remove Voyager's hull # to send mail)
\_^_/
20 January 2017: the End of an Error.
Probably ReiserFS. I and several of my acquaintance have used it as primary FS in some pretty ugly environments (think laptop with loose battery). I didn't notice any performance hit, but the slowest machine I ran it on was a PII-366. I haven't converted any of my slower home machines yet.
2.4 probably won't "ship" with anything until something is out of beta -- but just because the kernel team hasn't approved it is no reason not to use it. Mandrake 7.1 ships with ReiserFS and you can use it now.
"Nothing was broken, and it's been fixed." -- Jon Carroll
You mention the fact that ext2 was the technically superior product. But, I believe in this day and age where people want to get in on this 'Linux craze' you will see at least 2 winners. The technically superior and the easiest to use. Not everybody will be interested in the fact that thier filesystem journals data and metadata, some people will like the fact that, if they push this shiny little yellow button, that my computer will be a little bit better.
I recently tried out reiserfs on a large partition, and ran some benchmarks (Mandrake 7.1, 2.2.16). It was consistently slower than the ext2 partition on the same hard drive running Bonnie. They were about equal running Dbench. What I found disappointing, though, is that reiserfs still has a maximum file size of 2GB. So I guess the only selling points for reiserfs at present are the better used of disk space and journalling. They clain that small files can read and write faster, but I have not seen that yet.
For files > 2GB on IA32, we have to wait for XFS or JFS. Too bad!
I played with reiserfs some recently, because ext2's performance on directories with lots of small files is a real problem where I work. Reiserfs is lots faster, but there's one show-stopper that means we won't be using it. Reiserfs doesn't support quotas, which limits its utility quite a bit.
--
The scalloped tatters of the King in Yellow must cover
Yhtill forever. (R. W. Chambers, the King in Yellow)
The scalloped tatters of the King in Yellow must cover
Yhtill forever. (R. W. Chambers, the King in Yellow
I didn't notice any increase in speed of file accessing, but it was fast at rebooting.
From my personal experience (haven't had a look at the internals of ReiserFS yet) ReiserFS is faster than ext2fs when accessing files and especially directories. We have a couple of servers running ReiserFS with large directories (~ 150,000 files per directory). It takes a couple of minutes to list such a directory with ext2 but only seconds on a ReiserFS partition.
Does anybody know the status of quotas in relation to all of these filesystems? I know that Reiser doesn't support them, but I'm wondering what future plans are and what the status is on the other projects.
A musician without the RIAA, is like a fish without a bicycle.
What about data that is cached? Does Linux immediately write data to disk or cache it? I know also from my 3 years with Linux (RH, Caldera, Mandrake and now Debian) that if I don't unmount my floppy before I eject it, I don't get what I want on that disk.
So if this is the same for fixed disks, I'd be careful with that reset button. You might have egg on your face along with egg on Tux face in the eyes of the ignorant.
War crimes, torture, lies, illegal spying... Would someone give Bush a blowjob, already, so he can be impeached?
OK, later on you mention that it's built on top of an exisitng fs, but you seem to neglect that here. Remember that ext3 is built on existing, proven filesystem technology - it's the actual journalling that's beta, not the filesystem itself.
-Matthead
-Matthead
Will 2.4 ship with a JFS?
Linus said that 2.4.0 will not include a JFS, but expects ReiserFS to appear in a 2.4.X release sometime after. If not it will go into 2.5/2.6.
Out of all the filesystems XFS looks the best for scalability.
However SGI have scaled back the full performance of XFS to get it going in Linux.
It will be interesting to watch the development of the Linux storage systems including filesystems, logical volume managers and various media support.
Why is this exciting? Because its needed for Linux to get into the hi-end server market, hence lots of input from SGI and IBM. That's not to say this is the only area required; we also need better SMP, support for NUMA and so on. Again all areas under active development.
If you don't think the development of the Linux system is exciting, then you know less about computers than you think.
MacOS?
You've got to be fucking kidding me.
MacOS X is the first modern operating system (and I'm talking 1994 definition, here) to come out of Apple, and it hasn't shipped yet. Why don't you take your toy fifteen year old OS and go outside and play?
Put up or shut up, indeed.
-
Just because it works, doesn't mean it isn't broken.
YEAH! The first NOFX fan (besides me, of course) I've ever seen on /.
This dead horse won't stay dead.
Hmmm, just like those other carcassessess 'best distro', 'best editor', 'best wm', 'best news site', 'best cola', 'best os', 'best reply to a idiotic posting', etc...
Fran
:):):)
1st 1st Poster of the new Millennium!
On a P133 w/ 64MB of ram and 80MB of ram, reiserfs ;>... now i have a p166mmx w/ 128mb of ram, and i'm sure reiserfs will be that much faster...
proves to be loads faster, almost made me think the system was scsi based
I use ReiserFS. It seems to be the most mature of the 4, seems to work well and is reasonably fast. I still don't use it on any production servers, but I'm close to migrating my come server to it (I already use it for my home workstation)
I really shouldn't have used someone else's email address for this account.
Can't fill a niche in the Linux software line? Then go help at www.beunited.org, where every developer counts.
Because there are damn few people willing to work for free on a commercial project.
The current Slashdot moderation system is made by gay communists!
In addition to the points mentioned above about how to use LILO w/reiserFS you can use GRUB.
/Duncan
Duncan Watson
Duncan Watson
I am having a hard time finding information on what to expect from the new kernel FS. Will 2.4 ship with one of these?
No, none of these will be in vanilla 2.4. Vendors might add them to their particular version.
Do any or all of them break the 2GB filesize limit?
All, as does ext2. This is more related to the kernel in general and glibc. In fact, if you install the enterprise kernel coming with Red Hat Linux 7, you can already use it.
Aren't Snapshots and Phase Trees conceptually the same?
In Phase Tree, you take the changes made over a small priod of time, and make a "snapshot" out of that (called a phase, iirc) and begin to write it to the disk. While this is happening you have a new phase that all of the newer modifications are written to.
Doesn't this remind you of the description of snapshots?
There: Something at a specific location.
Their: Owned by someone.
Please make sure your english compiles.
IBM, believe it or not, is reasonably clueful on these types of issues. They recognize that thier business is selling support, not software or hardware. This is why they've done so much work on Open Source Software in the past few years. They had their own, proprietary web server, which they dumped in favor of Apache when it became clear Apache was better. You could probably buy an IBM support contract for your toaster - just as long as the bill gets paid. In fact, it wouldn't surprise me if you called up IBM right now and said, "Guys, I need to pay you to support my XFS install," if they didn't say, "Sure thing - just sign right here!"
I too have been using Reiserfs for many months with zero problems. Reiserfs runs my root partition with journalling, and with access time, journalling and tails disabled, it is much faster for my squid partitions. Very versatile and stable.
the sole purpose for journalling in JFS is to reduce fileystem check times from unexpected disk unmounts (before a sync has been performed).
the logical volume manager in aix has nothing to do with jfs - jfs is the default fixed disk filesystem in aix, and aix can't be run with a logical volume manager (it's not optional like hpux or solaris), but the two are mutually exclusive and don't have anything to do with each other.
http://www.oss.sgi.com/projects/xfs/manpages/xfs.h tml
http://oss.software.ibm.com/developerworks/opensou rce/jfs/index.html?dwzone=opensource
Sounds like a clock hog... MS office findfast (shitty background disk indexer for those who never had to suffer through it), ring any bells, how about slashdot's favorite paperclip.
Read my plan to save the Bengals
Actually, you don't need to write journaled data twice - you can just write data to emptly blocks and after that change the metadata so that it points to the new blocks. This way you write as many block as with metadata only -journaling.
Of course this will lead to fragmentation problems, so filesystem using this kind of journaling would probably need to do some kind of defragmenting in the background.
All kernels 2.2.14 and beyond (or is it .15?) can do more then 2GB. ext2 itself handled >2GB fine.
------
Not a typewriter
Took a day for me to reply to this, but . . .
At work we use a server running Linux+Samba on a 45 GB hard drive. hda1 (/) is 100 MB, hda2 (/usr) is 5 GB, hda3 (swap) is 64 MB, and hda4 (/madtimes; I know thats against convention) is all the rest (~40 GB).
Upon installing Debian 2.1, the first boot up ran fsck on hda4. Wondering if there might be something wrong with the drive, I low-level formated the drive. On reinstalling, it did it again. Thats when I found out that
However, you are correct taht the 2 GB limit still exists, but its rare that you need more then that. One time when I did was when I was getting an image of my 6.4 GB hard drive onto my 30.7 GB hard drive for backup (I was reinstalling Windows on the smaller drive and wanted to fall back on the orginal if soemthing went wrong). To get around the limit, here's the command I ran (or some variation of it, can't remeber exactly):
dd if=/dev/hda1 | split -b2048M - hda1
Then, if I needed the back up (fortuantly, I didn't) I could run:
dd if=hda1* of=/dev/hda1
------
Not a typewriter
Lilo will boot Reiser if the filesystem is mounted with the 'notails' option. John is also working on supporting tails. Grub currently boots Reiser partitions w/ no problems.
-b
Real scientist don't care for politics. Naturally, the orig. writer really meant the scientist worked for the former sovjet union. There's difference between being an idealist socialist or being someone who just enjoys science/exploration and doesn't care much for economical gain.
(-% TwistedMind %-)
It's allready possible to create a encrtypted (loop back) filesystem/partition, where every file is encrypted. Unlike NT with it's driveletters, it works quite transparantly. On the other hand, integrated encrypted per file would be very useful, if one could choose the encryption (via modules or something like that).
(-% TwistedMind %-)
Hmm, even if you would turn *everything* (all included 'drivers' too) on, you wouldn't even use all of the source, since it is multi-platform.
(-% TwistedMind %-)
since journaled filesystems by definition write everything twice, once to the journal and then finally to the location on disk where the data belongs
got any more info/links (eg. white papers) on this? i have searched every site mentioning journalling FS' that I have been able to find, but tell me whether a real journalling FS is supposed to log metadata only, or both.
I can plug any electric product into any outlet and be reasonably sure it will work. I don't have to have other types of outlets installed in order for it to work.
Sorry, try again.....take your US hair dryer, take it with you to the UK, and see how long it lasts when you plug it in :-)
In this sense, these two things you are trying to contrast are more alike than you seem to realize.
One word: Challenger
__________
Not true. If you want to run a KDE-based app, you need to have KDE installed, but that's only for the libraries -- you don't need to run KDE. Same for GNOME apps. Every KDE app I've tried works under GNOME, and vice-versa. There might be exceptions I don't know about, of course.
I imagine there must be KDE-app features that aren't supported under GNOME (and V-V again). Then again, there seems to be a deliberate effort for each desktop to allow cross-compatibility. For example, GNOME has a status dock that's compatible with KDE status dock apps, even though there are almost no GNOME status dock apps.
__________
typedef long long size_t;
Comment removed based on user account deletion
Plan9 is not unix!
Maybe I've spent too much time trying to learn the ways of Linux these past few months, but what the heck is a Journaled FS and where can I read up on it ? Journaled as in "hierarchical" ? retaining older versions of files for rollback purposes ??
-Billco, Fnarg.com
"Opinions are like *ssholes, everyone has one"
Me too space shuttle projects probably don't need to be failsafe.
Nuclear weapons should probably be built to marginally or slightly higher engineering standards.
I'll see your senator, and I'll raise you two judges.
After skimming through the whitepaper, I have to wonder.
What if the third tree in cache is growing too rapidly before the second cache tree can be updated to disk? Then do we get into a situation where we have to block all additional modifications to the filesystem until the next phase transition occurs?
This would be a problem. What download speed would you have to achieve before you could cause this to occur? (Probably so high it's not worth considering.) But just the thought of having to temporarily halt the saving of all that p0rn to disk -- shudder.
I'll see your senator, and I'll raise you two judges.
Yeah. I can just imagine. If the driver were to spill his coffee, I could have a serious burn on the back of my head or neck.
I'll see your senator, and I'll raise you two judges.
First let me correct my sentence, "Biggest concern is stability." should have been, "My biggest concern is stability." I do like to talk in complete sentences.
Now let me point out that the ext2 basis of ext3 is one of my problems with it. The fact that it is so new, and merely grafted on to ext2, makes me wonder how reliable it is.
ReiserFS was built from the ground up to marry journaling to data storage, so the fact that it is beta-quality does not raise concerns of backwards-compatibility-induced errors.
Furthermore, ReiserFS is also based on existing, proven filesystem technology--it uses the superblock/bitmap/inode metadata-structure present in ext2, and it organizes this with a B* balanced tree structure for fast searching and sorting.
One nice feature of ReiserFS over ext2 (and quite possibly ext3) is the ability to dynamically allocate inodes, so I never have to worry about having too many files. Small features like these are the rewards for not choosing a filesystem for backwards-compatibility.
I do not belong in the spam.redirect.de domain.
All the recent Bezilla builds have been green in tinderbox. That means it is compiling without errors. So... I think they have a workaround.
-- ERICmurphy -- www.jabber.org for open-source, XML-based IM
My only political goal is to see to it that no political party achieves its goals.
this article is an excellent overview of journaling file systems in general and the ones available for Linux. http://www.linuxgazette.com/issue55/florido.html
This will probably last two seconds and then it's flamebait. WinMe works for me. The only way I got it to crash was by smashing in a soundblaster Live and installing w2K drivers for it without removing my older Opti soundthingy first:-).
People replying to my sig annoy me. That's why I change it all the time.
Survival of the fittest is pretty useless when there is only one possibilty.
Which one is the only possibility?
Today we face another choice and I sure that is darwinian way we'll select the right next generation filesystem.
If evolution is at work on software, why do we have KDE, Gnome, and now Xfce?
Early adopters like you make me sick. I'm a FAT12 man ... once it's stable I may consider FAT16.
BeOS has the best Filesystem (not the fastest though). It's 64-bit, journaling and make use of the attributes and mime types. For more about it, click here.
This is not, properly, a function of the file system.
Instead, you want a file-monitoring daemon, independant of the many fs's you may be accessing. This will include monitoring the changes across NFS, etc., and removable-media.
SGI Irix does this beautifully with its File Alteration Monitor (FAM) and Inode Monitor (IMON).
If you've ever used the 4Dwm filemanager, you'll know what I mean.
Fortunately, SGI has added these projects to its OSS roster, and documented, portable sources are available at: http://oss.sgi.com/projects/fam/
Jeremiah Cornelius
"Flyin' in just a sweet place,
Never been known to fail..."
Isn't security built around this very thing?
Since the context of the conversation revolved around things root does, I replied in the context of someone who is root, so no.
Furthermore, as hardware moves more and more to firmware, there are bugs that can turn a machine into a boat anchor. I would very much like the OS to refuse to perform these operations.
Not me. I want it to require confirmation that I REALLY REALLY mean to turn it into a boat anchor, but if I confirm, it should DO IT. (Possably, I know something it doesn't! Like the hardware is modified or I want the disk dead for some reason).
Consider 'rm -rf /' for example, it WILL do it.
In the context of mounting a cleanly unmounted ext3 filesystem as ext2, that thing that I know and it doesn't is that I did unmount it cleanly, and from the documentation, I know that this is safe.
If you can't deal with it, you should not allow the user to use the feature at all.
That's not the Unix way. An OS should NEVER absolutely refuse to do what it's told to do (if it's possable). In this case, it IS possable. This is an especially good thing in a developmental file system. People who can't or won't follow that simple instruction shouldn't be messing with a pre-beta filesystem at all.
Your experiances and mine are quite different. The only files I've lost in a power fail situation were ones that were new and would have been suspect anyway.
Any filesystem will loose recent data if the power fails simply because there will be uncommitted buffers.
Of course, killing off kflushd and various other 'performance' tweeks can and will greatly increase the odds of loosing data.
If you want journaling, I have tried ReiserFS in production, and found it to be quite nice in versions for 2.2.x. It recovers much more quickly than an fsck.
However, I believe that this limit only exists on 32-bit architectures -- perhaps there was a bug which prevented 64-bit architectures from using big files too, and that's what was fixed in 2.2.14 (or
--
The key word here is background. A defragmentation process running with a low priority that flushes its changes to disk regularly will not affect applications running at normal priority. Of course, if you're burning a CD or something, you will have the option of turning it off temporarily.
As an aside, the Indexing Service on Windows 2000 isn't bad. The original version was written by one of these people.
--
"Where, where is the town? Now, it's nothing but flowers!"
ext3 journals both data and metadata. Part of the current effort is being directed into allowing it to journal metadata only. Metadata-only journalling is useful in situations where good write performance is required, since journaled filesystems by definition write everything twice, once to the journal and then finally to the location on disk where the data belongs.
--
"Where, where is the town? Now, it's nothing but flowers!"
NeXTStep used the Unix file system and thus has no more attributes than any Unix system. They used ".app" directories, which is a good example of storing such information in a way that is easily copied to other systems, and in a way that avoids complicating the file system. Each "attribute" is a standard file and thus can be read/written using the standard file I/O mechanisms. MacOS X does the exact same thing.
I would argue that "directories as files" should be supported, but this can be done entirely at the application library level. This would mean that reading a "directory" and then writing the same bytes to another system would produce an identical directory with identical contents on that system. I would argue that this is an area where Linux should abandon Unix/Posix and really try to make something better. But it is still not an "attribute" in that there is nothing special about the internal data and it is not inserted into a database.
I also think that the Unix owner/group/date stuff is a mistake and should be imbedded in the file data somehow.
The basic problem is we need a simple block of data that 100% describes the file. To most users, the date, creator, comments, document type, etc, are all parts of the file and thus anything that does not replicate it is not user friendly. And I don't for a minute believe that application designers are only going to put "unnecessary" data into the attributes.
Therefore I think *all* attributes (including the Unix date, permissions, group, etc) are a mistake. ALL data should be copyable by reading the file and writing the same bytes to a destination. Anything else, especially a system with low-level knowledge like BeOS, is going to make computers hard to use.
If you can afford 12 9GB SCSI disks, surely you can afford a UPS! The power failures are going to seriously shorten the lifespan of your hardware (which point reboot speed is meaningless).
I have a total of 12 9GB disks, all SCSI, and I'm running them as two RAID0 devices. Both of the metadevices run ReiserFS. You won't BELIEVE how frickin' fast things are :)
Our neighborhood sees power failures almost weekly (bastards don't want me to have a good uptime, I guess :), and my box doesn't flinch. Granted, physical power up takes ages (twelve disks don't exactly spin up as fast as one :) but once the Linux kernel starts up, you can't even *tell* it's running the journal replays since it scrolls by so damned fast. It's very reliable (never lost any data), and fast.
I was doubly impressed that the kernel patching went so smoothly (linux 2.2.16 + latest (at the time) raidtools + reiserfs). One try, it all worked. Wow, open source is fun sometimes -- ever try adding support for a new filesystem type to an NT box? ;)
Read my stuff.
We don't add encryption to the files, but the other things you say are correct. Glad you had a good experience.
> An OS should NEVER absolutely refuse to do what it's told to do
;)
Isn't security built around this very thing? Furthermore, devices are designed to refuse to perform an operation that would put them in an invalid state. Furthermore, as hardware moves more and more to firmware, there are bugs that can turn a machine into a boat anchor. I would very much like the OS to refuse to perform these operations.
Hey it's kind of a theological question, should the OS that can do everything be able to create a virtual rock it can't lift?
I've finally had it: until slashdot gets article moderation, I am not coming back.
> Don't try this on Linux! The ext2 fsck is horrible after a powerfail, and I've lost superblocks and had to re-install :( .
Doesn't it scan for backup superblocks? It should have created at least a dozen or so of them. At the very least, there should have been some utility runnable off a rescue disk that could copy a backup superblock.
I've finally had it: until slashdot gets article moderation, I am not coming back.
Any word on ReiserFS for FreeBSD? I've searched around, and found several people asking, and no one seems to have an answer.
I've finally had it: until slashdot gets article moderation, I am not coming back.
Heh, journalling filesystems are to handle the case of the system crashing. It doesn't sound like it's really done anything for you. :) But I suppose it's nice knowing that it's there.
I'm not sure if JFS doesn't support mixed case or is case-insensitive; they weren't clear on that point.
Case sensitivity in general is very important for Unix systems. I'm quite certain that it is required by Posix and other standards. I've seen a good number of cases where case was all that separated two files (e.g., Makefile vs. makefile) in the same directory. Now you can easily argue that this is a bad idea, but it certainly is part of our heritage, and I, for one, like it.
XFS is optimised for dealing with streaming media, and so deals well with high IO and large files.
This is not entirely true. XFS will stream media if you want using GRIO (guaranteed rate IO) features if you set your file system up that way, though I am not sure the Linux version will have that.
I would refer users to the website for a more comprehensive view of XFS. Basically XFS has been running for about 6 years on IRIX machines. It is a 64 bit file system, end to end. It is journaled. It is designed for speed, both at the OS level and at the hardware level (you can hit and sustain in excess of 97% of theoretical max drive performance in various cases with SCSI systems, and large block IO). It is designed so you can have millions of files in directories, with files in the petabyte size if needed.
Basically XFS is really one of the best file systems out there.
Saying it does streaming media is like saying Linux can be used via a vt100 emulator to edit files. It can do so much more, and it does it very well.
Combine XFS with a well designed volume manager, and you can have your file system saturate your IO bus. This is nice if you need lots of IO capability. XFS based filesystems (atop XLV) sustained 7 GB/s (thats gigabytes, not gigabits, per second) several years ago in a test, reading and writing to a single file. The limiting factor was the number of spindles one could attach to the machine. We used 864 if I remember the number correctly.
XFS scales provided the LVM scales.
This isn't true. At least as of the 2.2.16 kernel, the 2GB limit still exists. (I know for sure, since I just tried it.)
The on-disk filesystem structures of ext2 can handle files much bigger then that.
The kernel code can handle relitive seeks anywhere within the file. I'm not sure about glibc.
You run into problems with absolote seeks in programs which cannot handle a 64-bit type for the seek. This includes glibc in some cases.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
Linux isn't growing at all in the consumer desktop market.
Source?
Think hard about Linux. Who decides if you run GNOME or KDE? Not *YOU* but the programmers who write the programs you need to run.
This is sorely flawed reasoning at best, and absolutely bogus flamebait more likely.
First of all, if we except your rather flawed reasoning, then anyone who runs programs they didn't write are slaves. That includes you and your favorite OS.
But again, your reasoning is flawed. I can run KDE programs and GNOME programs and old Xt programs and even terminal programs, all at the same time. I need to have all the libraries a program requires installed, yes, but go ahead and show me a program that will run without a library it needs.
Better still, since most of this stuff is Open Source, we can take the program and re-write it to use the desktop environment of our choice. We can even change the desktop environment if we need to.
With BeOS, on the other hand, you're locked in, and cannot change it. Sounds like you're the real slave to me.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
What makes Linux bloated isn't the kernel (which weighs in just under 2 million lines, about 500,000 more lines than BeOS)...
... but all the crap around it (XFree, Mozilla, GNOME, KDE, and the dozens of libraries)
Lines of code isn't a terribly accurate measurement of code bloat. Perhaps those 1.5 million "extra" lines in Linux go to support things BeOS does not. Drivers. Platforms. Features. Stability. Security. Whatever. Maybe they're documentation. Maybe they're copyright notices. But then, we really don't know, do we? We can't know, because BeOS holds us hostage with its source code.
The nice thing about Linux is that nobody is forcing you to run all that "crap", as you so eloquently put it. If you want a stripped-down, bare-bones system with nothing but X11 and a single app, you can do that. Don't need the GUI? You can toss that, too. If you have the horsepower and the desire, you can also run GNOME with Enlightenment and every silly graphics effect you can think of turned on, plus sixteen different versions of Mozilla at once. Your choice.
That is one of the things that makes Linux so popular: Choice. We like having the ability to make decisions about we use. We like being able to choose the software that best fits our needs. We do not like companies that tell us they know better then we do, and no, we can't make changes and we can't see the source.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
I usually don't reply to AC posts, but this was just too good to pass up...
Windows95 had a minor version of this failing, in that, at the login screen, you could do a ctrl-alt-esc, select "run" and run "explorer", and get in without providing a credential to the OS.
Um, you could also just hit that "Cancel" button on the logon dialog and skip right past it...
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
I don't even know what a superblock IS.
...ID3 information is injected into attributes...
So, you know very little about Linux, yet you have no problem pointing out all the things it does wrong.
Word of advice: Don't complain about things you do not understand. You don't see me complaining about implementation details of BeOS. Why? Cause I've never used the thing.
Can you please explain WHY attributes break portability?
Because they're not portable. They don't work on other systems. Anything that depends on them will not be portable to other systems. Thus, attributes aren't portable.
Is that clear enough, or do I need to draw you a picture?
You complain constantly about all of the "Linux zealots" who ignore problems in their favorite OS, yet you seem to be the biggest perpetrator of that particular vice.
Gee, like how it is already embedded in the MP3 itself? Next you'll be saying attributes can also store the title of an HTML document. What progress!
Innovation should not be held captive to those who cannot innovate.
This is true, but does it apply? Are BeOS attributes really trying to improve things, or are they trying to pull that favorite industry tactic of locking us into a single vendor? (I actually suspect the former, but I have to consider the possibility.)
The case can be made that application-level attributes do not belong in the system, but in application-level libraries. By keeping such information in the files themselves, they are easily transfered to other systems, and do not require system-level support. Meanwhile, you can still provide a standard API to get at the information with an application-library. Thus, you get the best of both worlds.
And I couldn't care less what *YOU* think is an outmoded model. The minute you can get an economist or somebody with knowledge of the industry to tell me propriatory is dead, then I'll listen. (Emphasis mine.)
No you wouldn't. Slashdot has had countless stories about such things, but you continue your tired crusade every chance you get. Meanwhile, you will happily ignore yet another legitimate complaint about your favorite OS: That anyone using it is locked into a single-vendor solution.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
I said nothing about BeOS, just the filesystem.
... BeOS VM and disk cache ...
... my BeOS machine ...
Oh really? To wit:
Obligatory BeOS plug
Not only are you a troll, you're not even a good troll.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
Yes you can. As pointed out, you can boot ReiserFS as long as your kernel containing partition is mounted with notails, and you can boot off a kernel floppy or with loadlin in any case. EXT3 will work fine - see the documentation. LILO pretty much works by reading a bunch of blocks off the drive and assembling them into a kernel, so is pretty filesystem agnostic providing the filesystem doesn't do strange things like tie together the tails of multiple files into a single block in order to save space (such as ReiserFS, hence the notails option for /boot). I don't know enough about the on-disk layout of JFS or XFS, but personally I'm not inclined to use either of them for my root filesystem yet :)
No! JFS is a good filesystem but it does not do data journalling, only metadata journalling. It is left to the application to do data journalling (like an RDBMS doing transaction logging).
You misread the post. An ext3 fs only has to be unmounted cleanly if you want to remount it as ext *2*.
I have seen the future, and it is inconvenient.
Once snapshots are ready, waiting for fsck on an FFS filesystem will become unnecessary. If a filesystem is dirty, on bootup the system will create a snapshot, and run fsck on that snapshot in the background.
ReiserFS is signifcantly faster than Ext2. I'm speaking from having run ResierFS on my computer for a few months.
A deep unwavering belief is a sure sign you're missing something...
When did I say something about BeOS? I'm talking about bfs the file-system.
A deep unwavering belief is a sure sign you're missing something...
Why is it such a problem? Unlike the Mac, BeOS attributes are not flaky. (For example, files aren't chained to the program that created them.) BeOS stores file-types as a MIME attribute. So if you want all your /text-html files to be opened by a particular problem, its just as easy as going to file-types and associated /text-html with StyledEdit. Its the exact same thing as going to the GNOME control panel and associating ".html" with GEdit, except you don't have any ugliness in the file-names, and the system automatically figures out what type files are. If you don't like attributes in general, then please explain why.
A deep unwavering belief is a sure sign you're missing something...
Tell that to all the BeOS people. Seriosly though. If I talk about the advantags of bfs during a discussion on file systems, does it automatically mean that I'm saying "Linux sux, BeOS rules?" I never said ANYTHING about BeOS. I'm talking about the filesystem!
That said, your arguement has some merit. A bunch of important software WAS canceled. However, you're not totally right that BeOS is dying. Software is coming to the OS at quite a good clip. We've already got a good video editor, and other media programs are being made. BeUnited is helping a lot by starting projects to address gaps in the software line. Tracker and deskbar has been OSSed (under something similar to the BSD license) so the GUI wil continue to evolve. The system is modular, so the system itself can be improved by users. BeOS is far from dead.
Your point about the focus shit is totally off base. Be never said that BeOS is dead. They focus shift-ed. Sure, it sounds like a cop-out, but consider this. BeIA and BeOS are more or less the same thing. Be's plan is to be able to pull out a BeIA release out of the BeOS source-tree anytime it wants. Meaning, that any cool stuff integrated into BeIA gets into BeOS too. And this is not bullshitting on the part of Be. OpenGL, Java, networking, and a good browser are all critical for BeIA. And they're important for BeOS too. So guess what, Be is bringing them to BeOS! And before you say, "oh, that's just vaporware" remember, a lot of these things are in late beta. www.betips.net is already running on the new BeOS networking architecture, a lot of people aready have the OpenGL beta, Java-personal has already been ported to BeIA, and Opera is already working on a web-browser for it. (The Opera browser hasn't been released yet, but already works and can pass standards complience tests.) And with Compaq making a BeIA-powered IA, you can bet that these technologies are going to get to the public.
A deep unwavering belief is a sure sign you're missing something...
1) probably true, that Linux wouldn't need reinstalling in these case. However, I'm not going to take time to fix what fsck can't. It is easier for me to just copy over my CD-backup than to fix the fs. Seriously though, if fsck gives me a "lost superblocks error" and the system won't boot into multi-user mode, then what the hell am I supposed to do? I don't even know what a superblock IS.
4) Can you please explain WHY attributes break portability? How exactly does it lock you into the OS? You forget four things:
1) Dealing with attributes is required for anyone writing a BeOS fs driver. Thus, any attribute data can be extracted by the fs driver if they are reading a bfs disk from another system.
2) The system takes care of attributes when sending things outside the OS. If send a picture to a non-BeOS user, the attributes are stripped out. If you copy stuff to a non-bfs disk, then attributes are taken out. If you post something, the appropriate extensions can be added. Its not a big stupid system that automatically makes file prorpiatory. If you're using a GIF file laden with attributes, it will still be a gif file when you send it to a Linux user. They just won't get the benifets of the attributes.
3) Attributes kick ass! No stupid MP3 database programs, ID3 information is injected into attributes (which are stripped out when you send them to Napster!) and the FS serves as your database engine. Email can be indexed by user, sender, date, time, subject, whatever. You can set up custom searches to get all email from your boss written within the last 5 days. Best of all, there is no central database to take care of, since all this info is in attributes. Think of it as UNIX-style file data (creator, data, group-info, etc) taken to the next level.
4) Innovation should not be held captive to those who cannot innovate. Nobody if forcing you to take advantage of attributes. If your cross platform program needs to share the same file-types between OSs then DON'T STORE ANYTHING IN THE ATTRIBUTE DATA! Keep a central database or whatever you want. However, if you couldn't give a flying fuck if other OS can take full advantage of your files (as long as they are a standard type, other OSs will always be able to *READ* them) then by all means, take advantage of the technology.
And I couldn't care less what *YOU* think is an outmoded model. The minute you can get an economist or somebody with knowledge of the industry to tell me propriatory is dead, then I'll listen.
BTW> BFS is documented in a book, and an GPL'ed clone is part of AtheOS. Go have fun with it.
A deep unwavering belief is a sure sign you're missing something...
Really, or are you being anal?
A deep unwavering belief is a sure sign you're missing something...
No you can't. Lilo won't boot a ResierFS partition. You have to make your /boot partition an ext2, and another ReiserFS partition for /usr and everything else. I would guess the others are similar.
A deep unwavering belief is a sure sign you're missing something...
ReiserFS is signifcantly faster than Ext2. I'm speaking from having run ResierFS on my computer for a few months.
A deep unwavering belief is a sure sign you're missing something...
I think that would only happen when you were genuinely exceeding the transfer speed of the disk, in which case, "you dummy". tux2fs probably *will* take more memory (substantially more?) than ext2 or a journaling filesystem, but with the amount of memory that most systems have available for file cache, I doubt that is a problem.
Embedded devices that need consistant IO rates with little RAM for buffering probably want to look at a journaling FS.
i already posted this once when discussing xfs beta article. anyway there is great article on journaling filesystems at linux gazette.
it explains different features and concepts related to the 4 different journaling filesystems. XFS, JFS, Ext3 and ReiserFS.
-- http://electronicintifada.net --
Back in the old days (around 1994) one had the choice between minix, xiafs, ext and ext2 as filesystems for ones linux box. ext2 stayed
Today we face another choice and I sure that is darwinian way we'll select the right next generation filesystem.
The big difference is however that the filesystems that existed back then didn't have any money (companies) behind them who could blur the choice and ext2 survived because of its technical superiority over the others.
Something else that may lead to different filesystems suviving is that the Journaling filesystems mentioned all have their own disign and possible uses. On some systems it may be worth the performance penalty for data security on others it may not.
-
open Open for writing starts a new version of the file, which initially shares all the pages of the old one. Writes are copy-on-write, so you can append to a file at low cost. Until the file is closed or committed, other reading processes see the old file.
-
commit Commits the file; the new version replaces the old one. This can be done without closing the file.
-
revert Reverts the file to the last commit; the old copy becomes the current one.
-
close Commits and closes the file.
-
exit Commits and closes all files.
-
abort Reverts and closes all files.
-
SIGKILL, etc. Works like abort.
This gives database-like transactions at the file level; either the update happens completely or not at all. There's an "unfrozen" file mode, for files that aren't updated as a unit and need to be looked at by other processes during update, but it's not needed much.Transactions are really valuable when accessing a remote file system, where you may lose connectivity during an update. LOCUS was a distributed UNIX in an era when networks weren't very reliable, so that made sense.
XFS and JFS allow an easy migration path to/from "mid-sized" iron to/from Linux. If you have a JFS volume from an RS/6000 you want to mount on Linux, you're out of luck with XFS, you need JFS. Same thing with XFS and SGI. This is a major migration issue in shops that have these boxes.
Also, there is the issue of IBM openly supporting, lets say XFS. While that might make good business sense, it would mean lots of bad PR and midshare loss.
The "market" will eventually make the winner(s) the defacto standard, but it never hurts to have direct support for other fs's you may need to interact with. I use support for the windows file systems all the time (fat32), but I would never dream of really using it as a main Linux volume....but it is always mounted.
-Pete
Soccer Goal Plans
It appears that this is NOT something to hold in opposition to Ext3 - but rather that this is part of the latest versions ext3!
Sort of like saying: "Forget all that Red Hat and Debian crap... LONG LIVE LINUX! " ?
Thanks for the link, however.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Comment removed based on user account deletion
I installed the ReiserFS kernel patch and utilities today, and just finished replacing all my partitions with ReiserFS partitions. Let me tell you why I chose ReiserFS over the other three mentioned.
Biggest concern is stability. Ext3 is version 0.0.2e, which is not a reassuring sign. JFS is 0.0.3 (or somewhere around there), again not very reassuring. I don't know the version number of the XFS port, but it does say on the website something about the FS port being beta, and that it "may damage" data. Furthermore, it is only available for kernel 2.4.0, which doesn't seem to work with my sound card module (I'll never by a product based on Aureal hardware again). ReiserFS is up to version 3.5 on stable kernels, which indicates that it's been around for a while. Plus, the testimonial on their web page says sourceforge uses RFS for half their servers. If it works for them, I figure it works for me.
Another problem with Ext3 is the fact that it's just Ext2 with journaling grafted on. I want a filesystem built around journaling, not journaling built around a filesystem.
XFS and JFS bother me because they are early ports of software for different platforms. I trust Ext3 over either of these, simply because Ext3 was meant specifically for Linux. If SGI or IBM can make their ports stable, they'll be worth looking into.
RFS was built from the ground up as a Linux-native, journaling file system. People have used it and loved it. I love it too, at least as much as I can after knowing it for only a few hours. I've reset it a few times now, beaming as the screen says, "3 transactions replayed in 2 seconds" and mounts my partition, rather than saying "fsck forced... 1%...2%...3%..." and mounting my partition five minutes later. So far, I've not found any lost data. The only thing that bothers me is the lack of extended attributes, but I never use them anyway.
FSCK forcefully wiping out data in order to boot was what sent me crying back to WinNT about a year ago. It was the only grudge I held when I went crying back to Linux from NT two months after that. I treated my power button delicately, and shut my PC down every time a thunderstorm started.
With ReiserFS, this could be the beginning of a beautiful friendship, where I can tempt the fates by continuing to work in a thunderstorm, and I don't have to tremble with fear that I might accidentally pull the wrong cord if I need to unplug something on the same outlet as my PC.
Whether it's XFS, JFS, RFS, or Ext3, at least we can say that NT no longer has ANYTHING over Linux. Choice is good.
I do not belong in the spam.redirect.de domain.
"I think that XFS and JFS should both be integrated with it. ext3fs is only useful for scared admins, as it has none of the basic speed advantages of the others"
f
XFS is a journaling system on it's own and it is SGI's crown jewel achievement - It does not need to be integrated in any way with ext3 as that is not the goal here. Eventually XFS on linux will be used with CFXS which is journaling with clustering which ext3 does not support. XFS on IRIX is one of the fastest journaling file systems on the market and it scales incredibly high. No other Journaling system for linux, not even reiserfs, can match the speed or scalability of sgi's xfs.
On a side note, I use reiser on my box at home and I'm very pleased with it. It is not however, as fast or as scalable on an enterprise scale as XFS. For more info on XFS please see http://oss.sgi.com/projects/xfs/papers/xfs_GPL.pd
The specs are impressive. While the code is beta, they hope to achieve the same level of performance on linux as has already been achieved on IRIX.
Do you really mean that JFS doesn't support mixed case filenames, ot did you intend to say that JFS is case insensitive? I know I'm not the only one who thinks that case sensitivity is something that should be reserved for the written word and passwords. Mixed case filenames are fine, just don't force me to use them -- C makes it really easy to shift a byte's value by 64...
--
"A witty saying proves nothing" - Voltaire
Okay, so which one is closest to replacing Ext2? I am having a hard time finding information on what to expect from the new kernel FS. Will 2.4 ship with one of these? Do any or all of them break the 2GB filesize limit? With 2.4 so "close" to release, you'd thing there'd be more *easily* available info out there (I'm sure it's there and I just can't find it...
http://www.globalfilesystem.org
A file system for SCSI over fibre that have journaling.
I'm curious...does anyone know how much more RAM/CPU Tux2/ReiserFS/... need over and beyond Ext2?
I wrote about RAM in another article - Tux2 won't use a lot more, and because of its ability to throttle the dirty cache, could use significantly less than we're used to.
As for CPU - I haven't really noticed a big difference vs Ext2 but I'm not in real-world testing situations yet. The only CPU intensive thing Tux2 does is make extra trips into the block allocation bitmaps. There are roughly zero extra trips the first time you write a file, and 100% extra trips if you rewrite the file without truncating it first - actually, a pretty rare thing to do. A database might do it though, and for that case I'll provide a per-file disable of the copy-on-write - the metadata will still be protected but the database will be responsible for doing its own crash recovery for the file data, which most databases do pretty well. In fact, the phase tree algorithm started life in a database.
--
Have you got your LWN subscription yet?
Everyone loves to ask those really stupid questions about combining open source projects. Being an open soruce developer, I find these very irratating. The following reasons should be enough to either make you understand, or prove you can't understand...
- Compatition is key to all things being good...
- Each FS has an almost compleatly different techincal design, and each has a chance of being the "best" general purpose FS.
- Each FS might in fact be the "best" at some things
- Each developer is working on their respective project because they think it is the right thing to do. You tell them "No, stop that work on your project and help this one" and they will. They will see that the comunity thinks of their project as meaningless wheel-reinvention and quit.
There are many more reasons that others will point out as well. Look at other industries. Why are there different radio stations? Different cars (even made by the same company)?
Hey, if I stop one more person from telling my friends and my self that we should join this other project or that then this note is worth it.
Got a problem with my view? I'm easy to find.
Jamie B
As far as I can tell, all of these filesystems allow files vastly bigger than 2GB, but the interface between VFS and LIBC still nicely enforces the 2GB limit for most purposes.
This is the wrong thread on which to try to find resolution to that issue; take it up with the folks that defined ISO and ANSI C...
If you're not part of the solution, you're part of the precipitate.
Avoiding the obvious question of why can't the effort going into four diffrent projects be channeled into one
That's like asking all the *BSDs to work together with Linux. Diversity is a very good thing. Personally I think SGI's XFS is going to kick some serious butt since the SGI folks are all for high performance and huge I/O throughput performance. They've also shown they can do it (on Irix). Not that the other parties will stay behind. I just think SGI has a better chance at dominating the Journalling FS landscape in Linux...
-adnans
"In short: just say NO TO DRUGS, and maybe you won't end up like the Hurd people." --Linus Torvalds
I have production systems now using Ext3, which is little more than Ext2 with full data journaling (and completely reversable). This is NOT an endorsement of Ext3, but the fact is that it is the only use usable at this point (largely because it is just a slight evolution from Ext2, unlike the others).
I am in the middle of a ~30 page HOWTO on NFS+Journaling. Contact me direclty if you are interested in a copy. Again, I have production servers and workstations running with Ext3 and sharing data out via NFS v2/3.
-- Bryan "TheBS" Smith
-- Bryan "TheBS" Smith
Independent Author, Consultant and Trainer
Okay, today, I'm actually going to preach the benifets of a BeOS technology.
.stupid-extension because file-type is stored within attributes. The user and edit attributes all they want, and custom file info (like gamma-info for a GIF) can be imbeeded into a file. For example, if you've got a special program that can display a GIF with variable gamma settings, it can embed those gamma settings into an attribute. Those attributes are ignored by other programs, and stripped out when sent to another OS (unless you use .zip compression.) However, when displayed in a program that recognizes that attribute, it can be used.
With all this talk of journeling file system, I'm surprised that bfs got ignored. BFS has several things going for it:
1) It's fast. While it is a journaling file system, on Bonnie, it is about 20% faster than my ReiserFS partition (which is closer to the outside of the disk too) on straight reads and writes. It is also a good bit faster on the per-char tests. Best of all, the CPU utilization is 30% lower than Linux in the sequential, and 50% lower in the per char (where Linux pegs the CPU). However, the rewrite tests, it is significantly slower. Something I think has more to do with the BeOS VM and disk cache than the file system.
2) It is ever so solid. I regularly (read: three times a week) shut down my BeOS machine with my power button. Not yet have I gotten a lost file, block, or data corruption. Linux regularly needs reinstalling if I turn it off in the middle of something important, and even NT bugs out on me for not shutting down. (I just hosed the system two days ago.)
3) It has had database capabilities for years, while ReiserFS still has them in planning. That might be a "gee-whiz" features, but nothing beats having your MP3s automatically entered in a database based on ID3 info. (Or emails, or pictures, or whatever.)
4) It has a flexible system of attributes. No more
A deep unwavering belief is a sure sign you're missing something...
Uhm, no.
:)
Linux 2.4 has a set of 64bit file calls which work natively on 64-bit achitectures, and work also on 32-bit architectures by using double word operations. You take a performance hit on 32-bit systems, but it works fine. The glibc 2.x has it, as does the kernel. You just have to ensure that the libc was compiled with support for it.
Remember, this is opensource. We've patched the libc and vfs layer with little trouble because of it. Now it just needs testing
--
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
You have to remember that XFS and JFS journal all data and metadata changes whereas ReiserFS and EXT3 only journal metadata. Thus, [XJ]FS protect your data from becoming corrupted where the others just help you boot faster after a crash (with respect to their journalling).
Ext3 can be used (once it's stable) on a preexisting Ext2 filesystem. The others cannot directly migrate from anything 'official'.
I like the ideas of all four. It may be that you want to have a combination of all of them in your system, but that would be pushing it.
I've used JFS a lot and it is really bulletproof if you set it up properly. It's heavily tied to the LVM in AIX, so I wouldn't expect much progress without LVM for Linux being a stable API. So, call it post 2.4.0 at least.
At work I have five workstations and two servers running ReiserFS. These have performed flawlessly over the past several months, as they have been eased into production.
The ReiserFS folks have been really good about finding and fixing bugs. Recently, a bug was discovered which crashed the system with ReiserFS-3.6 systems if you saved a file from Star Office on top of itself on an NFS server. That bug was eliminated with ReiserFS-3.6.17 in just a few days after being reported.
Since ReiserFS isn't merged into the official kernel tree, when you want to try out the latest kernel, you have to patch ReiserFS into the system yourself, but that is quite easy.
I look forward to the day when ReiserFS and these others are merged into the kernel. By the way, the 2.4 kernel is quite nice. Try copying a file several times your memory size from one disk to another (a 600 mb iso image should do the trick) on both 2.2.x and 2.4.0-test9pre-whatever. And try to do something with your system during the copy. You'll become addicted to 2.4.0, I promise you. Its wonderful.
I'm curious...does anyone know how much more RAM/CPU Tux2/ReiserFS/... need over and beyond Ext2? Journaling is an impressive feature, yet some of the machines I monitor aren't cutting edge; Pentium 200/64MB, Celeron 300/96MB. I've already tweaked these systems in other ways (no extra consoles, MTTR settings, ...).
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
Haven't we had this discussion umpteen times before? Such as... two days ago? There's even a link to a discussion on the four competing filesystems. Sheesh. Flogging the dead horse.
McKusick's Soft Updates has also a nice feature: unlike the journaling file systems, it does not have the burden of writing blocks to a logging device. So a soft-updates enabled kernel runs at the speed of traditional asyncronous file systems (ie, default ext2) while providing a very good level of reliability (it is not a syncronous file system, so it runs at a very enjoyable speed).
You can boot a Soft Updates file system without fscking it, the file system will be in a functional state. The only problem is that you might start to loose free blocks that are believed to be busy. So every 100 or 200 crashes you might want to run fsck to free those 100 blocks.
I agree with you regarding the ext2 file system when running in async mode: when there is a lot of activity on the disk, and a lot of changes to the file system, crashing an ext2 file system will loose a considerable ammount of data. ext2 fsck will not be able to recover your file system properly (it has happened to me a couple of times already).
For non-SoftUpdates kernels and non-Journaling kernels, if you are running a system with sensitive information, I suggest turning syncronous access on the file system (add option sync to it).
The sad part here is that the BSDs have traditionally been optimized for the syncronous case, so they run at acceptable speeds. Linux ext2fs has never been optimized for this case so in practice it is very slow.
I am using ReiserFS on my laptop, but on a server, if I had to choose, I would run SoftUpdates for BSD kernels and ReiserFS for Linux kernels.
Miguel.
When I was at the Ottawa Linux Symposium, there were talks on XFS, JFS, and ext3fs. It seemed clear that XFS was near beta, so the recent announcement was no surprise. Ext3fs also sounded near beta. Ext3 takes the simple approach of adding journaling to ext2 in such a way that as long as you unmount cleanly (so there's no need to play the log back), you can take an ext3 partition and mount it as an ext2 partition. From the talk, it sounded pretty much ready.
JFS was another story. My take on the talk was that people who atteneded it learned one important thing: JFS is the journaling file system to ignore. The Linux port comes from OS/2, instead of directly from AIX. It lacks such things as support for mixed case filenames. The answers to most of the questions were, "We hadn't thought of that," or, "We'll have to look into that." If JFS didn't have the "me-too" ego of IBM behind it, the developers would have realized that they were better off working on one of the other file systems.
I tried to ask this question a few months ago, but with no luck getting it posted I did some research on my own. I wanted to make a 60GB file server that would give me some insurance on my data. I was close to using the IBM JFS, but kept hearing about ReiserFS and gave it a try. (Heck, sourceforge uses ReiserFS on their servers, so it's good enough for mine.) Anyway, after a little more reading, I realized that ReiserFS doesn't just add journaling to a partition, it also restructures the filesystem into B-trees which can enhance access speeds, and it also adds a bit of encryption to the filesystem since it uses a hashing algorithm to sort the files.
In my opinion, you just get more. I also found the installation and recompile fairly easy to do. I've been using ReiserFS for the past 3 months with absolutely no problems.
Sometimes I doubt your commitment to Sparkle Motion.
tux2fs probably *will* take more memory (substantially more?) than ext2 or a journaling filesystem, but with the amount of memory that most systems have available for file cache, I doubt that is a problem.
.1% of the total in a filesystem. My guess is you won't notice any extra load on the buffer cache.
I've analyzed that question and I think tux2 will only use a little more cache memory, not a lot more, and it could even be less - see below. Tux2 uses per-block copy-on-write, and when the old version of a block won't be used any more (the normal case) that means you can just change the disk block number in the buffer - no extra memory used at all. The only time extra memory is used is when a file block is written over and over again, every 10th of a second or so - then you will sometimes get two copies of it in memory at the same time. The first copy will disappear as soon as it finishes being transfered to disk. This kind of writing pattern is rare with normal data but is common with metadata. Fortunately metadata is about
In fact, I think Tux2 will take a load off the buffer/page cache because it doesn't let dirty data hang around a long time - it starts writing to disk a fraction of a second after you start writing to a file. My plan is to have Tux2 shorten its phase length under heavy memory pressure, so the space needed for dirty buffers will drop down to just 100-200K, and you'll still get good performance.
Cache memory for reading under Tux2 is the same as Ext2 and most other filesystems.
--
Have you got your LWN subscription yet?
XFS is optimised for dealing with streaming media, and so deals well with high IO and large files.
JFS has been around for years under AIX. It's a well proven general purpose journalling filesystem.
ReiserFS is the best established of the Linux journalling filesystems. It has several fairly innovative features and is more efficient than ext2 in terms of space utilisation. People are using it as their primary filesystem now, although it's still in development.
EXT3 is (unsurprisingly) a development of EXT2. It lacks most of the pretty features of the other journalled filesystems, but has the significant advantage that you can turn EXT2 partitions into EXT3 (and vice versa) without any trouble at all.
For similar crash protection, you might want to try out McKusick's "Soft Updates" that appear in *BSD systems. Essentially, they are ordered disk writes that makes sure data gets on the disk before metadata is altered. They go through the buffereing system, so performance isn't bad.
:( .
As an experiment, I pulled the plug towards the end of 5 FreeBSD kernel compiles (SMP `make -j 4`). In all cases, the fsck upon restart was minor, just freeing inodes. In four of the cases, `make` just picked up where it left off, and finished the kernel compile, losing only ~40 seconds work. In one case, a `make clean` had to be done because something was incomplete.
Don't try this on Linux! The ext2 fsck is horrible after a powerfail, and I've lost superblocks and had to re-install
It is also proof that open source software does not just 'chase tail lights' - the work is substantially innovative.
Phillips is also implementing tailmerging (a feature from ReiserFS to efficiently store small files) for ext2/ext3/tux2.
For more details, check his web pages here, and the linux-fsdevel mailinglist.