XFS Merged into Linux 2.4

Alphix writes "As noted on KernelTrap Marcelo has merged XFS into 2.4 after a code review by Christoph Hellwig. The mail from Marcelo on LKML is here. Apparently it touched very little VFS code so people not using XFS shouldn't see any ill effects from this (it's even supposed to fix some VFS bugs). XFS is described by SGI as '...a journalling filesystem developed by SGI and used in SGI's IRIX operating system. It is now also available under GPL for linux. It is extremely scalable, using btrees extensively to support large and/or sparse files, and extremely large directories. The journalling capability means no more waiting for fsck's or worrying about meta-data corruption.' Let the stability vs. new-features flamewar begin."

Comments

Wonder...

[triptolemeus]

What Dave has to say about this.

Re:Wonder...

[The Almighty Dave]

Not a whole lot, really. I haven't used XFS and haven't read the article, so I don't have anything useful to say on the subject.

Re:but NTFS

[triptolemeus]

Not.

Original ntfs.sys

ext3vs XFS?

[dummkopf]

back in the days when ext3 was still in our dreams i downloaded the SGI XFS kernel from their site and installed it on my wife's laptop. it was extremely stable and had the advantage, that her "oops, i have to run off and just close the lid"-atacks would not corrupt the filesystem (which i would have to clean up...).

nowadays i use ext3 on my machines because it comes default with RH (by the way EL is now available for academia, woohoo!). hence my question:

can someone offer a nice comparison of ext3 versus XFS?

Re:ext3vs XFS?

[Trigun]

Here ya go.

Re:ext3vs XFS?

[Gudlyf]

You can always look back at this old Slashdot article.

Re:ext3vs XFS?

[HidingMyName]

Does linux have an XFS dump/restore ported to it? That makes a difference for our installation. Currently we use ext3fs so that we can dump to tape (in spite of Linus's hate for dump, the admin features of dump are very useful).

Re:ext3vs XFS?

[mcbridematt]

yeah, just hold the poweroff button :)

For me, ext3 doesn't offer much data redundancy over plain old ext2. It still suffers from ext2's dataloss problems. Infact, with ext3, I had the horror of i/o errors. Once I had a bad powerdown, and I came close to reformatting just because it wouldn't let me into php.conf.

Personally, anyone looking for a data-redundancy fix should use ReiserFS (which we have had for a looong time), JFS or XFS.

Re:ext3vs XFS?

[Molander]

Yes, there is an XFSdump and XFSrestore.

Re:ext3vs XFS?

[Ozric]

FASTER ...

ok next.

Re:ext3vs XFS?

[_|()|\|]

can someone offer a nice comparison of ext3 versus XFS?

Ext3 can grow or shrink an unmounted file system. XFS can grow a mounted file system.

Ext3 and XFS both have dump utilities, which many sys admins prefer for backup.

Ext3 supports three modes of journaling: writeback (risky metadata only), ordered (metadata only), and journal (all data). I believe XFS is comparable to ordered ext3.

Ext3 has been widely deployed on Linux, and it trivially reverts to ext2. The XFS design is mature, but its implementation on Linux is less proven.

Re:ext3vs XFS?

[_|()|\|]

Does linux have an XFS dump/restore ported to it?

Yes.

Re:ext3vs XFS?

[Fruit]

ext2dump is unsupported; in particular I recall a quote from Linus to the extent that anyone who uses ext2dump might just as well not make backups at all.

xfsdump on the other hand will work correctly.

Re:ext3vs XFS?

[Nothinman]

dump is not recommended with ext2 or ext3 because it opens the block device directly which bypasses the page cache and can give you corrupt data if there are dirty pages that havn't been flushed to disk.

I'm not sure if xfsdump is any smarter about it because of the DMAPI stuff available, but I'd be carefull.

Re:ext3vs XFS?

[Nothinman]

Personally, anyone looking for a data-redundancy fix should use ReiserFS (which we have had for a looong time), JFS or XFS.

I would definatley not recommend reiserfs if you care about data. I've had atleast 2 occiasions where it lost data for no good reason. I would rather use ext2 than reiserfs any more, but when available I use XFS.

Re:ext3vs XFS?

[Nothinman]

I hope you verify those backups because Linus' hate for dump has a reason behind it.

Re:ext3vs XFS?

[_|()|\|]

dump is not recommended with ext2 or ext3

Dumping a live file system can result in bad backups. Dump is best used on an unmounted file system. It can also work well with snapshots or split mirrors.

Re:ext3vs XFS?

[MSG]

xfsdump is definitely smarter because of DMAPI, and is safe to use on live filesystems.

Re:ext3vs XFS?

[Three Letter Acronym]

from the article:

"Of course ext2 isn't a journaling filesystem and therefore it isn't a journaling filesystem and therefore has less advantages then xfs, jfs, reiserfs, ext3."

you see, i just find stupid stuff like this funny and therefore i just find stupid stuff like this funny. So sue me and sue me!

Re:ext3vs XFS?

[Trigun]

Heh, Didn't notice that.

I just found the reference, I didn't write it, therefore I didn't write it. Nice catch tho...

And btw, nice catch.

Re:ext3vs XFS?

[xenocide2]

Another comparison I can vouch for by experience

XFS will ruin your day if you unplug the computer by accident, nosy dog or act of northeastern powergrid. Ext3 will just take a minute longer to reboot than normal.

Re:ext3vs XFS?

[tzanger]

XFS will ruin your day if you unplug the computer by accident, nosy dog or act of northeastern powergrid. Ext3 will just take a minute longer to reboot than normal.

Cites? References? They're both journalled filesystems. What's the point of journalling if it can't handle unclean shutdown?

Re:ext3vs XFS?

[Merlin42]

in response to:
http://aurora.zemris.fer.hr/filesystems/
Thi s seems like a pretty poorly designed benchmark. One of the major tests was copying b/w two partitions (which is a valid test), but they put both partitions on the same disk! Whichever partition hapened to be allocated near the outside edge of the disk would have a clear advantage. Also it is not clear if the read, write, and delete portions of the test were done using the exact same partition or if some filesystems were handicapped by being on the inside partition when others got the outside partition.

Re:ext3vs XFS?

[HidingMyName]

I hope you verify those backups

I don't but my scripts do :-).

because Linus' hate for dump has a reason behind it.

Yeah, Linus broke it (albeit in an attempt to improve file systems performance). It works O.K. for us because our file systems consists of a large number of (relatively small) seldom shared files (and a few larger but not very frequently updated files). What other back up tools are worth using? The admins around here vastly prefer dump. Tar,dd an cpio leave a bit to be desired as far as I can see.

Re:ext3vs XFS?

[Nothinman]

dd would have the same problems as dump, so that leaves tar, cpio or a 'bigger' offering like Arkeia.

The page cache is necessary, do you know how slow Linux would be if it had to make sure every on disk file was in sync with pages in memory?

The real 'problem' is there's nothing like DMAPI for generic block devices to allow things like dump to work like they should. Unless you want to use LVM snapshots.

Re:ext3vs XFS?

[Trigun]

You say poorly designed, I say 'End-user Pertinent'. SysAdmins do not get advice from Slashdot*

*Even when they do, they don't say they do

Re:ext3vs XFS?

[lgftsa]

We intent to. We have an Arena 8611 FC RAID unit connected to a Xeon rsync server. All the other servers back up to it daily, and it can then take a snapshot and write it to tape.

Stage two is multiple snapshots with enough free disk to keep dailies/weeklies going back as far as possible before being freed.

Re:ext3vs XFS?

[groomed]

Journaling doesn't mean that you no longer need to clean the filesystem after a crash. It just means that the filesystem is consistent in some (rather nebulous) sense.

Whether your filesystem is journalled or not, you need to check it after a crash. For XFS, you can use xfs_check & xfs_repair for that purpose. Of course to check your startup volume, you need to boot from a bootable CD or a different partition.

Re:ext3vs XFS?

[xenocide2]

Circa one year ago. You'll just have to trust me I suppose, but my roommate trashed his install pretty quickly by accidentally unplugging his computer FROM THE UPS. Hilarious. He spilled water on the floor next to a surge protector that powered his guitar amplifier, water heater and stereo. Went to pull that from the UPS (not sure why it was in there--"extra sockets" he says) and pulled the computer out. Rebooted and x crashed to console, the emacs file was full of zeros, and various other nuisances. All his user files were intact, thankfully. Sure would suck to lose the art he was workin on.

In contrast, I didn't bother with a UPS. Just ran ext3, and every time some jackass thinks its funny to pull the plug, or when the shitty middle of kansas power grid fails me, it just means an fsck on reboot.

The lesson is that not all Journallings are made equal. You can set ext3 to suck just as much as xfs, but it seems foolish. XFS only journals the meta data, rather than the data itself. Since I'm enlightening the ignorant, I'll go ahead and even link to SGI's faq on the subject. Basically their excuse is "thats a feature not a bug." Go figure.

Re:ext3vs XFS?

[mcbridematt]

I've had no real problems with it myself.

Re:ext3vs XFS?

[Wolfrider]

--I've been using reiserfs ever since it was a Suse install option (6.x?) and no problems.

Benchmarks

[Space cowboy]

Any up-to-date comparisons between the 3 main journalling filesystems (ext3, xfs, reiserfs), for both speed and reliability ?

I like xfs on the SGI - it's never let me down yet. I have to admit I'll be sorely tempted to try out xfs now that it's passed the 'seal-of-approval' and made it into the kernel - surely the best benchmark of all :-)

Simon

Re:Benchmarks

[martinde]

Don't forget IBM's JFS, it's in 2.4 AFAIK, and the last time that there were benchmarks linked from slashdot, it actually seemed the best overall, even over the highly anticipated reiser4.

Re:Benchmarks

[AlxRogan]

http://www.newsforge.com/os/03/10/07/196222.shtml? tid=2

Re:Benchmarks

[Space cowboy]

Cheers guys :-)

Simon

Re:Benchmarks

[Thaddeus]

Unfortunately, JFS requires you to fsck an unclean filesystem and the process is quite slow, whereas XFS replays the journal automatically. Also JFS has no quota support.

Re:Benchmarks

[noselasd]

No it doesn't. replaying the jfs journal(initiated by fsck) is just as fast as xfs.

Re:Benchmarks

[Thaddeus]

Maybe on a small filesystem... on LBD's, the difference is quite noticable.

Re:Benchmarks

[minion]

Don't forget IBM's JFS, it's in 2.4 AFAIK, and the last time that there were benchmarks linked from slashdot, it actually seemed the best overall, even over the highly anticipated reiser4.

Best overall is still not best in most cases. =)

JFS has been shown on several benchmarks to be VERY slow when dealing with large amounts of files in one directory.

Re:Benchmarks

[Junta]

I actually think this is a more correct behavior. In the event of an unclean shutdown, the filesystem is dirty and refuses to mount rw, and only mounts ro. When you mount ro, under ext3, xfs, and reiserfs, it seems that all of them replay journal data. Note that intuitively this seems wrong, you are mounting a partition read-only, it is reasonable to expect that bytes on the disk will not be touched. Replaying the journal log on read-only mounts is incorrect behavior from this look.

Mounting rw, while it is unreasonable to expect no modifiactions to happen at that point, it seems odd that the mere action of mounting an fs results in significant changes to the data on disk. JFS implementation of putting the responsibility of journal replay into fsck seems reasonable. And I have not noticed any performance issues with that journal replay versus XFS (which, in my experience on a 360 GB volume, takes forever to complete Journal replay, and it feels more painful because it occurs after a mount and provides no visual feedback as to how things are going, if there are problems and such. I always assume the journal is being replayed, but it could be the case that IO errors are happening, etc.) This strategy allows first the kernel driver not to be bloated by FS consistency restoration code that will only rarely be used on unclean mounts at mount time, and allows a more robust implementation be implemented in fsck. Feedback on replay progress, and in instances where Journal is corrupt, smoothly transitions to a full FS check, and in XFS when the Journal replay fails, it gets real messy real fast.

On the other hand, JFS has annoyed be significantly in that the Journalling frequently fails to protect against having an invalid FS structure. I frequently have to drop into a forced full fsck for the FS to realize that a FS entry is corrupt. My lost+found directory is filled with all kinds of stuff I have yet to identify. XFS has performed well in terms of consistant FS structure, but has lost data for me.

Reiser, btw, was way too much of a pain in the ass. Tried, and on failure, sometimes things would break to the point of a --rebuild-tree, which is only allowed on a completely unmounted volume, which means a simple task like fsck your root volume requires a rescue disk boot.

Ext3 has been slow to me, but pretty reliable. I use mainly JFS and Ext3 now, but the apparent issues with Journalling not adequately protecting FS consistency are making me want to simply go Ext3 all the way.

Stability has been there for a long time.

[Anonymous Coward]

Let the stability vs. new-features flamewar begin.

It's already been stable for years, since VERY early in the 2.4.x cycle. It's just a detail in the naming that makes it merged as part of 2.4.x itself.

Careful with LILO

[slashnik]

Be careful those of you who still use lilo

Q: Does LILO work with XFS?
This depens on where you install LILO. For MBR installation: Yes. For root partitions: No, because the XFS superblock goes where LILO would be installed. This is to maintain compatibility with the Irix on-disk format. This will not be changed. Putting the Superblock on the swap partition is reported to work but not guaranteed.

Re:Careful with LILO

[mark_lybarger]

LILO? dude, that's like, so 199^N^N^N^ er, debianish. that bacon's done moved over ages ago.

Re:Careful with LILO

[big_groo]

Doesn't everyone keep a boot disk or 2 lying around?

That simple exercise has saved me a couple of times.

Re:Careful with LILO

[Viol8]

And which of the crappy bug ridden alternatives would you suggest? Lilo is small, compact and does what it says on the tin. All I want is
a loader to boot linux & BSD, I'm not looking for a mini OS to fuck around with 101 pointless options that are nothing more than someones wet dream.

Re:Careful with LILO

[Blob Pet]

does the same apply with grub?

Re:Careful with LILO

[mst76]

LILO? dude, that's like, so 199^N^N^N^ er, debianish. that bacon's done moved over ages ago.

What else would you use? According to their website, GRUB is not quite ready for public consumption yet.

Re:Careful with LILO

[slashnik]

Grub's OK

Q: Does GRUB work with XFS?
Yes there is native XFS filesystem support for GRUB starting with version 0.91 and up. There is a GRUB rpm that supports XFS in the download section for the 1.0.2 installer on the FTP sites.

Re:Careful with LILO

[Jeffrey Baker]

I guess you've never run into LILO's "timestamp mismatch" error, which is undocumented and has nothing to do with timestamps. It prevents machines with large numbers of SCSI devices from booting. This is also precisely the market XFS serves.

GRUB is good. Boots anything. Wish we had OF.

Re:Careful with LILO

[Anonym0us Cow Herd]

I was not aware that the alternatives were bug ridden. I use Grub because it was what SuSE 8.2 wanted to run by default. My first reaction was "Oh God, why did they have to change what I was already familiar with?". But after reading up on Grub, I would never want to go back to LILO. Grub is extremely convenient. What you can do at boot time is more powerful. Changing kernels or initrd images is much easier. You can just rename kernel files so that the new kernel has the name of your old kernel. Grub goes by the filename, because it has a mini-filesystem driver for several filesystems.

You setup a smallish /boot partition in any of the supported filesystems, and put the kernel and initrd into it.

I can think of a number of enhancements for modern bloat loaders....

• Screensaver while idle at boot screen
• games while booting up
• support for latest 3D chipsets and audio drivers for the eye/ear candy while booting

...of course the end result of these enhancements will mean that we'll have to make Linux more like Windows so that people will see the boot loader more often.

Re:Careful with LILO

[Dan-DAFC]

And Mandrake I believe, though it does give you a choice.

Re:Careful with LILO

[Anonym0us Cow Herd]

Initrd is something for the good new days, not good old days. The idea is that during boot, the boot loader load two things: (1) kernel, and (2) initrd. The initrd is a ramdisk image of a smallish filesystem. The kernel is started with the smallish filesystem. The kernel runs a program from this filesystem prior to completing the boot process and running /sbin/init. The program that gets run prior to boot completion could do anything you needed to do prior to starting /sbin/init. Some useful ideas...

• Load kernel modules for additional hardware that was not compiled into the kernel. This allows you to use a more generic stripped down kernel and focus on customizing the initrd which is much more flexible.
• Load kernel modules for filesystems not compiled into the kernel. In fact, you could compile a kernel with none or only one filesystem. This leads to fewer generic kernels, and you focus on editing the initrd script to load which modules you need.

In fact, in past versions of SuSE, the entire installation process was actually run from initrd! You boot from the CD ROM. The kernel loads, mounts the initrd ramdisk. Runs the initrd program from ramdisk. This is an extremely large, complex program, that not only loads the additional kernel modules needed, but then goes through a GUI installation process (frambeuffer device, no X) and then after the hard disk has been partitioned, filesystems written, and jillions or maybe even gazillions of files copied from multiple cd's into those newly created filesystems; then the initrd program ends, and the kernel "finishes" booting into the hard disk system that did not even exist when you started the kerenel booting up.

You could conceivably have a server running for several years whose kernel was first loaded from the CD during installation onto an empty hard disk. Compare to number of reboots during Windows install.

Re:Careful with LILO

[Tux2000]

Since the old days when there was a 1024 cylinder limit for the ROM BIOS and so for LILO, I usually have a very small partition (10 to 50 MBytes) somewhere in the first 1024 cylinders mounted at /boot. LILO is installed in that partition's super block, and the partition is marked active.

So if I would use XFS for / and other partitions, I still would keep that trusted little helper /boot partition with an ext2/ext3/minix/whatever filesystem that works with LILO.

Minor drawback: You have to uncomment export INSTALL_PATH=/boot in /usr/src/linux/Makefile or vmlinuz will be installed to the / partition.

Tux2000

Re:Careful with LILO

[Vlad_the_Inhaler]

try: man initrd
for a start. SuSE (for example) have the support for all filesystems except ext2 in as modules. They use the initrd mechanism to load the rest of the stuff. This is not an approach I like - it means one more thing that can go horribly wrong if you compile your own kernel - but that is another matter.

Re:Careful with LILO

[Viol8]

How can you create an initrd one yourself and get the kernel/lilo to load it?

Re:Careful with LILO

[SnowZero]

You can also make /boot a symlink to the mini-partition. This is what I do; I like the idea of the kernel image being on a separate partition so it doesn't really matter what I do with my root FS.

Re:Careful with LILO

[akedia]

Make sure you enable "Initial ramdisk support" in your kernel config and once you have your bzImage built and installed in /boot and your kernel modules built and installed, you can then

mkinitrd -o /boot/initrd.img /lib/modules/[kernel-version]

That will build a nice ramdisk for your kernel which will preload any necessary modules. Then you just need to add a line to /etc/lilo.conf under your kernel section:

initrd=/boot/initrd.img

And of course reinstall lilo to the MBR by running /sbin/lilo.

Re:Careful with LILO

[Anonym0us Cow Herd]

And of course reinstall lilo to the MBR by running /sbin/lilo.

Or not, if you are running Grub, which will find those files by their filename and path. You can reconfigure Grub by just editing a text file. Grub will find its config text file, by pathname, and obey it. Really convenient if you really do switch kernels or manage several of them.

Furthermore, you could use the same /boot partition under several different ?NIX like OSes, and conveniently edit your grub config.

An Overview

[Gudlyf]

SGI has an overview on the XFS filesystem, just briefly pointing out some highlights. I also recall reading somewhere that it was possible (moreso than ext* filesystems) to undelete files on an XFS filesystem, although I'm skeptical.

Re:An Overview

[Fruit]

No. That is wrong. It's usually *harder* to retrieve a deleted file from an XFS filesystem than from ext2/ext3, not in the first place because the on-disk structures are more complicated.

Re:An Overview

[Zork the Almighty]

Until the SEC investigates, unearthing evidence of fraud which was supposed to have been deleted.

Comparison

[Alphix]

For all those that are looking for a filesystem comparison, I found this story to be quite interesting...or go here for the test details and results.

XFS Rocks

[fmlug.org]

I use XFS on serveral different servers, mainly because I belive it performs better then ext3, or any other fs. Also because Alot of the servers I run are samba servers and the ACL support is built native into XFS. And last I looked ACL support was still not quite stable in ext2/3 it has been awhile so it could be stable by now.

Re:XFS Rocks

[oracleofbargth]

And last I looked ACL support was still not quite stable in ext2/3 it has been awhile so it could be stable by now.

As of the patch for kernels 2.4.19+, acl support is very stable for ext[23]. In fact, I've been using it in production for over 2 years now. (I did help write some of the ext3-xattr+acl code, though, so maybe that means I'm a little bit more trusting of the code.)

The only big issues I've ever had is when using them in conjunction with quotas, but even when stress testing the filesystem, I haven't been able to coerce the code to crash since the 2.4.19 patch.

There is a performance hit to NFS when using the ACL-over-NFS code, but you never see it unless you're reading or writing the ACLs themselves. (ie. only when you're using `star --acl` to backup or restore files with their acls, or using a recursive (get|set)facl on an NFS mount.) Of course, the nfs code is less tested, so if you try it and have problems, please submit bugs to acl-devel@bestbits.at

NTFS not GPL, FAT not free

[James McP]

SGI released XFS into "the wild" and has ensured its longevity with little to no support on their part and increased the number of "out of box" coders they can hire to work on FS projects.

Microsoft....hasn't. Heck, MS is preparing to charge media makers (CF, SM, MMC, etc) to use FAT.

I say media makers switch to using XFS or another GPL'd journaling file systems. Won't take long for other platforms to support it in bulk (make/ config.....) and for stuff like flash where corruptions can occur often, I'd like a bit of journaling to minimize the impact.

Re:NTFS not GPL, FAT not free

[AllUsernamesAreGone]

The problem with using jouranling filesystems on removable media like CF is that it takes more processor power and space than normal filesystems (for example, resiserfs partitions allocate 30Mb for journal space by default). That journal always takes some space, and the less space you allocate, the less effective the journal is for handling problems.

ext2 on its own though....

Re:NTFS not GPL, FAT not free

[Merlin42]

Actually IMHO journalling on flash would be a bad idea. Most flash memories give you only about 100k write cycles before giving up the ghost. For mp3 players or digicams this is just fine. But, the point of the journal is that it is flushed to disk immediately on a write operation, so depending on usage you could wear out the memory cells that contain the journal file an order of magnitude faster, killing your flash memory REAL FAST.

Re:NTFS not GPL, FAT not free

[Fruit]

Journalling helps against sudden crashes, not against media corruption. Soft updates might be a much better option.

Re:NTFS not GPL, FAT not free

[Anonymous Coward]

Actually, this is not quite true. Most modern flash file systems are built upon a wear-leveling structure, so that rewrites to a particular sector are remapped uniformly over the remaining freespace. This prevents a single location in the flash from receiving too many rewrites. In practice, this makes the device last virtually forever. (Though knowing the wear-leveling pattern, you could probably force the worst case, in practice, this will not occur.)

Re:NTFS not GPL, FAT not free

[Fembot]

Microsoft ARENT preparting to charge CF/SM/MMC mfrs etc. (Though that is a pleseant side effect for them) They're preparing a war of patents and litigation (An "IP" war if you will) against Linux. Remeber who is funding a large part of SCO's FUD campagin right now, so if Microsoft can "show" the media that linux is built on IP quicksand they can nip linux desktops in the bud nicely and possibly make a buck or two in the process.

Re:NTFS not GPL, FAT not free

[ozzee]

JFFS addresses the flash write frequency concern. It would be good if someone was to create a tiny JFFS auto loader that would load off the flash filesystem into windows automagically. That way you can make it seamless.

This is not an endorsement of JFFS, it's just an example of a flash friendly journalling filesystem. (I have not used it - it may be the best filesystem ever, I don't know).

Re:NTFS not GPL, FAT not free

[updog]

This is not a problem for a well designed journaling flash file sytem, such as JFFS2. It writes the journal along with the data in a circular manner across the flash device (i.e., levelling friendly), so it's not going to kill your flash memory any faster (except for a tiny bit of overhead for the journal).

Re:NTFS not GPL, FAT not free

[repetty]

"JFFS addresses the flash write frequency concern."

Bad web site. It's almost like they don't want anyone to try out their work. Another example of why developers should stick with developing and get their little nephews to put up their web sites for them.

Re:NTFS not GPL, FAT not free

[modecx]

Really, we aught to be using special file systems for flash drives anyway. FAT dosen't make sense. It was designed in an era where seek times were slow, so it sought to minimize seeking...

There are free file systems out there that are better optomized for memory based storage. This is especilly something important to consider for flash memory, because it can wear out eventually. A smart file system that distributed data across the entire memory space (so wear would occur more evenly) would be much better. Seek times are not a worry.

The problem is: everything out there (cameras, MP3 players, operating systems, etc) already use FAT almost exclusively.

It would take an incredible push from both manufactures and operating system developers to change this, and end it's going to be the customer with legacy cameras and so-fouth that would be screwed... And that's the reason it's probably never going to happen.

Re:NTFS not GPL, FAT not free

[swillden]

Bad web site. It's almost like they don't want anyone to try out their work. Another example of why developers should stick with developing and get their little nephews to put up their web sites for them.

Right, because a web site without a lot of nice colors and pictures might scare away my Grandma when she's looking for a new file system for the embedded Linux distro she's building. I mean, she really sees value in a file system that is compressed and offers near-optimal wear leveling while minimizing page churn, but all that black and white text makes her nervous.

Re:NTFS not GPL, FAT not free

[AndyCap]

Regular filesystems on flash memory is probably not a sterling idea due to a limited number of writecycles available. I'm not sure of the merits of using FAT on flash memory other than the fact that it's widely supported (Maybe not for much longer. :-P). Still there is a filesystem available called jffs2 that is designed for flash. Support for reading jffs2 on the other desktop platforms is probably unlikely with GPL phobia and all :)

Re:I say what....?

[irc.goatse.cx troll]

Faster, More trusted (SGI's been using it for how many years now?), not sure how it compares cpu-wise.
The main thing that keeps me on ext3 is ext2 backwards compatability. You dont have to worry about having custom repair/bootdisks to recognize your install, and its easier to do stuff like mount under windows (great for dual booting)

I love XFS

[Apreche]

Back in the day I would always just use ext2. Then I realized, hey there are other filesystems to choose from, so I decided to shoot in the dark and try reiser and xfs and ext3. xfs has always been the most awesome.

Just one thing. Now that we've got the source code for dealing with xfs, can someone write a driver so I can mount my xfs partitions from windows xp? It would really help out a lot of us dual booting types. I would do it myself, but I don't know jack about how filesystems work. I just know which ones do what.

I hope they put the xfs into 2.6 also. Maybe it wont be necessary to have seperate xfs-sources in gentoo anymore and xfs will finally be included in the gentoo-sources.

Re:I love XFS

[Anonymous Coward]

The reason we see so few file system drivers for windows is that it's blody hard :( On Linux, adding a filesystem to the kernel is really easy, the API is clear and it doesn't touch other parts of the system. With Windows its a blody mess. And yes, I've tried to write an ext3 driver for W2K. Now I'm not knocking MS specifically, but it clearly shows that they never thought anybody would like to do somethign this.

Re:I love XFS

[kill-1]

XFS has been in 2.6 for a long time. It was merged early during the 2.5 development cycle.

Re:I love XFS

[glwtta]

Maybe it wont be necessary to have seperate xfs-sources in gentoo anymore and xfs will finally be included in the gentoo-sources.

Well, yeah, that's pretty much the gist of merging features into the kernel - distro maintainers don't have to patch them separately.

Re:I love XFS

[pantherace]

Obviously haven't seen the patchset for gentoo-sources (if you don't want patches, in gentoo look at vanilla-sources (2.4) or development-sources(2.6))

I was going to paste a selection, but given that it seems to be ~250patches, I will just note that it's a LOT of patches :)

Re:I love XFS

[smcv]

And yes, I've tried to write an ext3 driver for W2K. Now I'm not knocking MS specifically, but it clearly shows that they never thought anybody would like to do somethign this.

If you were knocking MS specifically, depending on your level of conspiracy theory, you might even think it shows the opposite :-)

Re:I love XFS

[Vlad_the_Inhaler]

?? - I thought that there was a product available for windows which allowed you to access ext2 filesystems, although it may not be free. If an ext3 partition was correctly unmounted, it can be treated behave exactly like ext2 - just leave the journal alone.

Re:I love XFS

[LearnToSpell]

You mean Explore2fs?

Yes, it reads ext3. And yes, it's free.

Re:I love XFS

[DrPascal]

That is true. Explore2fs can read ext[23] great. However, it doesn't assign it a drive letter or anything, its just a standalone app. (A nice, standalone app). :-D

Re:I love XFS

[red floyd]

The reason we see so few file system drivers for windows is that it's blody hard :(

Not to mention fscking expensive. The regular DDK is free (for NT/2K) or cost of media (for the XP/2K3 DDKs), but the IFS Kit is $1000 US. And not well documented, from what I understand.

If anyone is going to try to write these for Windows, may I recommmend you check out OSR?

Disclaimer: No relation with OSR except as a satisfied client of their driver class.

Re:I love XFS

[Vlad_the_Inhaler]

After having posted that (and I had to head off in a hurry afterwards), I thought about how I would implement something like that. Disclaimer - I program on mainframes, not PCs.

• get hold of the highest 2.0.x kernel
  the 2.0 series had comparatively few interfaces to kernel structures and ext2 was already rock-solid by then
• pick up the ext2fs coding from there, treat it as a module
• bolt on the windows front-end (this is very simplistic!!). If you go for ro access then it will be as root, rw means making decisions on ownership
• the result will probably have to be GPLed because it leans heavily on GPL source.
• implementing ext3fs (only makes sense for rw access) would also theoretically possible along those lines, although I can't remember it ever having ported to 2.0.

That whole list is probably total bowlocks, but that would be my strategy for it.

its about time

[Buckwheatz_tm]

I have been using it for for almost 3 years now. It has never let me down unlike some of the other journaling filesystems. No corrupt superblock like jfs. Kinda slow for small files when the parttions reach around 80% full. Plus its sure to piss SCO off :) but for video editing or files over 100kb it can't be beat. Reiserfs is great for directories like /var since 2.4.18 but I have lost too much data in the past to the bugger.

Re:its about time

[Zeelan]

Aye... I'm sure that it will piss SCO and even more so then normal sense Christoph Hellwig was once a SCO employee. He did a ton of work on linux back when he was working at Caldara before and after they got their paws on the old UNIX tree.

He was a big pusher of putting JFS on linux.... that is probable why SGI got ahold of him.

Re:its about time

[WhiteDragon]

I like it because it is incredibly fast at dealing with directories, ie a du on a huge system only takes a fraction of a second, and rm -r on a directory with hundreds of thousands of files is nearly instant. I work with a lot of SGI machines (including SGI 1100 PCs running linux patched to include XFS) and it is quite nice.

Why so much fuss over JFS?

[Anonymous Coward]

Just wondering, why does everyone get so excited about journaling filesystems? Many distros default to ext3/reiserfs now for even home boxes, but it's like a big band-aid.

If your box is crashing enough to make fscking a chore, you already have bigger problems. Sure, I can see where JFSs are sometimes useful, but on dekstops and most other machines the better-performing ext2 is a much more appropriate choice.

Re: Why so much fuss over JFS?

[Pike65]

I never find fscking a chore!

Quite the opposite in fact . . .

. . . we're talking about different things here, aren't we?

Re:Why so much fuss over JFS?

[larien]

It's not always crashing that's the problem, power failures are another (UPSs can cover that, but...). Also, component failures can often result in a crash.

In any case, when you have 500GB of filesystems, you really don't want to fsck that if a CPU fails.

Re:Why so much fuss over JFS?

[virtual_mps]

Well, one of your mistakes is assuming that the non-journalling fs will be faster. XFS will wipe the floor with ext2 on certain workloads. The other is assuming that it takes a number of crashes to make fscking a problem. A single fsck on a large filesystem could take upwards of an hour.

Re:Why so much fuss over JFS?

[Lumpy]

It's based on the value of your data.

if you are running a single IDE drive, then your data is not worth much. SCSI = worth a bit more... SCSI raid 5 with 4 drives and a journaling filesystem = worth alot more.

then we get to the level of hotplug scsi drives with tons of redundancy hardware raid 5 (works awesome with linux) and a real tape backup system (DLT) means your data is actually valuable....

and we can go up from there to extreme redundancy and data safety.

JFS = cheap security for valuable data... but remember you do get what you pay for.

Re:Why so much fuss over JFS?

[Nothinman]

In this case journaling is only one of the main features of the filesystem. XFS is much faster than ext3, supports logging to a seperate device to speed it up even more, nicely supports huge filesystems and file sizes and has much nicer userland tools.

Re:Why so much fuss over JFS?

[GooberToo]

I remember about 7 years ago, I was working ona project that had a VAX cluster for our Sybase DB. It crashed. He had to wait almost 4 hours for it to finish fscking the disks.

Re:Why so much fuss over JFS?

[drinkypoo]

I agree wholeheartedly. It seems like every other time my power went out I lost data on ext2. I've had many bizarre failures (including some fun kernel panics, and plenty of cord-pulled-out-of-wall debacles) while using XFS and never lost anything. Journaling is a must, EVERYONE should be using journaling.

Want another anecdotal data point about journaling filesystems? Windows NT from 5.x has a Journaling NTFS. (I've heard people say that NTFS has ALWAYS had some form of journaling, and that it wasn't turned on by default, but anyway.) I haven't lost any data on my NT boxes since NT5 due to power failure or mysterious crashes. (I've lost it due to shaky drivers, though. I shouldn't have had to tell myself this, and I know I don't have to tell you this, but I will anyway - avoid Silicon Image's Medley Software RAID. Their drivers are craptacular.)

Re:Why so much fuss over JFS?

[DavidTC]

And even then, you can't guarantee that the file saved is the newest version, the version you want. All you can do is guarantee that it's not half a file and some random gibberish.

Re:Why so much fuss over JFS?

[Lumpy]

How? Duh it's really simple....

you buy a $99.00 IDE drive for your data? It's not worth much to you.

you spend $4000.00 on raid hardware for your data and another $2000.00 for your backup? now it's worth much more than the silly man using the IDE drive.

if your data is valuable to you thne you spend the money to protect that data.

if your data is worth $100,000.00US only a complete fool will save it on a $99.00US IDE drive with no backup.

do you understand now? This is why big companies use real server hardware instead of having the corner PC shop build a gamer PC for them.

Re:Why so much fuss over JFS?

[MikeCapone]

if your data is valuable to you thne you spend the money to protect that data.

Here come the capitalists!

The value of data is not only about how much money it's worth, you know. It's not because someone doesn't have big bucks to spend on something that it's worthless in the grand scheme of things.

Or maybe you keep your family photo albums and personal diary in that freaking big vault in your basement?

Re:Why so much fuss over JFS?

[virtual_mps]

I often find AC's posting about a purportedly definitive benchmark without providing any link to be highly credible. Not this time, though.

Re:Why so much fuss over JFS?

[Samrobb]

The value of data is not only about how much money it's worth, you know.

He wasn't talking about how much money the data was worth, in and of itself - he was talking about how much you were willing to spend to protect and preserve your data. Two entirely different things, really. An example: my Grandfather's old 8mm video is almost certainly worthless out in the marketplace, but we were willing to spend a good chunk of change to have it preserved and transfered to VHS a few years back.

And please - before you go saying "Ah-HA! Those 8mm tapes must have had value to you, because you spent so much to preserve them!", think about that for a second. The only reason you know they have some value to me is because I told you that I spent some effort (time, $$$, materials) to preserve them. If I told you that I left them lying in a puddle in the basement for 20 years, you could quite reasonably conclude that I didn't care what happened to them.

I think that is the point the original poster was trying to make: the higher the perceived value of data, the more you are willing to spend in order to archive it.

Re:Why so much fuss over JFS?

[MikeCapone]

His point seemed to be that normal people shouldn't use journaling file systems because if their data had value they'd spend money on hardware to protect it.

My point is that things can have value without having high monetary value, and so using something free such as a journaling file system makes sense.

Of course spending the big bucks on RAID5 and SCSI would be better - and should be used for things with high commercial value, for sure - but for other things that still have value but where it's hard to justifify an increase in budget using free solutions such as a better filesystem or floppy disk backups or whatever can be totally justified, in opposition to the original poster whole said that nobody with single IDE drives needed anything else than EXT2 (IIRC).

We've just begun

[fserb]

I agree with Marcelo's action on this. We're still in a very early stage of the 2.6 (stable) branch to feature freeze 2.4.

I know we need the maximum user base for 2.6 testing, debugging and to recieve those "My TV stopped working when I installed kernel 2.6" messages. But we have to take it easy.

2.6 rocks. And a lot of distros have plans to release 2.6 based releases in the first quarter of 2004, which will greatly improve the user base.

IMHO, a good feature freeze, as Marcelo said somewhere in LKML, is 2.4.24 or even 2.4.25.

It's no time for a flamewar to begin. The Beaver is in the building. :)

I wonder

[AndyFewt]

.. what SCO will have to say about this...

Re:I wonder

[sharkey]

"I sue you!"

Re:I wonder

[DeathPenguin]

What's more important is what SGI has to say about this:

"October 1, 2003

To the Linux Community:

As one of many contributors to the Open Source movement and to Linux,
SGI takes the subject of intellectual property rights seriously. Our
contributions are a valuable expression of ideas which contribute to
the intellectual richness of Linux.

Over the past four years, SGI has released over a million lines of code
under an open source license. Throughout, we have carried out a
rigorous internal process to ensure that all software contributed by
SGI represents code we are legally entitled to release as open source.

When a question was raised by the community earlier in the summer about
the ate_utils.c routine, we took immediate action to address it. We
quickly and carefully re-reviewed our contributions to open source, and
found brief fragments of code matching System V code in three generic
routines (ate_utils.c, the atoi function and systeminfo.h header file),
all within the I/O infrastructure support for SGI's platform. The three
code fragments had been inadvertently included and in fact were
redundant from the start. We found better replacements providing the
same functionality already available in the Linux kernel. All
together, these three small code fragments comprised no more than 200
lines out of the more than one million lines of our overall
contributions to Linux. Notably, it appears that most or all of the
System V code fragments we found had previously been placed in the
public domain, meaning it is very doubtful that the SCO Group has any
proprietary claim to these code fragments in any case.

As a precaution, we promptly removed the code fragments from SGIs Linux
website and distributed customer patches, and released patches to the
2.4 and 2.5 kernels on June 30 and July 3 to replace these routines and
make other fixes to the SGI infrastructure code that were already in
progress at SGI. Our changes showed up in the 2.5 kernel within a few
weeks of our submission, and the 2.4 changes were available in the
production version of the 2.4 kernel as of August 25 when the 2.4.22
kernel was released. Thus, the code in question has been completely
removed.

Following this occurrence, we continued our investigation to determine
whether any other code in the Linux kernel was even conceivably
implicated. As a result of that exhaustive investigation, SGI has
discovered a few additional code segments (similar in nature to the
segments referred to above and trivial in amount) that may arguably be
related to UNIX code. We are in the process of removing and replacing
these segments.

SCO's references to XFS are completely misplaced. XFS is an innovative
SGI- created work. It is not a derivative work of System V in any
sense, and SGI has full rights to license it to whomever we choose and
to contribute it to open source. It may be that SCO is taking the
position that merely because XFS is also distributed along with IRIX it
is somehow subject to the System V license. But if so, this is an
absurd position, with no basis either in the license or in common
sense. In fact, our UNIX license clearly provides that SGI retains
ownership and all rights as to all code that was not part of AT&Ts UNIX
System V.

I hope this answers some of the questions that you and the Linux
community might have. We continue to release new Linux work, and are
very excited about the growth and acceptance of Linux. We are
continuing full speed to do new work and release new Linux products.
We take our responsibility to the open source community seriously and
are confident that we have an effective process to verify the quality
and integrity of our contributions to Linux.

Rich Altmaier
VP of Software, SGI
richa@sgi.com"

Source: http://oss.sgi.com/letter_100103.txt

Patch size.

[rf0]

Fun for the whole family with guess where the patch was applied From the snapshot directory

bk6 - 424K

bk7 - 964k

bk8 - 1.2M

Well thats increased the kernel by about another 5-10%. However I would say I do like xfs and its proven quite stable now.

Rus

Re:Patch size.

[kill-1]

XFS is about 650K bzip2-compressed. The kernel tarball is about 28M bzip2-compressed. So XFS increases the kernel by about 2.3%. It is a lot of code especially compared to other filesystems, but nowhere near 5-10% of the kernel.

Re:Patch size.

[opk]

You need to look closer than that. There isn't a bk8 yet and you've not spotted that there are both files with .gz and .bz2 extensions. Taking just .bz2, sizes are:

bk5: 349K
bk6: 361K
bk7: 964K

Working out when XFS went in becomes a little easier with the right figures don't you think?

As for XFS, I've used it for over a year with no problems and am glad to see it go in. Have long liked SGI stuff: I own an old Indigo and Indy.

Re:Patch size.

[verbatim_verbose]

The size increase only matters if you actually compile XFS into the kernel when you set it up. If you're not using it, you're going to get the same thing as before in the end.

Christoph Hellwig, former SCO (Caldera)!

[infolib]

after a code review by Christoph Hellwig

Incidentally, this is the Christoph Hellwig who contributed code to the kernel on Calderas behalf. His contributions may become an important point in the SCO-IBM-RedHat battle.

YEAH, WOO HOOO - ALRIGHT!

[cluge]

After patching every single kernel thats come out since the early 2.4s, I now have a kernel that I don't need to patch. WOW, about darn time!! Perhaps I'll even get lucky enough that RedHat and others that do not support XFS yet will build it into their kernels. That will make MY life easier, and updates go faster.

We chose XFS after lots of serious testing. It beat all comers at the time and we've been using it ever since. The only downside to XFS is file deletion times are a bit long, especially compared to Reiser, but when you have a server that is uner HEAVY load (Databses, mail servers) and with LARGE files (log server) nothing beats XFS.

Thanks guys, this is one of those merges that has made me estatic!

Angry People Rule

Re:YEAH, WOO HOOO - ALRIGHT!

[Fruit]

Delete times were mostly fixed in XFS 1.3, if I recall correctly.

Re:YEAH, WOO HOOO - ALRIGHT!

[Malc]

How does it compare with JFS?

Re:YEAH, WOO HOOO - ALRIGHT!

[GooberToo]

-XFS
+JFS

Looks like they are pretty close. I think the only difference is the X and J. ;)

Re:YEAH, WOO HOOO - ALRIGHT!

[halfelven]

The only downside to XFS is file deletion times are a bit long

Only on lots of small/medium files, and even then the situation got pretty much fixed in 1.3.
Deleting large files is instantaneous on XFS no matter which version, while it takes a long time on any other FS.

Re:YEAH, WOO HOOO - ALRIGHT!

[WhiteDragon]

I frequently delete very large directories with about 10000 - 60000 files (each file is typically 20Kb) and am consistently amazed at how quick this is under XFS. Mind you I don't delete the individual files, but the whole directory (with rm -rf).

Re:Vendor pressure

[mark_lybarger]

yeah, they probably sent Vito down to visit Marcelo with a suite case full of jacksons and a simple message to deliver. "Marc... look we've got this fine filesystem here. you've seen the code. its nice code. but, you've got our competitor filesystem code in your kernel. now before we get our brawls all in an up roar, all we're asking is you HIGHLY consider putting this here XFS filesystem into that there kernel you're maintinging. Now, be a good boy, Marc, I know you'll do the right thing"


me i always hate seeing commercial quality products ending up on free software. the disgust.

No Complaints

[LightForce3]

Mandrake has offered XFS since at least 9.0, my first Linux distro. I've been using XFS (at the suggestion of my friend who helped with the install) for at least 6 months now, with only instance of a problem (not sure if it was a fault in the filesystem itself): lost or corrupted an inode or two, and fixed very easily once I knew what to do.

It works with both GRUB and LILO, is reasonably speedy, and has enormous partition and file size limits.

Count me a happy customer.

~~LF

Re:No Complaints

[coyote1]

Mandrake has offered XFS since at least 9.0

Mandrake has had XFS earlier than that; I have an 8.2 system that uses it (when you've got 80Gb files, you needs something other than ext2/ext3).

Re:Vendor pressure

[fishnuts]

This "big merge" has nothing to do with vendor pressure. The XFS patches have been available and well-tested throughout most of the 2.4 kernel's life cycle and since its (XFS') stability has already been proven to play nicely with the rest of the kernel, it's quite appropriate to do a merge so late in the 2.4 tree's live cycle. The team at SGI that handles merging the XFS code into the kernel have done a very good job of keeping up with bug reports and changes in the kernel vfs code.

Marcelo probably shares my opinion in that the current XFS code has been around long enough, demonstrated stability, and successfully merged with every recent 2.4 kernel back to at least 2.4.1x, that it's more than suitable for inclusion in the main kernel source without risk of introducing instability.

The only clashes I've ever seen with XFS and other code was with other 3rd-party patches, such as the ACL support in grsecurity. Those are now "switchable", anyway.

Re:Vendor pressure

[knuffie]

That is not true. The biggest hold back during the past 3 years has been the fact that the VFS layer needed a number of alterations and so far Marcello did not merge XFS because of this.

It wasn't untill Cristoph OK'd the VFS changes that Marcello merged the XFS core.

SGI as a vendor has had nothing to do with it. Buy a altix 3000 and they would happily maintain any special patch you would need for that (IA64) machine.

I think I know what I'm talking about since it's my name on the XFS FAQ. And no I don't work for SGI.

Re:I say what....?

[mcbridematt]

well, ext3 is built over ext2, and still suffers ext2's dataloss problems.

I wouldn't even dare put ext3 on a production system. It isn't worth the advantage over ext2 (counting my own experiances)

ext3vs XFS: The article in English.

[golan]

Here is the same article, but in English.

Oddly Enough

[thesolo]

This was just mentioned here on /. the other day, but according to this article on Groklaw, Christoph Hellwig is (was?) a Caldera (SCO) employee.

SCO is going after SGI for XFS, when one of their own employees was working on it.

Re:Oddly Enough

[Vlad_the_Inhaler]

was.

Groklaw said in the article that he 'also' had 2 e-mail addresses at SGI but not that he has been working for them for several months.

Re:Oddly Enough

[haraldm]

Ohmygod. When do people learn to read history before blahing around. The Caldera development team in Erlangen was sold in its entirety to Su^HUSE when Caldera renamed itself SCO. But maybe this is news in good old America, Groklaw or not. Christoph was never a SCO employee. Duh. So stop using SCO for headlines.

Re:I say what....?

[fishnuts]

Extended ACLs, btree filesystem structures to facilitate huge files, fast sparse files, large directories, fast deletes, and a couple other niceties that would have required huge functional changes to ext2/ext3 to implement. It's also completely 64-bit clean, as it has from its conception.

The btree-based storage structure is already employed by reiserfs in a similar manner, but XFS' implementation has been stable (used in IRIX) for quite a bit longer.

Finally

[LynXmaN]

I've been using XFS in production servers for more than two years already without any problem, it was time that it was merged into 2.4 kernel...

Maybe this way RedHat begins to support it for their installations

Re:I say what....?

[lxs]

The main thing that keeps me on ext3 is ext2 backwards compatability. You dont have to worry about having custom repair/bootdisks to recognize your install...

you don't need a custom repair disk for xfs if you can boot from CD. Knoppix comes with xfs support and with the xfsrepair utility. (as I found out one day, when my debian system suddenly refused to mount the corrupted xfs partition, which also happened to be the filesystem root. Booting Knoppix allowed me to fix the problem in minutes.)

I haven't used ext3 or reiserfs yet, so I can't make a direct comparison.

Re:but NTFS

[Simon Lyngshede]

So?

Windows doesn't even have read support for reiserfs, XFS og JFS. What is your point?

Journals are for girls?

[Dynastar454]

from the journals-are-for-girls dept.

Huh? I always thought it was diaries that were for girls... at least, that's what I told my friends when they made fun of my journal. :-)

Re:Journals are for girls?

[ElliotLee]

Journals are for girls? Ah, THAT's why every Slashdot user is given a Journal...

Re:I say what....?

[glwtta]

So what's the big advantage of xfs over ext3?

Performance; scaleability (something around 9 million terabytes, if memory serves); stability - in the sense of a longer proven track-record - while ext3 is quite stable, XFS is simply a lot more mature; features, like ACLs and other small things.

XFS

[mindstrm]

I seem to recall XFS has some issue with bad blocks.. if there is a bad block in the FS, it will unmount, and cannot remount. It has no facility to scan the FS and map around bad blocks, either.

Re:XFS

[ktulu1115]

Yes, but buy then any normal computer user would go out and RMA the drive anyway (or replace otherwise if needbe) ...It's pointless for a filesystem to deal with bad blocks if it won't be running on a drive with defective media in it.

I guess it's just the attitude SGI took awhile back, but it's what I follow as a professional in the field along with my colleagues (and what makes sense) so I don't view that as a limitation of the filesystem.

Hellwig's role in all this

[einhverfr]

Iirc, isn't he a former SCO/Caldera employee who was heavily involved in developing SMP and the Linux port of JFS? Iirc, Groklaw has a thing on this.

Now he works for SGI. The question I have is this-- it seems as if he has a conflict of interest to give his employer a beneficial review of the driver in order to ensure that it is included. I wonder how independent he is in his review.

That being said, he has been an important contributor to the kernel, and so I will give him the benefit of the doubt. I just wish that some sort of third-party review would have been done.

OTOH, there has been some speculation that Marcelo was biased against the inclusion, so maybe this balances things out.

Re:Hellwig's role in all this

[MonkeyDluffy]

it seems as if he has a conflict of interest to give his employer a beneficial review of the driver in order to ensure that it is included. I wonder how independent he is in his review.

The same could be said for many other kernel developers who work for companies that can benefit from new kernel features - after all, that's one reason to pay people for kernel development. But if anyone of them would give a good review of a "not ready" driver, it could hurt their reputation (and hurt their company, too, since the kernel feature they wanted added would get a bad reputation). So there is a big incentive to do it right.

-MDL

Re:Hellwig's role in all this

[Vlad_the_Inhaler]

A 'code review' consists of going through the coding and checking it for correctness / capacity to cause damage. There is no conflict of interest here because everyone involved knows that he is wearing two hats in this business:

• An employee of SGI (and developer of XFS)
• One of the top ten contributors to the Kernel

Consider it a case of CH anointing this patch with sacred Penguin Pee, something Linus used to have to do before people got used to other people being Kernel maintainers.

Linus always was more likely to accept patches from people he knew and who had a good track record, Marcelo is doing the same thing here.

Quotas!

[mattbee]

From what I could find out XFS is the only Linux filesystem which stores quota information as meta-data-- there's no risk of an XFS filesystem getting its quotas "out of sync" with the contents of the disc and having to run a tedious quotacheck. We recently deployed it as a backup server and it's working very well!

Re: FINALLY!

[jbeamon]

I've been patching XFS support into my distro for a few months now. Let me get you caught up briefly. We had ext3 patched with acl support. That patch tends to lag behind kernel versions a little, which is not a problem if you're running a distro's standard back-ported kernel. However, I just undertook a migration on a live production box from ext3+acl to XFS on a second attached disk array, so I've been in Patch Hell for the last few weeks. I will say up front that I am not a k3rn31 h4ck0rz, but I get more done with it than probably anyone I personally know. That I got as far as I did in this story amazes me even today!

SGI ports their patch up to the latest kernel within a few days, but they have a nasty habit of removing older versions from their downloads when newer versions come out. When I only had the ext3+acl patches for kernel 2.4.20, and acl.bestbits.at was down for over a week ('grumble'), SGI only had XFS patches out for kernel 2.4.22. Andreas was kind enough to personally provide me some 2.4.22 ext3 patches. By the hardest, I got my 2.4.22 kernel built on my file server with ext3+acl and XFS.

The next DAY, I read of a root exploit in 2.4.22. The patch from kernel.org rendered my ext3+acl patches incompatible, and I'm not the type of guy yet to divvy up patches into even smaller pieces on any sort of schedule. I had to either forego backward compatibility or maintain a shell exploit in an environment where people do have shells.

I found, just yesterday, that Red Hat's newest kernel package includes xattrs and acls for ext3 and the 2.4.22 exploit's bugfix. It won't accept my xfs patch for 2.4.23 over some posix_acl and kdb conflicts. I found yesterday that SGI's latest kernel image has XFS and ext3+acl, but not the bugfix. The 2.4.23 patch broke my build. I find today that XFS is about to be added onto the native kernel tree, which just received both the bugfix and the ext3+acl extensions.

It's about TIME! :-D

Bechmarks

[kompiluj]

You can find the benchmarks on:
http://epoxy.mrs.umn.edu/~minerg/fstests/results.h tml, or a copy at: ReiserFS homepage.
Of course your mileage may vary but I generally got results consistent with those cited.
My own experiences (I have used both reiserfs and xfs with 2.4.20 kernel:

• reiserfs is a little bit faster than xfs
• xfs gives you 2 times bigger CPU usage than reiserfs
• both are still much better than jfs
• the reliability of both xfs and reiserfs is satisfactory
• the results are still order of magnitude worse than those I get with UFS2 with softupdates on FreeBSD 5.1

Re:Bechmarks

[Pow.R Toc.H]

These benchmarks have been run with Reiser 4 which, AFAIK, is not shipped by default with no Linux distribution. Even Mandrake guys, who are fond of experimental software on their distributions hasn't included Reiser 4 on Mdk9.2. Most distributions include 3.6.28, IINM.

OTOH, I've been using XFS to store and edit 36-bit film scans (40+ MB file sizes) and XFS has been serving me extremely well, without data corruption of any kind - differently from Reiser 3, which needs a reiserfsck every time I boot Win2k (not that this happens very often).

Finally, according to Oracle, even ext3 blows ReiserFS 3 off the water:

http://otn.oracle.com/oramag/webcolumns/2002/techa rticles/scalzo_linux02.html
--------
Outgoing mail is certified Virus Free - after all, I use Mandrake Linux!
Get my public PGP signature at http://www.paulo.fessel.nom.br

Re:Bechmarks

[kompiluj]

Ok, real-world performance can be VERY different from the test results, for instance mounting options are really important - I have heard about ext3 running much faster when mounted in (theoretically) slowest mode data=journal. But for applications that write to the same location many times (e.g. databases) it means that the data are kept in memory and don't get written to disk (remember - the journal resides in memory before it gets synced). The same for mail servers - like postfix. In these cases ext3 data=journal can be a good choice, however it has some limitations.
On the other hand, when comparing reiserfs and xfs you must remember that reiserfs and reiser4 are generally systems that are tuned with many little files in mind and xfs is made for big transfers. So reiserfs shines when you have many little files (e.g. my ext3 broke having to deal with about 500 thousand files in a directory tree of average depth 10 - moving them to reiserfs helped), and xfs is good when you have hardware capable of big transfers - like the SGI(tm) architectures, or at least some ultra-mega-fast-SCSI.
However, I would still stick to the following points:

• FreeBSD 5.1 with Berkeley FFS (a.k.a. ufs2 - some authors mistake ufs for s5fs - no longer in use) with soft-updates is order of magnitude better than any system on Linux 2.4 - don't know how it is with 2.6
• ext3 is the least CPU intensive
• xfs is the mostCPU intensive
• reiserfs can be nasty in a case of real crash (when you need to reiserfsck --rebuild-tree)

A few other nice XFS features

[isoga]

...no one has mentioned yet:
(from http://www.sgi.com/software/xfs/overview.html)

Guaranteed Rate I/O
XFS is the only file system available that provides a guaranteed rate I/O system, which allows applications to reserve specific bandwidth to or from the file system. The file system can determine the available bandwidth and guarantee that a requested level of performance is met for a given time. This functionality is critical for media delivery systems such as video-on-demand or data acquisition.

Expanded Dump Capabilities
Unlike traditional file systems, which must be dismounted to guarantee a consistent dump image, you can dump an XFS file system while it is being used. The XFS dump utility, XFSdump, can dump an entire filesystem, a directory tree, or specific files. XFSdump is restartable, which allows a large dump to be spread over an extended period of time or to be resumed after a system restart.

-->tech stuff

Re:A few other nice XFS features

[drinkypoo]

Guaranteed Rate I/O is not available on Linux, which is probably why no one has mentioned it. It would likely require significant modification to the scheduler (what, again?)

Re:I say what....?

[Cthefuture]

... not sure how it compares cpu-wise.

In my testing, not good. But it makes a good server filesystem.

XFS seems to steal cycles from deep in the kernel itself therefore making it fairly fast for file operations but at the expense of everything else.

I didn't like it for a desktop system. It uses too much CPU and user responsiveness suffers.

XFS for Win32?

[CausticWindow]

I'm about to move a lot of my data over to a new disk. I want to use a filesystem that's readable (and preferably writable) by both Linux, and Windows.

So far, my best bet seems to be NTFS. But do anybody know if there exists XFS drivers for Win32?

Re:XFS for Win32?

[TheRaven64]

Your best bet would be FAT32, since that can be mounted rw on both Windows and Linux (it's even possible to boot Linux on a FAT partition, although far from sensible).

The second option would be ext2/3, since there is an installable FS driver that lets Windows NT mount ext2 partitions, although I haven't looked at the progress for a while.

The final option (the one I use) is to put your personal data on a cheap *NIX box (mine's FreeBSD box with a UFS disk) and SMB mount it for remote access under Windows and NFS or SMB mount it under *NIX.

Re:XFS for Win32?

[drinkypoo]

The final option (the one I use) is to put your personal data on a cheap *NIX box (mine's FreeBSD box with a UFS disk) and SMB mount it for remote access under Windows and NFS or SMB mount it under *NIX.

I guess you missed the article on Using the Real ntfs.sys Driver Under Linux, eh?

Re:XFS for Win32?

[lpq]

One of the XFS authors said anyone who wants to undertake such a port -- "go for it".

Considering the difficulty in ensuring data integrity and support for B-tree arranged data, Microsoft would not look kindly upon XFS being ported to NT, since their next generation OS is supposed to include database like features to speed up indexing and accessing data like XFS already has built-in. It would really rain on their parade. Also, benchmarking shows NTFS is considerably slower than XFS (or FAT32 for that matter) for large files and NTFS has no support for Real-time I/O partitions or journals being located on separate disks.

NTFS also requires (according to ad-copy) constant defragmentation due to their primitive block allocation scheme while XFS does quite well even without the XFS FSR (File System Reorganizer). XFS's FSR was created for 1 specific customer who had a particular application that generated excessively fragmented disks. Before that, an FSR (/defragmenter) wasn't considered necessary because XFS is intelligent about how it lays out files when they are written and how it stores free space (with free space also stored in ordered B-tree's by powers-of-two size of the free space blocks.

The only benchmark I've seen XFS run noticeably slower on linux, on is deleting large numbers of small files -- something one doesn't notice on IRIX, since the space deallocation happens in background on IRIX, and only the inodes need be marked deleted before the user prompt returned. I seem to remember on Linux the space had to be deallocated synchronously for some reason or another.

Makes sense given the way free space is managed -- when files are deleted, free blocks are recursively combined with adjacent free blocks to create the largest possible 'free block size' (I think up to 128k blocks, default=4k block size) (my numbers may be a bit rusty). Free space blocks were combined asynchronously, under IRIX (as I understand it), in a system thread after the last reference to an inode was released. Linux, if I remember correctly, didn't support the facilities for such a background thread -- thus the block combining happens synchronously, explaining the performance hit for file tests that delete lots of small files: there are many small free blocks that are candidates for being merged with adjacent free space.

I'm not entirely sure why a special "XFS_del" process couldn't be started at system run time who's sole purpose was taking unreferenced inodes and doing the space combining in background, allowing foreground programs to continue asynchronously after simply marking the inode as unusable and enqueing it to the XFS "free space" combining process. It is quite possible some of this has been implemented and my information is dated. But free space combining on cleanup is one of the main reasons why, historically, XFS file systems, didn't need to have _continually_ running programs like Executive Software's, _DiskKeeper_, running, full time in background: because XFS had it's own built-in defragmentation every time a user did a file-delete.

For the degenerate case -- *one* customer was not getting sufficient speed for real-time, uncompressed video recording to disk (back in the early to mid 1990's when disks were much slower). The swat team, assigned to the problem, found that the customer's particular use kept many small files around while deleting some files in a way that prevented automatic space consolidation. This odd usage was just enough to slow down direct-to-disk video recording (something quite difficult on systems in the early to mid 90's when disks were not so fast and SCSI-2 was still state of the art). To solve this problem for *one* customer, the "xfs_fsr" util was written.

To make the most of the efforts spent on the one customer, SGI incorporated xfs_fsr into the general OS to be run occasionally to stave multi-month/year buildup of possible, similar degenerate cases. I.e. XFS customers considered fragmentation such an unlikely / non-issue, that the X

SCO

[einhverfr]

Plus its sure to piss SCO off :)

That is not the half of it. You see-- Hellwig is a former SCO employee who when he worked there, worked with IBM closely on their port of JFS to Linux. He was also heavily involved in the SMP development process too. Just do a search for his name and SCO and Caldera on your favorite search engine. I think it will be hard for him to avoid a deposition ;-)

Now he works for SGI.

Re:SCO

[xcreature]

#!/usr/bin/perl

use LWP::UserAgent;
use HTTP::Request;

my $useragent = new LWP::UserAgent();
$useragent->agent("New SCO Suit");

my $request = HTTP::Request->new(GET => "http://www.caldera.com/scosource/complaint3.06.03 .html");

my $complaint = $useragent->request($request);

$complaint =~ s/IBM/SGI/g;
$complaint =~ s/AIX/IRIX/g;

print $complaint;

Re:SCO

[haraldm]

Dummes Zeug. Christoph was never a SCO employee. He belonged to Caldera's Erlangen development team that was sold to SUSE. SUSE by the way has been offering XFS in their distros for ages now. So really folks: THERE IS NO NEWS HERE!. It's just that long existing and provenly stable SuSE patches are integrated into the stock 2.4 tree. Duh.

No.

[Booker]

Christoph was not working on XFS while he was employed by Caldera.

No quotacheck

[Booker]

XFS journals quota info, so you won't have to
wait for a quotacheck on a huge filesystem.

Hmm..

[NegativeK]

I happened to stumble on the thread while browsing the lkml a while ago.. I noticed that someone from Fermilab tossed in their support for merging it, as they run a 300TB or so setup. You think, maybe, this has something to do with it?

On a different note, I've been running XFS on my 2.6.0-test box for a while.. Now that it's going to be in a stable kernel, I can't wait to back up everything and switch. =D

Can't XFS also dynamically create more inodes?

[emil]

AFAIK, ext3 can't, so when you run out, you get to backup, recreate the filesystem with more, then restore.

Re:Can't XFS also dynamically create more inodes?

[DavidTC]

parted can resize ext3 now.

Even the old version, without ext3 support, can resize it, because they can resize ext2, and it's trivially easy to turn a cleanly (And if you didn't cleanly unmount it you really shouldn't be resizing it.) unmounted ext3 filesystem into ext2, resize it, and turn it back. In fact, I suspect that's what the new versions of parted do, they just do it automatically.

Re:Can't XFS also dynamically create more inodes?

[WhiteDragon]

all three of ext3, xfs, and reiserfs support growing live filesystems while still mounted, just mount with the "-o remount,resize=12345678" option. ext3 and reiserfs can shrink unmounted filesystems, but I don't think xfs or jfs can. I also don't know if jfs can grow a live mounted filesystem.

Re:but NTFS

[JohnFluxx]

That's certaintly very interesting, but to run it requires a copy of windows NT. Makes it a bit hard for a rescue disk.

no GRIO on Linux

[Booker]

GRIO is not available on Linux, because it requires a lot of other support in the kernel proper, in the various I/O subsystems etc.

however, the realtime subvolume, which is a component of GRIO, is available for use on Linux.

How will that help?

[mindstrm]

MS licensing the use of FAT is for flash media devices.. like all the digital cameras and PDAs out there... they use FAT because everything recognizes it. Using XFS would be pointless, as only linux and irix would be able to read it. Further, for flash, xfs would be rediculous anyway.. there are better systems specifically designed for flash.

If you don't use a format windows recognizes... there is no point.

How to securely delete journaled data?

[Anonymous Coward]

When using the shred utility, I get warnings about not using the utility for journaled filesystems.

So what utility can I use to securely delete data on a journaled file system?

And being a previous windows user, I really liked the BCWipe utility that securely wiped unused areas of a partition. Is there an equivalent in linux systems?

What happened to sync?

[jhines]

I remember way back in the v7 days, there was a command called sync, which wouldn't return, until all data had been flushed to disk.

It would seem that his would take care of that problem, at least on a non-active system, which is what you'd want for a backup anyway.

Re:What happened to sync?

[Nothinman]

sync still exists, but you can't be guaranteed that nothing will get modified in memory while the dump is in progress unless you unmount or remount the filesystem read-only. Or use something like an LVM snapshot like was mentioned by another poster.

What exactly do they mean when they say.....

[lobsterGun]

It is now also available under GPL for linux.

as near as I can figure it the above either means

They are trying to restrict the GPL version of XFS to Linux.

or

XFS has been released under the GPL and a linux implementation is available.

I'm not entirely sure. Can someone clarify this for me please?

Re:What exactly do they mean when they say.....

[DeathPenguin]

>>They are trying to restrict the GPL version of XFS to Linux.

It means what you said the second time: "XFS has been released under the GPL and a linux implementation is available."

It was released a long time ago, too. I've been using XFS since before Mandrake started supporting it in version 8 (With an early 2.4 kernel). So "has been released" is a reference to an event that occured a couple years ago.

Re:but NTFS

[Ianoo]

Not really, since you can get ntfs.sys off the NTFS drive using the Linux-NTFS ReadOnly driver. It's quite likely that a system will have ntfs.sys somewhere on it if it contains NTFS formatted drives!

Re:So much for "no major changes"

[fishnuts]

XFS would help 12,000,000 people if those people used XFS. I'm sure 12,000,000 people would agree that having a more reliable filesystem that's less likely to be corrupted or lose data during a power failure or emergency-laptop-shutting, and takes no more than 5 seconds to run consistency checks during startup, regardless of filesystem size, is more important than something that improves the responsiveness of the system in certain high-load situations.

Now, do your part and advise 12,000,000 linux users that there are low-latency preemptive kernel patches and a filesystem more robust/stable/reliable than ext2/ext3. Let them decide, instead of assuming that everyone in the world is like you.

Re:Changes to the stable kernel?

[fishnuts]

Now that the VFS layer has been stabilized and supportive of such FS drivers as XFS, very little needs to be changed to add XFS support. It's almost completely "additive", rather than modifying the existing code.

Re:So much for "no major changes"

[MikeCapone]

You make this sound as if we couldn't have both.

Of course any new feature should only be added if it's stable and doesn't affect the rest of the kernel when it's not used (for all the people who cry everytime a line of code is added to the stable kernel)...

Anyway, personally I'm waiting for 2.6 with great anticipation. I've had some problems with my USB mouse in 2.6-test11. Hopefully Slackware 9.2 will ship with it and it'll be better set than what I could do with my modest h4x0r skills...

Re: FINALLY!

[cluge]

When we first tested JFS wasn't really ready. We couldn't get it to work on the boot partition and had some bad oopses under heavy load. I understand that these have been fixed now. As far as straight testing is concerned they were pretty close. When the DB files got really big (1.3 GB or larger) XFS started to outshine JFS.

Angry People Rule

XFS = more CPU usage (but how much more?)

[MikeCapone]

I've read in a few places that XFS takes a bit more of your CPU than some of the other popular file-systems (such as reiser).

I was considering using XFS on my desktop, but since my CPU is getting old (K6-2 450mhz), I'm now hesitating; does it really make a noticeable difference?

Right now I'm using reiserFS; lets say that it's 100%. How much more would XFS take? 105%? 115%?

Thanks,

Re:I say what....?

[aminorex]

Against your comments, there are many of us
who have managed lans of ext3 systems in
*ahem* differently-powered environments and
found that it is, in its current state of
maturity, bullet-proof.

Compared to the relatively few users of
SGI machines, I think the RedHat userbase
constitutes a much more reliable statistical
base.

XFS in production

[lloydy]

We've been using XFS in production from the start.

We're a film and TV VFX / animation company, and have XFS on everything - about 600 machines, from large servers to workstations and render machines. It's absolutely core to our business. We came to XFS from using SGI machines, but it's nearly all linux now.

As far as I know, many of the other studios are doing the same.

Another thing that we've noticed recently is the proliferation of black box NAS servers, which when you look closely are linux/XFS boxes with a fancy gui.

XFS pretty good

[bobcat7677]

Just to add my $0.02... I used XFS with several Mandrake versions running in server configuration and had good success overall. The thing that kept me coming back was just how fast the filesystem worked. I did have an issue once with the whole filesystem blowing up after a power outage. But most of the time the systems would come right back with no problems.

Re:Nothin' like a dead GRUB...

[Jeffrey Baker]

GRUB has changes in CVS as recently at October 21, 2003, which is less than two months prior to this writing. I'd say it's alive, and even if it's dead, who cares as long as it works properly?

Re:I say what....?

[justins]

Faster, More trusted (SGI's been using it for how many years now?)

Not for as many years as ext2 has been in use.

My experience with XFS

[Phucilage]

I've been patching XFS into 2.4 kernels and have had nothing but great performance so far. It's been extremely fast. Recovery when my machine hasn't been shut down properly has been worlds faster than ext3 and reiser3x (i've yet to try reiser4).

I haven't run into a SINGLE problem at all and recommend it to all, as it's speedy and does a good job. It's also encouraged me to install GRUB which just rocks, plain and simple.

Re:Nothin' like a dead GRUB...

[M1FCJ]

if ain't broke, don't fix it.

Good morning everyone

[haraldm]

Just for those who live behind the moon. XFS has been available in Su^HUSE Linux and UnitedLinux for quite a while now. Guess who Christoph Hellwig works for? So folks, there is no need to spread XFS FUD between the lines - I have been using it exclusively privately and for customers for ages (whatever is ages in the IT universe).

Re:Journalling and HDD write cache

[haraldm]

You don't want to run write-behind filesystems and a harddisk write cache at the same time.

This is not just ext2/3

[pr0ntab]

UFS on Solaris can also be bitten by the same bug. :-(

You should always remount ro if possible before backing up; otherwise make a FS snapshot of the disc after syncing to ensure nothing changes underneath.

If you can't get sane access to the block device, not all is lost. The dangers of operating the dump utility at the device level doesn't always result in problems in that dump operates conservatively when scanning directories, etc. A common scenario is when a file is opened for writing between the time you read it's inode and the time you attempt to extract n blocks from it. In a simple case, this is okay since the copy of the blocks you pulled from the stat structure are probably still valid for the old copy of the file; any new blocks written are not likely to reuse the old blocks unless the filesystem is close to full. Backing up a file that is currently being accessed R/W is not advised, however. Quiesce any databases if you value them.

It's better than nothing. If you run the dumps often enough, you can recover from one bad one by merging in the missing directories from a previous one. I've never had to do this in the past, though, maybe I've been lucky. :-)

woah, Woah, WOAH, wait a minute, calm down.

[pr0ntab]

What, did Molnar, Trovalds, and Tosatti all rent Rider trucks and run over your dog a few times?

Re:So much for "no major changes"

[aminorex]

I've had enough experience with XFS and ext3
to know that there is no or marginal gain
in reliability from moving to XFS. What there
is, is xfsdump, but, honestly, how many Linux
users will ever dump a filesystem?

*Everybody* wins from low-latency and preempt.

somewhat OT question....

[Cnik70]

When I have a power failure and my system comes back up (I'm running SuSE 8.1 with reiser) I get notices that truncations have been made. Does this mean that files have been deleted?

Re:I say what....?

[mcbridematt]

Well, I'm going put my own server up soon, I've had enough of static html, and my servlet skills are getting better.

Re:An Overview (OT)

[AndyCap]

Undeleting on ext2 is possible with for instance recover while it is afaik impossible on ext3. recover on ext2 does have its limits but it has saved my skin a
couple of times. ;)

wrongo, ext2 is old than XFS

[justins]

XFS for SGI machines has been out longer than ext2 for Linux (as Linux used the Minux filesystem for some time).

ext2 has been around since early 1993:
http://web.mit.edu/tytso/www/linux/ext2intr o.html

XFS has been around on IRIX since late 1994:
http://www.ncsysadmin.org/files/xfs_linux.p df

Nice try, thanks for playing. Maybe you'd like a copy of our home game?