Benchmarking XFS, ext2, ReiserFS, FAT32

blakestah writes: "Well, it looks like someone on the LKML has taken upon himself to do some benchmarking of ReiserFS, ext2, and XFS using the 2.4 kernel series. It is not a real benchmark test, but kind of interesting nonetheless. See the results (in Spanish) at this LUG in Mallorca. Simple runs of dd, tar, and rm are shown, and for most of the tests XFS is pretty dern fast, beating all the others. The exception is removal of a large source tree (the kernel source), for which XFS is the slowest by a fair amount. See this kernel post for the translations of important words. It will be nice to see more such open benchmarking posted, because benchmarks provide developers goals." The contrast between FAT32 and XFS is particularly interesting to see.

Comments

ReiserFS and rm -rf

[Anonymous Coward]

It is amazing how fast ReiserFS can remove a source tree to the Linux kernel. Almost as soon as you hit the "return" key, it is finished. Blows me away. It is so fast it could be dangerous to the careless. It works so fast that by the time you realize your mistake *everything* would be gone--in the blink of an eye.

Re:Does it bother anyone...

[Anonymous Coward]

Deleting files on a FAT* partition doesn't actually delete the files. It just marks the first character of the filename with an invalid character (0 ASCII IIRC) so the filesystem thinks the space is free. That's why undelete utilities work -- they just search for all files with the invalid first character and then change the character to something else (which the program prompts the user for). Assuming the space the file had before hasn't since been used by another file, you get the file back.

My guess here is that FAT32 just marks the root source directory as deleted and then moves along. (This is how it works under DOS/Windoze at any rate.) If this were so, FAT32 is hardly fast here -- it takes 6.7 seconds to do just one write to the FAT table!

To be honest though, I'm not sure how the various Linux FSes work, so maybe they're doing it the same way, though I doubt one FS entry would take 10 - 20 seconds.

Those "tests" aren't good benchmarks

[Anonymous Coward]

Benchmarks... bah. I don't think the ones linked to on that Spanish site are worthy of the name. For those that can't see it because it's /.ed, it makes out XFS to be quite a bit faster. Some relevant comments from the LKML... [the last one showing that XFS is not always faster] Alan Cox: "reiserfs seems to handle large amounts of small files well, up to a point but it also seems to degrade over time. ext3 isnt generally available for 2.4 but is proving very solid on 2.2 and has good fsck tools. Ext3 does not add anything over ext2 in terms of large directories of files and other ext2 performance limits. XFS is very fast most of the time (deleting a file is sooooo slow its like using old BSD systems). Im not familiar enough with its behaviour under Linux yet." Andi Kleen: "On one not very scientific test: unpacking and deleting a cache hot 40MB/230MB gzipped/unzipped tar on ext2 and xfs on a IDE drive on a lowend SMP box. XFS (very recent 2.4.4 CVS, filesystem created with mkxfs defaults) > time tar xzf ~ak/src.tgz real 1m58.125s user 0m16.410s sys 0m44.350s > time rm -rf src/ real 0m50.344s user 0m0.190s sys 0m13.950s ext2 (on same kernel as above) > time tar xzf ~ak/src.tgz real 1m26.126s user 0m16.100s sys 0m36.080s > time rm -rf src/ real 0m1.085s user 0m0.160s sys 0m0.930s ext2 seems to be faster and the difference on deletion is dramatic, so at least here it looks like Alan's statement is true. The test did not involve very large files, the biggest files in the tar are a few hundred K with most of them being much smaller. The values stay similar over multiple runs. I did not do any comparisons recently with reiserfs, but at least in the past reiserfs usually came out ahead of ext2 for similar tests (especially being much faster for deletion)"

Re:Those "tests" aren't good benchmarks

[Kiwi]

I don't think the ones linked to on that Spanish site are worthy of the name

Keep in mind that the article starts out by saying "This is a little more rigourous than what was posted to the mailing list the other day". Also says "This comparison only measures one aspect of the filesystems, and there are other important aspects to consider when deciding on a filesystem".

- Sam

Re:Those "tests" aren't good benchmarks

[norton_I]

I just did some similar benchmarks on XFS vs. ReiserFS with and without the notail option, and ext2 (for reference -- I wanted to switch to a journaled filesystem).

My tests were 1) copy a several gigabyte file tree (/home) to a temp filesystem, 2) run du on the tree, 3) run Bonnie++, 4) build a kernel, and 5) rm -rf the whole tree.

My results were that ext2 and xfs had high read/write throughput, resierfs-notail was slightly lower, and resierfs without notail was a fair bit slower.

All metadata intensive operations (rm -rf, du, bonnie++ small files, etc) were blazing fast on ReiserFS, slow as molasses on xfs, and fast on ext2, but scaling poorly to high numbers of files.

xfs could do slightly more random seeks than either ext2 which in turn beat out Reiserfs, by about the same amount.

reiserfs doesn't have an fsck -- at least the one I have is a noop.

I switched to reiserfs-notail, and was proptly annoyed by the lack of fsck, but otherwise it is running well. I ran a pair of make -j6's, some directory copies, and an updatedb, then hit the power and it is running fine.

My personal preference would be to have ext3 in 2.4 along with Daniel Phillips' hash-tree directory patch stabalized--I think that would smoke reiserfs and still give me journaling.

Re:Those "tests" aren't good benchmarks

[JWhitlock]

You're sitting comfortably in front of the console of your server happily clickity-clicking away, the syslog quietly printing its timestamps every 5 minutes, load is normal, users are quiet and the world seems to be in place.

Suddenly alarm rings, your syslog becomes all red and the security specialist comes storming through the door shouting "someone executed 'rm -rf /&' as root"...

At this point, would you prefer to have a filesystem that is *sooo* fast deleting that it's already gone through /bin and moved on to /home/, or would you like to have one that has barely just started? :)

At this point, I quit my job, and wander the wastes, warning about using "password" for the root password...

Seriously - this is what full backups are for. Over the lifetime of the system, fast deletes by every user may compensate for the lost time of doing a full restore, rather than catching the global rm halfway through. Plus, a fast delete often means that nothing is really being deleted, file system references are just being dropped - there may be some easy recovery tool for this old worst-case-scenario.

Re:Why FAT32?

[volsung]

I would imagine they didn't test against NTFS because the kernel drivers are very unstable for it. Sure, they could have installed Win2K and tested that way, but then OS differences would affect the results. A pure filesystem test should use the same kernel setup with only the fs drivers changing.

Re:Whatever happened to Tux2?

[V.]

http://kernelnewbies.org/~phillips/

I've always been pulling for XFS.

[Omega]

I guess I've always been partial to XFS and I hope that it becomes the new default filesystem for Linux.

This guy Dave (I forget his last name now), from sgi gave a presentation to the DC-LUG back in 1999 and talked about XFS and how sgi wanted to release it as GPL to become a core component of Linux. He also talked about the history of XFS and how they had to invent a new size prefix to describe how large a filesystem XFS could accomodate ("exo-byte" = 1024 Gb). XFS has been used by sgi for their MIPS and Cray machines ever since 1984, and now that sgi has donated it to the Linux community, I think we'd be remiss if we didn't welcome it with open arms.

But that's just MHO. ;)

Re:I've always been pulling for XFS.

[landley]

zetabyte.

exobyte, yottabyte, zetabyte. x-y-z.

THEN you do a lottabyte. :)

Rob

Re:I've always been pulling for XFS.

[rangek]

they had to invent a new size prefix to describe how large a filesystem XFS could accomodate ("exo-byte" = 1024 Gb)

This is a slight exaggeration. A kilobyte is 1024 bytes, a megabyte is 1024 kilobytes, a gigabyte is 1024 megabytes, a terabyte is 1024 gigabytes. Oh wow, it is not even an exaggeration, it is just wrong, I guess. Anyway, if we keep going up the table of SI prefixes in my CRC, we could have petabytes, which would be 1024 terabytes, and then exabytes = 1024 petabytes, and so on. The is no need to invent things, it is all taken care of, from 10E-24 (yocto-) to 10E24 (yotta) (now thats a lotta stuff) ;)

Re:I've always been pulling for XFS.

[Dragonmaster Lou]

I believe 1024 TB is a petabyte. 1024 PB is an exobyte, and 1024 EB os a yottabyte.

Re:I've always been pulling for XFS.

[Enonu]

That sure is a yottabytes!

HAR HAR

NAS

Re:I've always been pulling for XFS.

[Boulder Geek]

This guy Dave (I forget his last name now)

Probably Olsen.

XFS has been used by sgi for their MIPS and Cray machines ever since 1984

You mean 1994. At I can assure you that using XFS in 1994 was a thrill a minute ;-).

Re:I've always been pulling for XFS.

[wjr]

A couple of years ago, one of the international standards organisations (I don't remember which) got annoyed at the "mega=10^6" versus "mega=2^20" confusion, and came up with new prefixes. According to these:
1000 bytes = 1 kilobyte
1024 bytes = 1 kibibyte
1000000 bytes = 1 megabyte
1048576 bytes = 1 mebibyte
and so on - gibibytes, tebibytes, pebibytes, ebibytes, each larger than the previous one by a factor of 1024.

Even though the prefix names sound silly, I Am Not Making This Up.

Re:I've always been pulling for XFS.

[enrico_suave]

"I believe 1024 TB is a petabyte. 1024 PB is an exobyte, and 1024 EB os a yottabyte."

Actually I believe 1024 EB is a lottabyte (*badoom tch!*) =P

e.

Re:I've always been pulling for XFS.

[foobar104]

While I agree with the substance of your message, you've got a couple of facts wrong.

First, the maximum filesystem size that XFS can handle is 18 exabytes. Since exabyte is also the name of a brand of tape drive, it's more common to hear of people talking of 18 million terabytes.

(Aside: an intresting statistic found on this page says that as of 1995, 5 million terabytes was about enough data to store all words ever spoken by humans, ever. Cool.)

Also, XFS was never used on Cray systems. XFS made its first appearance (if I remember correctly) on IRIX 5.3 back in 1994. Other than being off by four orders of magnitude in your sizing and by ten years in your dates, I think you're exactly right. ;-)

Re:I've always been pulling for XFS.

[Verteiron]

Isn't 1024 GB a terabyte? And 1024TB = 1 exobyte?

Re:FAT32

[slim]

"It is quite surprising to see the write time be so slow for linux, as quite frankly, FAT32 is so simple (no transaction) it *should* be only slightly slower than optimal in medium to large file size cases."

I would imagine that the reason FAT32 isn't highly optimised on Linux is that it's only there as a compatibility tool. Nobody running Linux would put data on a FAT32 partition when performance was an issue. Being able to get at the filesystem at all is enough.
--

Re:Either the site's slashdotted already...

[jd]

To get XFS to work with Linux 2.4.4, you'll need the version in their CVS repository.

XFS and MOSIX -might- work, with sufficient application of brute-force. Neither work with 2.4.4-ac6, and I believe there's a lot of problems with FreeSWAN's IPSEC, too.

The only patches I've found to be relatively unstressful are the International Patch, the POSIX Timer patch, and IBM's Next Generation PThread code.

Re:Either the site's slashdotted already...

[jd]

Absolutely understood. I've e-mailed the authors of the various packages, asking if there's some way of getting the packages to work together.

The closest to a printable response was some sneer to the effect that the writers weren't interested in supporting Alan Cox's "private OS".

(Curiously, the patches tend to break with every release of the official kernel. Hmmm. Maybe if these coders got off their high horses long enough to see how code migrates between the official kernel and the AC series, they'd have an easier time keeping up.)

As of right now, though, you're going to have to do some of the merge by hand. And it's likely to prove difficult.

Look on the bright side, though! If you =DO= get a merged patch, it'll be YOUR patch that's used. Who wants second-best?

Re:XFS size

[jd]

Size of Linux-2.4.4, with no patches: 3,239,950 lines (as calculated by doing a wc -l on all .c's and .h's in the tree, then summing the result), and 131,448,832 bytes (for the whole tree, using du).

Size of Linux-2.4.4 with XFS (as taken from SGI's CVS tree): 3,612,541 lines (same algorithm) and 145,367,040 bytes.

Either the site's slashdotted already...

[jd]

Or they've a serious PostgreSQL bug. :)

Seriously, XFS looks interesting. I did some stats on source size, and posted them to K5. :) Dropped in score so rapidly, it'll reach the Earth's core sometime in the next half hour.

Essentially, XFS is 10% of the entire kernel size, making it perhaps the single-most sophisticated driver available. On the flip-side, I can't help but feel that that much code is going to have -some- impact on the rest of the system.

Talking of benchmarking, how does IBM's "Next Generation PThread" code stand up? And how on Earth are you supposed to install it? It clashes with glibc, making an RPM install, ummm, of questionable safety. And once you start with RPM (or tarballs, or any other system), it's unwise to mix-and-match. Either you keep track of where things are, or the system does. Half-and-half is NOT good.

Lastly, anyone found a way to get XFS, JFS and AFS into the same kernel? (Without using a sledgehammer, preferbly.)

Re:Either the site's slashdotted already...

[Lazy Jones]

They configured PostgreSQL with a low limit for the number of client processes (default = 32 usually). Either that should be increased, or the connection pooling configured in a way so that this limit isn't exceeded. Limiting the number of simultaneous Apache processes also helps, sometimes.

Re:Either the site's slashdotted already...

[Dr.Dubious DDQ]

Essentially, XFS is 10% of the entire kernel size, making it perhaps the single-most sophisticated driver available. On the flip-side, I can't help but feel that that much code is going to have -some- impact on the rest of the system.

Well, it seems to break modules in the one patch of the 2.4.3 kernel that I've tried so far. (Unpack fresh kernel source, patch with both XFS patches, compile...XFS works [compiled directly into kernel], but every module I try to load [sound, drm driver, etc.] complains about unresolved symbols. Not sure what to do about that). Anybody else run into this one? I've been itching to try XFS on a couple of my systems...

Lastly, anyone found a way to get XFS, JFS and AFS into the same kernel? (Without using a sledgehammer, preferbly.)

If this isn't bad enough, I'd like to get XFS and MOSIX into the same kernel. Hows THAT for masochistic? Need to figure out how to get the XFS patch into 2.4.4, though...

---

Re:lousy metadata performance with XFS?

[johnnyb]

It's true. I ran bonnie++ on a test machine under ext2 and xfs. xfs had REALLY slow deletes, but fast throughput. XFS did 36 deletes per second, while ext2 did 802 deletes per second. As far as throughput, XFS did 538K/sec on block reads, while ext2 only got 436. Also, in general, xfs used less CPU resources. Also, ext2 kicked xfs's butt in all areas when there was only one bonnie++ process running. The benchmark data I've given was running three simultaneously on the filesystem. So, if the server only has one process, I'd NOT go with xfs. If the server has multiple processes, xfs probably beats most things, especially on block read performance.

Note, also, that my test machine was only a AMD K6-2 266 w/28M RAM

Re:Well, I guess..

[Kiwi]

Anybody know the Spanish word for "slashdotted"

Well, "slashdot" is Barrapunto when translated in to Spanish, so the verb would be "barrapuntar". E.g. ellos barrapuntaron el pagina (They slashdotted the page).

- Sam

I managed to get a copy of the page

[Kiwi]

Since I know a little Spanish, here is my very crude translation of the core benchmarks:

First benchmark:

Writing, reading, and deleting a fairly large file (256MB). The commands used are as follows (omitting the redundant 'time' command used in all cases):

• dd if=/dev/zero of=./prova bs=1M count=256
• dd if=./prova of=/dev/null bs=1M count=256
• rm -f ./prova

And the time these commands took (in seconds):

Filesystem Write Read Delete
ReiserFS 18.5 23.41 0.4
Ext2FS 20.3 21.38 0.57
XFS 16.32 19.42 0.26
FAT32 43.65 27.98 1.59 Second benchmark: This benchhmark was donw with the source code of Linux 2.4.4. The .tar.gz file was first uncompressed, so that all the work was done on the tar file (which is larger then 100MB). The commands used on the uncompressed .tar file are as follows (with the time command ommitted again):

• cp linux-2.4.4.tar prova.tar
• tar xf prova.tar
• rm -f prova.tar
• rm -rf linux

The times these commands took (in seconds):

FS Copy Extract rm file.tar rm -rf dir
Reiser 38.48 58.44 0.45 10.09
Ext2FS 21.31 59.19 2.88 11.12
XFS 16.21 35.44 0.18 21.96
Fat32 39.76 134.19 1.2 6.7

Give the poor site a chance...

[m2]

What the gut got:

$ddif=/dev/zeroof=./provab s=1Mcount=256

$ddif=./provaof=/dev/nullb s=1Mcount=256

$rm-f./prova

FSWriteReadrm

ReiserFS18.523.410.4

Ext2FS20.321.380.57

XFS16.3219.420.26

FAT3243.6527.981.59

$cplinux-2.4.4.tarprova.tar

$tarxfprova.tar

$rm-fprova.tar

$rm-rflinux

FScpextractrmrmdir

ReiserFS38.4858.440.4510.0 9

Ext2FS21.3159.192.8811.12

XFS16.2135.440.1821.96

FAT3239.76134.191.26.7



[1] ReiserFS has a high (reproducible) variance when copying the kernel tarball.

All test ran three times with the exception of the reiserfs kernel tarball copy, which ran six times. Machine is running in single user mode. Results timed with time(1).

Re:XFS size

[Booker]

Also, one more tidbit, the CVS tree also contains the kdb code, so that's a (small) part of the increase.

Were you counting the cmds/ dir?

[Booker]

Just realized, CVS has a kernel/ dir and a cmds/ dir.

Any chance your tallies include the whole cmds/ tree? That's all userspace stuff...

XFS size

[Booker]

I can't help but feel that that much code is going to have -some- impact on the rest of the system.

Although XFS is big, it doesn't stomp on the kernel that much.

If you're looking at it from a pure "volume of code" point of view, here's some info:

The XFS patches are split into 2 parts, one which contains kernel changes, and one which is the filesystem itself.

The "core-kernel" patch is about 190k, while the actual filesystem code patch is about 4.3M.

Bear in mind that a 190k patch to the kernel does not mean 190k of new code, either, since you have to take out the context lines, the headers, and the delete lines.

Overall, the impact on the kernel isn't as big as you might think, looking at the overall size.

Now, whether or not executing code in the filesystem slows down the rest of the system, I don't have any real data on that, although I have not noticed any detrimental effect on my system.

Re:No ext3?

[sl70]

I'm using ReiserFS now and I'm really happy with performance/crash recovery. I have a lot of cat-related outages...

I installed xfs a week ago. Yesterday I wrote about it to my ex-office mate, saying I couldn't wait for a power failure. Got my wish this morning at about 4:00 a.m. My machine took 15 seconds to reboot. My colleague's machine with ext2 took nearly 3 minutes. Cool messages in the logs:

May 10 03:55:11 musuko kernel: Start mounting filesystem: ide0(3,5)
May 10 03:55:11 musuko kernel: XFS: WARNING: recovery required on readonly filesystem.
May 10 03:55:11 musuko kernel: XFS: write access will be enabled during mount.
May 10 03:55:11 musuko kernel: Starting XFS recovery on filesystem: ide0(3,5) (dev: 3/5)
May 10 03:55:11 musuko kernel: Ending XFS recovery on filesystem: ide0(3,5) (dev: 3/5)

FAT32 vs. XFS

[MSG]

The differences between FAT32 and XFS may be interesting, but keep perspective. What you're seeing is the difference between the Linux driver for XFS and the Linux driver for FAT32, and not necessarily the inherent properties of either filesystem.

Don't get me wrong, I'm not comparing FAT32 to XFS by a long shot! But FAT32 is a fs that not a lot of hackers care about enough to improve the performance under Linux. Personally, I've always found that FAT32 access under Linux has been abysmal compared to access to the same filesystem under Windows.

No ext3?

[Jethro]

I used ext3 about a year or to ago. It worked a lot better than it should have. Or so I thought.

It claimed that it'll still have fsck run when you crash, but actually nothing was getting run... I'm pretty sure that it was just not repairing itself after crashes. I'm really surprised I never lost any data until, well, one day I lost the whole damn partition.

I'm using ReiserFS now and I'm really happy with performance/crash recovery. I have a lot of cat-related outages...


--

SI Prefixes

[rangek]

This isn't specifically for bytes, but the list of SI prefixes is here

The new prefixes for binary units (which nobody uses (the prefixes, not binary units)) are here

Re:Why FAT32?

[Scudsucker]

Maybe because this was a test of filesystems that work under Linux, not filesystems in general. To have a reliable test of NTFS, you'd have to use it under NT or Win2k, unless you wanted to do some shady benchmarking and use the developing NTFS driver for linux. Also, the point was do use the exact same commands for each filesystem...dd, rm, etc, which you couldn't do in Windows.

Fat32 was probably included because it's its so well supported and easy to do.

Re:Does it bother anyone...

[zatz]

Little time or patience required, I can explain the delete behavior easily. (Not that I would label myself an FS hacker... distributed, maybe.)

Modern inode-based filesystems scatter both files and metadata across the disk, to reduce average seek time and fragmentation problems (free blocks are always nearby when using cylinder groups, unless the disk is very very full). So deletion can involve visiting many areas of the disk, and also traversing the allocation tree for each file and adding its blocks back to the free list (or vector).

MS-DOS FAT filesystems have most of the metadata tightly clustered in the two file allocation tables... only subdirectories containing filenames, some metadata, and pointers into the FAT are stored outside. (32 bytes each.) So for FAT you have to seek, read, and write once (usually) for each subdirectory, but for all the normal files you just make one pass on each FAT, which probably involves no seeking, since they fit entirely within a single cylinder on modern disks.

DOS just made this process seem excruciatingly slow because you were typically using it without caching and on floppies. FAT-based filesystems are a bit of a naive approach, and their average-case performance, especially under load, can be quite poor if you can't cache the entire FAT in memory. But they aren't all that awful.

lousy metadata performance with XFS?

[zatz]

Hmm... I don't know exactly why metadata operations should be slow on XFS, but my understanding is that XFS is primarily useful for:

• Journaling, which depending on how it is implemented, would make exactly when metadata writes occur a lot less important. So you may just be seeing the forced sync of 5-second-old dirty metadata from bdflush fouling up the performance. Perhaps XFS was designed with a much looser disk/memory binding in mind.
• Very high throughput sequential I/O on very large files on very large disks/RAIDs (think video editing), which might involve optimizations or tradeoffs that are not synergistic with low-latency metadata operations.

Just my thoughts....

ntfs vs fat32

[LordXarph]

fat32 is an interesting control, but in an ideal benchmark, ntfs would have been used, as it is designed closer to other filesystems, as opposed to fat32, which is more like Baby's First Filesystem(tm).

Though I will grant that NTFS would have been hard or impossible to benchmark in this test, given the lack of robust drivers for it.

-Lx?

Re:ntfs vs fat32

[Malcontent]

Maybe it's not legal to benchmark NTFS.

Re:ntfs vs fat32

[lunatik17]

I think we all knew FAT sucks already. And besides, I've found the linux vfat driver to be at least as good on the native windows version. I'm really quite impressed with it, actually. Now, the filesystem itself is another matter...

Re:ntfs vs fat32

[slamb]

The other problem being that if the Fat32 test was run under Linux as well, all it's showing is how good the Linux driver is, not the filesystem itself.

That's true of all the filesystems being benchmarked. There's no way to seperate the performance of the filesystem from the performance of the driver, unless there are two drivers for the same filesystem on the same OS. Benchmarking it on another OS would change way too many things to get any meaningful results.

In fact, all of the filesystems except ReiserFS have drivers under other operating systems. (FAT32 is obvious. ext2, FreeBSD has it, I think there's an NT driver, etc. XFS was originally for IRIX.)

Re:ntfs vs fat32

[SuiteSisterMary]

The other problem being that if the Fat32 test was run under Linux as well, all it's showing is how good the Linux driver is, not the filesystem itself.

Why FAT32?

[ViceClown]

FAT32 has almost zero relevance here because if we were all using Win2k or NT we would be using NTFS just like Windows servers do. Why on god's green earth this test didn't test against NTFS is completely beyond me other than to have a weeker MS filesystem to poke fun at. Real objective, guys.

Slashdotted Already

[sid crimson]

Maybe they should benchmark their DB system.

;-)

-sid

Re:XFS deletion performance.

[be-fan]

Its not about how *much* space, but about how often. Space gets allocated much more often then space gets freed, and usually at more time critical moments. Take, for example, a media system. If you're capturing a video (what else do you do on an XFS sytem?), the file will get enlarged several times during itslifetime. It will get deleted only once. Moreover, you care how quickly you can allocate space, since it effects you're capture quality, but don't really care if deleting files is slower (within reason, of course.) Or, take log files. Log files grow many times, are deleted only once. Also, as another poster pointed out, you usually delete log files when the system is lightly loaded doing internal maintainance. There is almost no application that I can think of (save artificial benchmarks) where XFS's deletion performance would be more important than its file creation/enlargement performance.

XFS deletion performance.

[be-fan]

Yes, XFS does kick that much ass. Still, the deletion performance surprises me. Of course, the other speed things make up for it, but its still a puzzle. If you don't know, rm -rf has historically been a slow operation on XFS.

Here's my theory. Ext2 uses a bitmap to track free blocks, and I'm pretty sure ReiserFS does as well. Free block runs on XFS is managed by two B+trees, one keyed by address, one keyed by size. Thus, allocating space is very fast on XFS, and it is easy to keep things contiguous. However, inserting runs of blocks into both trees is a slower operation then simply clearing the appropriate bits. This would explain the difference between the file creation speed (extraction test) and file deletion speed (rm -rf test.) If this is the case, I think it is quite a good tradeoff, given that space is allocated much more often.

Of course, IANAXE (I am not an XFS engineer) so this is just my theory. I'd appreciate it is someone more informed about XFS could tell me the reason for the performance delta.

Re:XFS deletion performance.

[achurch]

If this is the case, I think it is quite a good tradeoff, given that space is allocated much more often.

Only if you've got an infinite drive...

--
BACKNEXTFINISHCANCEL

Re:XFS deletion performance.

[achurch]

You're forgetting temporary files: what about all those ccxxxxxx.[iso] files that gcc creates--and then deletes--for each source file you compile, or the test programs autoconf's configure script generates for every test it does?

I will agree that allocation has greater importance overall than deletion, though.

--
BACKNEXTFINISHCANCEL

Re:XFS deletion performance.

[vidarh]

Neither of those will usually comprise the majority of usage, though.

Re:XFS deletion performance.

[vidarh]

Or if you never delete the full contents of your drive - which most people likely won't.

It still holds for most cases: You'll copy a bunch of data to the drive over time, and some of it will be deleted, but most of the time a lot of the data you accumulate will be data that you'll keep around for the lifetime of the drive.

Which equals less deletion that allocation...

Perhaps not as significant as the original poster thinks, but his claim still holds true for most users.

More significant, I'd assume, is that a lot of deletion often is maintenance that won't take place during peak usage of the system, and that often may just be left running in the background, to complete whenever it completes.

Allocation is in many applications much more time critical, as allocation is mostly done by applications that the users of the system are much more likely to care about performance for.

Re:But reiserfs wastes 1/3 of disk space!

[Convergence]

Did you look through the mailing list for ReiserFS?

If you had, you'd have seen that Reiserfs is journaling (Well, actually log structured)

It reserves 32mb for the log... Ergo, 32mb of missing space. If I remember right, you can decrease this when formatting it.

exabyte?

[jimmcq]

1 kilobyte = 1024 bytes
1 megabyte = 1024 kilobytes
1 gigabyte = 1024 megabytes
1 terabyte = 1024 gigabytes
1 petabyte = 1024 terabytes
1 exabyte = 1024 petabytes

Data Powers of Ten

Which eventually yields. . .

[kfg]

a 'Yottafied' filesystem.

"Hmmmmmm,large it is. Strong is the file system in this one, hmmmmm."

KFG

Celebrate National Benchmark Week!!!!

[Lizard_King]

XFS, ext2, ReiserFS, FAT32
GeForce3
Kernel Benchmarks

Does it bother anyone...

[electricmonk]

...that FAT32 actually beat ext2 on two tests, and even beat EVERYONE ELSE on the test that involved removing the kernel source tree? Could an FS hacker with some time and patience on his hands please explain what's going on?

--

Re:I AM THE SYSADMIN OF BULA

[gallir]

I found two things while our site is being slashdotted:

1. The maxclients in php.ini doesn't work.

2. We want MySQL !!!

Right now, the author is doing more extensive tests. We will put them is a static page.

Hope the server is still alive when I arrive home.

Cheers, assasins;-)

--ricardo

Re:No real world benchmarks

[gallir]

It wasn't any formal benchmark, just a small test of a young student which wasn't released for Slashdot or any publication. Just a game... We didn't (and the author neither) wanted it to appear in site as slashdot.

Sorry if it annoyed you. But don't complain to a guy who likes just to play around with Linux.

--ricardo

I AM THE SYSADMIN OF BULA

[gallir]

And it's hard to maintain postgress running.

You should tell ne before you try to slashdot us ;-) So, I could have time to increase the PG backend.

Hope we can keep it running now... (poor PIII 500Mhz)

--ricardo

Whatever happened to Tux2?

[ortholattice]

Possibly off-topic, but what's the story on the Tux2 http://slashdot.org/features/00/10/13/2117258.shtm l file system? It sounded like a great idea, now it seems even the links are broken.

Well, I guess..

[hhg]

That their webserver can not keep up with their filesystem anyway...

Speed is one thing..

[baptiste]

While speed is important, lets not lose sight of one of the other main benefits of a JFS - recovery time. Even if XFS is a little slower than ext2, I'd still use it given the recovery advantage. Yes reiserFS will do it too, though I belive XFS to be a more 'compilant' JFS in terms of b-trees, etc and I believe as the drivers mature IMHO!!! that XFS will pull away from reiserFS in terms of performance. Only time will tell.

Needless to say I've got a pair of 61GB IDE's screaming for XFS once the 2.4.4 patches are out. Can't wait.

--

Slashdotted!

[Scoria]

Mirror here.

No idea why they're including something as outdated as FAT...

Re:Are the benchmarks on the Reiser site rigged?

[vidarh]

I can assure you that whatever the quality of Reiserfs, it beats ext2fs hands down in the cases it's designed to handle particularly well: Large (huge) directories and small files. The company I work for operate a mail server for about 1.1 million people, and we have quite a few million files, of an average size of less than one kilobyte (we're using maildir format for the accounts, and also store a lot of small XML objects as separate files for each user), and some of our directories has tens of thousands of entries.

Running something like that on ext2fs would (apart from the agony of fsck'ing 800GB of storage) be completely hopeless.

When it comes to setups with a few large files, though, the advantage isn't that great, and the numbers in the article makes reasonable sense.

Reiserfs should be your filesystem of choice if: a) you want to be able to put gazillions of files in a single directory when there is no logical hierarchy to the data (our tests indicate that Reiserfs handles shallow directory hierarchies with many files pr. directory faster than the opposite), and b) you want to be able to efficiently store a 100 byte file when grouping your data logically would give you file sizes of that magnitude, instead of grouping many pieces of data together.

We've been running Reiserfs for well over a year now, and it works great. It's important that you keep up to date on bugfixes, though, and that you're very careful about your recovery procedures - reiserfsck is really a last resort, and you should always ensure you copy out as much as possible of any data on a damaged volume, and preferrably take a raw copy of the entire volume, before running it. That said, I wouldn't hesitate to recommend Reiserfs to anyone with the specific needs I mentioned above.

FAT32

[rufusdufus]

I have not seen the FAT32 source linux, but I have for Windows and NT (as I wrote it). I can say that (as another posted has speculated) the quality of the source algorithms has a huge impact on the performance of the filesystem. It is quite surprising to see the write time be so slow for linux, as quite frankly, FAT32 is so simple (no transaction) it *should* be only slightly slower than optimal in medium to large file size cases. NTFS should certainly show slower on writes of larger files, and if it doesnt, you know you have a huge bug somewhere in the FAT32 driver. I dont know enough about the other FSs to comment, but I do suppose they should not be significantly faster.

a brief summary of the different file systems

[reposter]

XFS is optimised for dealing with streaming media, and so deals well with high IO and large files.

JFS has been around for years under AIX. It's a well proven general purpose journalling filesystem.

ReiserFS is the best established of the Linux journalling filesystems. It has several fairly innovative features and is more efficient than ext2 in terms of space utilisation. People are using it as their primary filesystem now, although it's still in development.

EXT3 is (unsurprisingly) a development of EXT2. It lacks most of the pretty features of the other journalled filesystems, but has the significant advantage that you can turn EXT2 partitions into EXT3 (and vice versa) without any trouble at all.

Re:troll?

[mixtim]

What many Linux users forget (or don't know) is that FreeBSD (and the other BSD's) default to synchronous writes with drive write caching turned off. This is done to prevent losing large chunks of data in the event of a sudden system shutdown. Linux defaults to asynchronous writes with write caching turned on. The other piece of information missing from the post is that /usr/ports on FreeBSD is 110MB with over 17,000 directories and over 62,000 files. If you plan on removing that many directories and files on a regular basis (not common on most servers) then just turn on asynchronous writes.