Linux File System Shootout

IpSo_ writes "Finally an extensive, human readable Linux file system benchmark has been unleashed upon us. Originally posted on the Linux Kernel mailing list, using two of the most popular benchmarking tools available, it compares all the major file systems, including their different mount options. The results are surprising."

Comments

Short summary

[mst76]

iozone benchmark
best: jfs
worst: ext3_journal

bonnie++ benchmark
best: ext2
worst: reiser4/reiser4_extents, ext3_ordered/ext3_journal

Re:Short summary

[JanneM]

Actually, at least in our case we thread the app, with one thread handling disk IO and other threads handling other aspects (such as CPU intensive stuff), precisely to squeeze out a bit more performance and so disk accesses do not interfere with and stall other stuff. You get this as soon as you try to do something in soft realtime (such as video applications). On one hand, you want to stream video to/from a drive as quickly and efficiently as possible; on the other, you want to do some CPU-intensive operations (filtering, resizing) on the video stream at the same time.

I'm not saying that trading CPU for filesystem speed is a bad idea; it isn't. What I'm saying is that it's not a simple "more is better" function, and that the cutoff for when it no longer makes sense does depend a lot on the application you intend it for. Again, to take an extreme, you would not want to have a system where the filesystem eats so much CPU the rest of the system essentially blocks, starved for CPU time, when the disk is used.

To take an even more extreme way of doing the tradeoff: you could compress and uncompress all data on the fly. That way you would increase transfer speed (and increase it quite a bit in the case of text files and similar) as well as decrease disk usage. It is not often done, though, because the tradeoff is not worth it in general.

For us, and our app, Reiser is on the wrong side of that cutoff point (and Reiser4 is not even on the horizon yet).

Re:Huh?

[matticus]

Well, here's IBM's page about it.

From what I've seen poking around USEnet, JFS seems to have the too little, too late problem. I've never seen it pwn a benchmark like it did today though.
I'm a little confused-I have been told XFS is the best designed, highest performing file system, and I would hate to think SGI is getting into a lot of this crap with SCO for a relatively slow journaling file system...

Re:Huh?

[Frodo420024]

I'm a little confused-I have been told XFS is the best designed, highest performing file system, and I would hate to think SGI is getting into a lot of this crap with SCO for a relatively slow journaling file system...

IIRC, XFS is more about guaranteed performance under various stressful conditions than about getting the absolute peak speed in calm conditions.

Summary

[samj]

Use XFS unless you want to do lots of deletes (as they are slow and expensive) in which case ext2 is probably a better bet since the files are probably temporary (Squid caches for example).

"linux reiserfs"

[bani]

type "linux reiserfs" when booting the installer, and you will have access to reiserfs during redhat install.

i've been using this method for ~2 years now.

Re:Throughput benchmarks only...

[zurab]

Have a look at Hans' benchmarks at namesys.com. Although he only compares Reiser4 to ext3, and may not be an objective party. But I'm surprised how well JFS performed anyway and that Reiser4 is unusually CPU-intensive.

Re:Results question

[NickFortune]

Is it fair to thow in a Non Journaling FS in a benchmark against a bunch of Journaling ones?

Yes. Of course. The ext2 numbers provide a baseline for the comparison comparison. Any journaled FS that could match it would have to be very good indeed. This isn't explicity stated anywhere - but this was posted to the kernel list. They can reasonably be expected to know the difference between ext2 and the rest. It's all data. Data is good.

I know we're used to seeing "benchmarks" used as corporate propaganda, but let's not forget what they're supposed to be used for

Ext3

[rf0]

Well ext3 might suck but when you've only got a resuce system that can read ext2 it can really save your neck. I would be intrested to see what is best in terms of stability though..

Rus

Worth Noting

[MajroMax]

It's worth noting here that the benchmarks were all run on files >= 1GB, if I'm reading the table correctly; this stresses the raw IO of the system, and doesn't really take into account the differences in tree-structure between the filesystems.

As for complaints about Reiser's performance -- last I heard, it was more optimized for many small files -- precisely the domain that this thing didn't test.

Re:Can't wait for Novell Storage System on Linux

[Anonymous Coward]

You forgot one thing. As an enterprise filesystem Novell was absolutely bulletproof long before RAID systems were in vogue. It took me awhile to even figure out why we needed one after years of running Novell as our main storage controller (flawlessly).

Novell could give *nix systems windows like (don't bash if you don't know) fine granularity over user access at the enterprise level along with true enterprise scaling. Again, if you have never worked in a cross enterprise environment, don't start bashing because you really can't appreciate some of Novells strengths until you need the features.

Re:human readable ?

[Psiren]

Since there was no legend explaining what the colors meant, I couldn't figure out anything from looking at them. Is the high number good? As in did the most work? Or is the high number bad? As in took the longest amount of time to do something?

Depends on the column. For K/sec, higher is better, so red cell shows lowest, and green shows highest. For %CPU, lower is better, so green shows lowest and red shows highest. It's not that complicated really if you take a few minutes to look at it. What you get from the data depends on what you were looking for in the first place.

Re:Sort of on topic...

[angle_mark]

There are some free and some commercial products which can offer full read/write + journalling access for ext3 partitions from Windows. I'd definitely recommend you pick ext3 over fat32.

Some examples..

Free: Explore2fs allows you to read ext2 and ext3. Limited write support is available.

Commercial: Ext2FS Anywhere don't let the name put you off as it has full read/write support for ext2, ext3 and I think reiserFS is supported now too.

this benchmark was performed using a 200Mhz CPU

[hansreiser]

which makes the whole thing pretty questionable in my view, especially when you consider that Nikita got completely different results on his more modern hardware (see www.namesys.com/benchmarks.html)

I don't really target 200Mhz CPUs in my performance tuning....;-)

Hans

Re:this benchmark was performed using a 200Mhz CPU

[Deagol]

How true a difference the hardware makes.

I took an old PII-350 w/ 128MB RAM and benchmarked ext2, ext3, jfs, reiserfs, and xfs on an old 5GB IDE drive. ext2 was the winner by a margin (raw throughput).

Now I'm beating up various hardware and software RAID configs on a dual Athlon MP 2200+ system w/ 2GB RAM and dual 3ware 8-port 7500 controllers w/ 180GB WD drives. JFS rises above the rest in terms of throughput (I didn't test XFS on this new machine), and, of course, reiserfs simply spanks everything in terms of file creation/deletions. The thing I noticed was the JFS had much lower CPU utilization for file creations/deletions and was twice as fast at it than the ext2/3 filesystems (it still got spanked by reiserfs, though).

If anyone's interested, the "best" overall was reiser w/ the mount options noatime,notail,nodiratimeall. Also, if anyone cares, on this machine, the Linux software RAID code at no less than twice the performance numbers over the 3Ware hardware RAID. Running RH9 with all RH updates applied.

Re:human readable ?

[TheCrazyFinn]

Never heard of Read-only filesystems?

Mount static filesystems read-only, and make them EXT2 for performance. Use a journalling FS for dynamic filesystems. Reap the benefits of both.

Re:DeFacto Standard

[drinkypoo]

he installed a hard drive, formatted it to Reiser, and moved the proxy cache to the reiser disk. I couldn't belive it. Just changing the filesystem caused an increase that was noticable across our network. At no cost!

He installed a hard drive. He didn't just format to reiser. The hard drive costs money.

If the proxy cache was formerly on a disk that was also doing other things, it would have sped up no matter what filesystem he used.

You will have to give us more information if you want your claims to have any merit.