Benchmarking Linux Filesystems In New 2.6 Kernel

An anonymous reader writes "KernelTrap has an interesting article about a recent benchmark conducted to compare five journaling filesystems available with the current 2.6.0-test2 Linux development kernel. The tests were conducted with a very simple shell script, mainly timing how long it takes to copy, tar, and remove directories. Looks like reiser4 is the fastest filesystem at the expense of consuming much more CPU, with ext3 trailing a ways behind."

Comments

fs issues

[Tirel]

speed is nice, but I think the more important question is; how stable is it?

Re:fs issues

[BrokenHalo]

I don't know about 2.6 yet, but Reiserfs hasn't let me down so far.

For a long time I resisted moving away from ext2, as I didn't want the journalling overhead and didn't mind the occasional e2fsck. After I got a few power outages I changed my mind :-) but I haven't been disappointed with Reiserfs' speed.

karma for Anonymous Coward

[Anonymous Coward]

The first item number is time, in seconds, to complete the test (lower
is better). The second number is CPU use percentage (lower is better).

reiser4 171.28s, 30%CPU (1.0000x time; 1.0x CPU)
reiserfs 302.53s, 16%CPU (1.7663x time; 0.53x CPU)
ext3 319.71s, 11%CPU (1.8666x time; 0.36x CPU)
xfs 429.79s, 13%CPU (2.5093x time; 0.43x CPU)
jfs 470.88s, 6%CPU (2.7492x time 0.20x CPU)

What's interesting:
* ext3's syncs tended to take the longest 10 seconds, except
* JFS took a whopping 38.18s on its final sync
* xfs used more CPU than ext3 but was slower than ext3
* reiser4 had highest throughput and most CPU usage
* jfs had lowest throughput and least CPU usage
* total performance of course depends on how IO or CPU bound your task is

Re:karma for Anonymous Coward

[dustman]

I think another interesting metric to look at would be cpu time used. If one of them took 90% CPU for 10 seconds, that would be a big winner imo.

reiser4 171.28s @ 30% CPU = 51.384s CPU
reiserfs 302.53s @ 16% CPU = 48.4048s CPU
ext3 319.71s @ 11% CPU = 35.1681s CPU
xfs 429.79s @ 13% CPU = 55.8727s CPU
jfs 470.88s @ 6% CPU = 28.2528s CPU

Speed is not of the essence

[mnmn]

There are other things about filesystems us sysadmins to know about. Which is the most stable and crashproof filesystem? Ive suspected it to be XFS from which I recovered data after doing dd if=/dev/zero of=/dev/hda1 count=2048576

Also what filesystem would require the lest or no syncing at all? Befs?

In server environements with stripped 15K cheetah SCSI drives, you'd worry more about stability than speed.

Re:Speed is not of the essence

[Molina the Bofh]

Pick one of the three answers:

With the new and improved Magnetic Forensics Tunnelling Electronic Microscope, used by the FBI

He had a backup

XFS takes so long to sync that by the time it would start to sync, he turned his computer off

Re:Speed is not of the essence

[mnmn]

I did a dd to dump the whole filesystem to a file in a larger filesystem for recovery purposes. The XFS tools that came with RedHat 7.1 seemed to crash so I did a small slackware install and went from there. xfs_restore and other tools sounding like xfs_sync or something found my third superblock. I used it to reconstruct the other superblocks. The root and early directories were gone, and the files I needed were in the root directory, well the xfs_restore linked them all to numbered directories and files. A few greps later I was in a directory that had those files all intact. I recovered around 80% of all the files, and 100% of the ones I was seeking. Just because I succeeded in one major recovery op from an XFS filesystem, I feel confident with it more than the ext3 or reiser guys. I havent really compared them.

Incomplete comparison?

[Futurepower(R)]

From the article:
"reiser4 171.28s, 30%CPU (1.0000x time; 1.0x CPU)
reiserfs 302.53s, 16%CPU (1.7663x time; 0.53x CPU)

What's interesting:
* reiser4 had highest throughput and most CPU usage"

The comparison seems incomplete to me. Reiser4 took about half the time, with twice the CPU usage. The

Total Work Done by the CPU = Percent * Time.

Reiser4 did the work in half the time, but the total work was roughly equal. Actually, ReiserFS was more efficient considering total CPU cycles.

Re:Incomplete comparison?

[makapuf]

Except that if you have a mostly idle CPU and your task is more & more waiting for the disk to complete, you don't care about 11 or 30 % if the other 89 or 70 % of the CPU are idle.

Comparing CPU cycles needed is NOT a fair benckmark, unless your task is CPU _AND_ IO Bound.(if it's not io bound, take whatever fs you have, it doesn't matter.)

The benchmark was right in giving results as TWO parameters : CPU used and time spent. Might be interesting to see how it depends of the drive type or the CPU arch, though)

Re:Incomplete comparison?

[p3d0]

I disagree. Consider:

• FS1: 1sec CPU time, 1sec IO time. CPU usage=50%.
• FS2: 2sec CPU time, 6sec IO time. CPU usage=25%.

FS1 is undoubtably better. It consumes less CPU and IO time. Yet noobs would complain about the high CPU usage of FS1. The truth is, CPU usage is higher only because IO is more efficient.

I can think of no corresponding pathological case in which CPU load would be a more appropriate predictor than CPU time.

if you have a mostly idle CPU and your task is more & more waiting for the disk to complete, you don't care about 11 or 30 % if the other 89 or 70 % of the CPU are idle.

Exactly. That's why CPU load doesn't matter.

Comparing CPU cycles needed is NOT a fair benckmark, unless your task is CPU _AND_ IO Bound

No, CPU cycles matters for CPU-bound tasks. I think that's pretty self-evident.

(if it's not io bound, take whatever fs you have, it doesn't matter.)

Wrong. In a compute-bound task, CPU time adds directly to the total execution time of the task, so you should choose the FS with the lower total CPU time.

Kerneltrap on new server..

[molo]

FYI, kerneltrap just moved to a new server (this week!). It used to be run on Jeremy's DSL line, which was why it would get shot to shit whenever it got slashdotted.

Now its in a colo on new (fast) hardware (paid for by users' donations), and upgraded to drupal 4.2 (faster here too).

So slashdot away!

-molo

Blah... This was on lkml...

[j.e.hahn]

And a number of people complained that it wasn't a great benchmark. Hans Reiser admitted it was just a quickie, and I forget who it was that said it, but ext3 has some performance enhancements that are on the cusp of merging into Linus' tree from the -mm kernels.

Wait until 2.6 is out folks. These numbers are still open for mass fluctuation.

Re:Stability will be fixed

[T-Ranger]

Speed problems can be solved by throwning hardware at the problem. Faster disks, more ram, more servers. A filesystem not desigined with stability in mind wont be stable regardless of the hardware.

Stability should be at the top of the list when desigining a filesystem. If its a quetsion of stability or speed, stability should win.

Oracle posted some stats

[Stone316]

With respect to filesystems and database performance. EXT3 came out on top.

CPU Usage?!

[metalmaniac1759]

30%-40% CPU Usage - whoa. What happens to poor me - with a PIII 550 MHz, 128 MB SDRAM and KDE running all the time.

I switched from RedHat to Gentoo and KDE stopped crawling. God knows what will happen with ReiserFS4.

Nandz.

quick question

[perlchild]

quick question, what mount options were used on each? Wouldn't e3fs appear slower than the others if it was data-journaling? (I read the article a bit fast, but I didn't see any "how we tested") Also wouldn't a good test take at least two hours on each filesystem?