Slashdot Mirror
Linux Filesystems Benchmarked
smatt-man writes "Over at Linux Gazette they ran some tests on popular Linux filesystems (ext2, ext3, jfs, reiserfs, xfs) and the results may surprise you."
← Back to Stories (view on slashdot.org)
← Back to Stories (view on slashdot.org)
The original article
I support the Center for Consumer Freedom
Not quite what I got from it. Ext2 was certainly faster for a lot of operations, but is, of course, not journalled. XFS and JFS were fast, but most importantly, when it came to large files, these two tended to really take the lead. XFS was particularly good at handling large files. Overall Ext3 was disappointingly slow surprisingly often.
Jedidiah.
Craft Beer Programming T-shirts
http://209.81.41.149/~jpiszcz/index.html
:)
it's not slashdotted yet
Ext3 provides a safe low-pain entry into the world of journaled file systems. No need to re-partition or reformat. It offers reasonably good performance plus the benefits of journalling.
Obviously (as you point out) a journallying filesystem is what you need. I went for Ext3 on my Debian servers. I/O throughput wasn't so important. The good thing about Ext3 is its backwards-compatibility with Ext2. If there's a problem and you don't have all the kernel modules or tools then you're still pretty much guarranteed access to the file system by mounting it as Ext2 as support for that system is almost universal under Linux.
No, in fact ext3 is one of the few that actually will journal data as well as metadata.
"The invisible and the non-existent look very much alike." -- Delos B. McKown
Here's a mirror of the article:l
http://www.gutenpress.org/links/LG/102/piszcz.htm
RFC2119
PNGs are smaller than GIFs, better compression, etc. (if you use same color-depth (8 bit)).
Move Sig. For great justice.
>Web site accessibility (use image type supported by all major browsers)
All the "good features" of GIF is supported by PNG in all current browsers. You'd have to go back in time fem years to find a browser that can't display a basic PNG. If you think otherwise, give me a link to one that matters that doesn't, and explain to me why, if it wasn't released/updated this year, using it isn't a security issue.
Since GIF doesn't support per-pixel-alpha to begin with, you lose nothing by using PNG for everything. After all, with GIF you didn't have the choice at all so there is no issue with simply "converting to PNG".
Score: PNG
>Bandwidth conservation
PNGs are always smaller where it matters (anything more complex than 1x1x1-images). In some not atypical cases a PNG can be 25% smaller than the corresponding GIF.
Score: PNG
PS. GIF-via-LZW is still encumbered in many countries.
More features, better standard, solid software, no licensing issues, smaller output == Winner: PNG
Belief is the currency of delusion.
name a "major browser" that won't support PNG. I don't know one. I use all PNG and have checked all my pages in enough "major" browsers to cover probably greater than 99% of people and they display fine.
...making Linux just a little more fun!
/dev/zero. /dev/null.
Benchmarking Filesystems
By Justin Piszcz
INTRO
I recently purchased a Western Digital 250GB/8M/7200RPM drive and wondered which journaling file system I should use. I currently use ext2 on my other, smaller hard drives. Upon reboot or unclean shutdown, e2fsck takes a while on drives only 40 and 60 gigabytes. Therefore I knew using a journaling file system would be my best bet. The question is: which is the best? In order to determine this I used common operations that Linux users may perform on a regular basis instead of using benchmark tools such as Bonnie or Iozone. I wanted a "real life" benchmark analysis. A quick analogy: Just because the Ethernet-Over-Power-Lines may advertise 10mbps (1.25MB/s), in real world tests, peak speed is only 5mbps (625KB/s). This is why I chose to run my own tests versus using hard drive benchmarking tools.
SPECIFICATIONS
HARDWARE
COMPUTER: Dell Optiplex GX1
CPU: Pentium III 500MHZ
RAM: 768MB
SWAP: 1536MB
CONTROLLER: Promise ATA/100 TX - BIOS 14 - IN PCI SLOT #1
DRIVES USED: 1. Western Digital 250GB ATA/100 8MB CACHE 7200RPM
2. Maxtor 61.4GB ATA/66 2MB CACHE 5400RPM
DRIVE TESTED: The Western Digital 250GB.
SOFTWARE
LIBC VERSION: 2.3.2
KERNEL: linux-2.4.26
COMPILER USED: gcc-3.3.3
EXT2: e2fsprogs-1.35/sbin/mkfs.ext2
EXT3: e2fsprogs-1.35/sbin/mkfs.ext3
JFS: jfsutils-1.1.5/sbin/mkfs.jfs
REISERFS: reiserfsprogs-3.6.14/sbin/mkreiserfs
XFS: xfsprogs-2.5.6/sbin/mkfs.xfs
TESTS PERFORMED
001. Create 10,000 files with touch in a directory.
002. Run 'find' on that directory.
003. Remove the directory.
004. Create 10,000 directories with mkdir in a directory.
005. Run 'find' on that directory.
006. Remove the directory containing the 10,000 directories.
007. Copy kernel tarball from other disk to test disk.
008. Copy kernel tarball from test disk to other disk.
009. Untar kernel tarball on the same disk.
010. Tar kernel tarball on the same disk.
011. Remove kernel source tree.
012. Copy kernel tarball 10 times.
013. Create 1GB file from
014. Copy the 1GB file on the same disk.
015. Split a 10MB file into 1000 byte pieces.
016. Split a 10MB file into 1024 byte pieces.
017. Split a 10MB file into 2048 byte pieces.
018. Split a 10MB file into 4096 byte pieces.
019. Split a 10MB file into 8192 byte pieces.
020. Copy kernel source tree on the same disk.
021. Cat a 1GB file to
NOTE1: Between each test run, a 'sync' and 10 second sleep were performed.
NOTE2: Each file system was tested on a cleanly made file System.
NOTE3: All file systems were created using default options.
NOTE4: All tests were performed with the cron daemon killed and with 1 user logged in.
NOTE5: All tests were run 3 times and the average was taken, if any tests were questionable, they were re-run and checked with the previous average for consistency.
CREATING THE FILESYSTEMS
(snipped. too many junk characters0
BENCHMARK SET 1 OF 4
In the first test, ReiserFS takes the lead, possibly due to its balanced B-Trees. (If the images are hard to read on your screen, here's a tarball containing larger versions of them.)
All of the files systems faired fairly well when finding 10,000 files in a single directory, the only exception being XFS which took twice as long.
Both ext versions 2 and 3 seem to reap the benefits of removing large numbers of files faster than any other file system tested.
To make sure this graph was accurate; I re-benchmarked the ext2 file system again and got nearly the same results. I was surprised to find how much of a performance hit both ext2 and ext3 take during this test.
Finding 10,000 files seemed to be the same except for XFS; however directories are definitely handled dif
Redhat 9 and Fedora Core 1 both ship with JFS support -- the graphical installer, however, does not offer it as an option.
So, what I usually do when installing a new copy of Fedora or Redhat is to drop to a console, and use fdisk + mkfs.jfs manually. Then, when I get to the right page in the installer, I can simply set it to not reformat the partition but to use it as the "/" mount point, and voila -- my computer has JFS.
There are options, or settings, that you can do for ext3, the default is slower, but it saves your data. Ext3 not only journals metadata, like XFS, etc... but it also journals data, which is the only filesystem that does that, if I understand this correctly.
/etc/fstab and reboot, no big deal.
"data=writeback" mode does no data journaling, only metadata journaling, and you would probably see better performance here. Although, you could lose data in the event of a power outage (no fun). Same thing applies to XFS, JFS - you could lose data because only metadata is being journaled, not real data.
"data=ordered" mode - inbetween, still no data journalling, but there are provisions that make it less likely to lose data in the case of a power problem. It has something to do with the way it journals the metadata and the way the filesystem interacts with the disk that makes is a little slower than data=writeback but also a little more secure than data=writeback if you get a power outage.
"data=journal" mode - this journals data and metadata, and with the exception of a few situations, is the slowest. The least likely to lose your data, but also much slower.
I am assuming, or at least it looks like, these tests were run with the default data=journal - so to be fair, they should have been run in data=writeback, or maybe even all three modes. Again, all you have to is specify in
It would probably be better to compare the ext3 in data=writeback mode.
My understanding is that journalled means the FS can't get into an inconsistant state and so does not need fsck-ing. It does not mean your data is safe from having the power pulled halfway through a write. If you want a super-safe home area you want some sort of logging FS and these are all far slower than journalled (I think).
IE is still too stupid to properly do an alpha channeled PNG. But it does do 1-bit, GIF-style transparency and displays generic, non-transparent PNGs just fine. And so the only place left to use GIF for is crummy animations.
Ita erat quando hic adveni.
I did a bunch of tests like this, but in 2.6 instead of 2.4. My conclusions:
* Ext2 is still overall the fastest but I think the margin is small enough that a journal is well worth it
* Ext3, ReiserFS, and XFS all perform similarly and almost up to ext2 except:
o XFS takes an abnormally long time to do a large rm even though it is very fast at a kernel `make clean`
o ReiserFS is significantly slower at the second make (from ccache)
* JFS is fairly slow overall
* Reiser4 is exceptionally fast at synthetic benchmarks like copying the system and untaring, but is very slow at the real-world debootstrap and kernel compiles.
* Though I didn't benchmark it, ReiserFS sometimes takes a second or two to mount and Reiser4 sometimes takes a second or two to unmount while all other filesystem's are instantaneous.
Whole thing available here
Just in case people want to read the article and dont have 30 minutes for it to load I put up a mirror here ...
Note to Mods: When I post mirrors, it's a best guess. I don't know for certain whether or not the site will go down!
ext3 does NOT journal data by default, it only "orders" it (data=ordered). This is still better than what any of the other filesystems do. You have to explicity define data=journal in fstab to get full data journaling. There is a performance hit but it's what I use on all my RAID5 data filesystems and it's great. I used to use reiserfs until a huge crash corrupted massive amounts of files and directories. I don't think I'll be going without data journaling protection for a while.
Not to mention mountable under Windows... for those of us that still need that. Or rather, EXT2 is mountable under Windows (with a 3rd party daemon), and thus EXT3 can be read as well.
1) I often find PNGs to be smaller than GIFs.
2) Who cannot see PNGs? IE supports them, Opera supports them, Netscape/libpr0n-based browsers all support PNGs. Hell, even Links 2 when run in X or svgalib supports PNG.
So, what I usually do when installing a new copy of Fedora or Redhat is to drop to a console, and use fdisk + mkfs.jfs manually
You're making things unnecessarily complicated. At the "boot:" prompt, just type "linux jfs". The graphical installer will then offer it as an option. Works with reiserfs, too.
More precisely, the PNG need to be in indexed mode (aka PNG8) for full transparency to work in IE. In The GIMP, click the "Image" menu, "Mode", "Indexed".
Some myth ("IE don't support PNG !!!") really die hard.
:wq
"Overall Ext3 was disappointingly slow surprisingly often."
I disagree, plus this test is obsolete, why did he use a 2.4 kernel?!
from: http://freshmeat.net/projects/linux/?branch_id=463 39&release_id=160407
"Linux 2.6.6
...ext2 and ext3 filesystem performance was significantly improved.
"
[...]
Changes:
And thats just from today's kernel release. What about all the changes between 2.4 and now?
Considering the conveniance of backward compatability, and the fact that ext3 wasn't the worst in every category, and it looks like maybe uses less cpu than some, it seems like ext3 is the hands-down winner of the test, not the looser. ext3 did as well in tests that IMO represent everyday use. Who creates 10k files in a folder? I would have liked to see a linux kernel COMPILE using the fs. Thats something we all could appreciate.
"In a time of universal deceit, telling the truth becomes a revolutionary act" - George Orwell
Reiserfs can at least be accessed under Windows.
My personal peeve with ReiserFS is, though, that I've had the main ReiserFS partition on my Laptop completely destroyed by a simple power failure once. Many files were in lost+found afterwards, but some had their contents destroyed. (And restoring files by looking at their contents is not fun for ~1000 files...) So I've kinda lost trust in it. ext3 may be slow, but I've never had a single problem with it. Seems very robust to me. So, reliability is more important than speed for me (whoever needs performant servers is of course entitled to a different opinion). XFS and JFS seemingly can't be accessed from Windows, so that is a Minus for some.
Here's some real world information on the state of large file support in 2004. Filesystem driver support is the least of your worries -- almost any linux filesystem you can think of (except for maybe umsdos) supports over 2GB files at the kernel level. The Linux LFS page, dated April 2004, contains reasonably updated information on large file support in linux.
The bigger problem is that many userspace applications cannot yet read or write to the large files. This failure arises from non-use of the LFS API combined with just plain unfortunate use of a signed 32-bit int in the wrong place. So for instance mkisofs will reject all input files larger than 2GB in size, and cdrdao will simply abort at 2GB if you try to rip a DVD larger than 2GB in size. In some extreme cases there are programs that can't even handle large files off of the disk; one example is
wget http://mirror.linux.duke.edu/pub/fedora/linux/core /test/1.92/i386/iso/FC2-test3-i386-DVD.iso
which fails spectacularly on any x86 linux system (hint: the DVD is not really 84MB in size). In general, the "core" system utilities such as dd, cp, mv, cat are fully compatible with large files whereas third party applications are much more hit-or-miss.
Even today, by far the most practical solution to large file woes is to migrate to a 64-bit system, the most affordable of which is AMD64 by a long shot. I've been using an Athlon 64 system for the past few weeks and it has handled large files perfectly in all respects so far.