Real-World Benchmarks of Ext4

Ashmash writes "Phoronix has put out a fresh series of benchmarks that show the real world performance of the Ext4 file-system. They ran 19 tests on Fedora 10 with changing out their primary partition to test Ext3, Ext4, Xfs, and ReiserFS. The Linux 2.6.27 kernel was used with the latest file-system support. In the disk benchmarks like Bonnie++ Ext4 was a clear winner but with the real world tests the results were much tighter and Xfs also possessed many wins. They conclude though that Ext4 is a nice upgrade over Ext3 due to the new features and just not improved performance in a few areas, but its lifespan may be short with btrfs coming soon."

Comments

ext2?

[jon3k]

What, no ext2 comparison? seems like a pretty glaring omission.

Re:ext2?

[Hal_Porter]

You joke but fat with big clusters is pretty efficient for media applications. It's easy to get a good cache hit rate on the FAT cache.

E.g consider a FAT32 filesystem reading from a contiguous file. You have 512 bytes sectors 32K clusters. You have a one sector buffer for FAT data in the filesystem and a cluster sized buffer to read ahead data in the application. Each read of the FAT tells you where 128 clusters are. So you read a sector of FAT and then you can read 128 data clusters (4MB) before you need to do any metadata access. That's a very low overhead. There are no inodes to be updated, no atime and no bitmap, just the FAT, data clusters and directory entries. You need to update the time in the directory entry only once when you close the file if it has been written.

A small amount of code can get read speeds that are close to the raw read speed of the device for a contiguous file because you spend a small amount of time on bookkeeping.

Of course directories aren't indexed but lots of applications don't have vast numbers of files in a directory. In any case reading ahead as far as possible and doing a linear search is probably quicker than doing a B tree access which involves moving the hard disk head around. Even fragmentation doesn't introduce much overhead, you'd just have to reload you FAT buffer from a different place in the FAT each time you cross into a new fragment. Traditionally inode based fiesystems are worse because you have multiple levels of indirection to find the data blocks. Ext4 uses extents instead but a FAT implentation could keep track of extents in Ram so reading a big contiguous file would require FAT reads only until the filesystem works out that the file is one big extent.

If you have large directories it slows down a bit but you can always cache the mapping from files to clusters.

Most people running inode based filesystems with atime updates on, the default, probably have a filesytem which is less efficient than FAT. On Windows for example NTFS is slower than FAT in the default config with atime updates on. Kind of embarassing given that NTFS is much more high tech filesystem with Btrees fo directories and extent lists in the inodes to track data blocks.

Of course NTFS has a logfile but that only protects metadata. FAT has bit in the first FAT entry to mark the volume dirty, the idea is that you set it before you write and clear it when you're done. If you mount a filesystem with the bit set you need to do a chkdsk. Chkdsking means reading all the directories and making sure that the allocations match the FAT. It's slower than a logfile based chkdsk but it will fix corrupt metadata, mostly by freeing lost clusters. You wouldn't want to boot your OS from it, but it's good for play MP3s or AVIs from. It's also incredibly widely supported - pretty much any PC OS, games console or media player will handle it. And it works for drives of up to 2TB. Really the only problem is 4GB max file size.

Re:ext2?

[Hal_Porter]

You could extend FAT32 to handle bigger drives. The 2TB limit comes from the fact the the volume size is limited to 2^32 sectors. With a hard disk that's 2TB.

One possibility would be to add a 32 bit volume size in clusters field to the FSInfo sector which already contains the first free cluster and the free cluster count. FAT32 has the upper 4 bits in a FAT entry marked as reserved which limits you to 2^28 clusters. You bump the filesystem version in the bootsector and use all of the bits in a FAT entry. With those two changges you could have 2^32 clusters which is 256TB with 64K clusters.

It's the same with the 4GB limit. You could use one of the spare bytes in the directory entry to have more bits of filesize. Some Doses do this and call it FAT+

Mind you, the reason people use FAT32 is because it is supported by everything. If you did either of these things you'd end up with a file system which wasn't supported by anyone. Old implementations would either corrupt volumes which were more than 2TB or had files bigger than 4GB or fail to mount them.

Now Microsoft have something called exFAT, a completely new filesystem which is incompatible with FAT32 and patented so it's not really in their interests to keep adding features to FAT32 which is now more or less open. At least I don't think many people paid them royalties and they haven't sued anyone to get them.

Short lifespan? I don't think so.

[Viol8]

Admins tend to stick with what they know and ext4 is a natural progression from ext3. btrfs however hasn't even reached version 1.0 yet - and to be honest who is going to want to use a 1.0 release anyway on something as fundamental as a filesystem? Also its development is being done by an Oracle team , albeit FOSS , which may put a few people off.

My prediction for what its worth is that ext4 will be around for a LONG time.

Re:Short lifespan? I don't think so.

[IceCreamGuy]

My prediction for what its worth is that ext4 will be around for a LONG time.

Like... how long? Longer than it takes to fsck an 80GB ext2 filesystem? Because that's a pretty long time.

Re:Short lifespan? I don't think so.

[juan2074]

You don't have to fsck btrfs (Butterface)?

Re:Short lifespan? I don't think so.

[diegocgteleline.es]

Hey, ext4 actually speeds up fsck quite a lot, so maybe ext4 won't be around a lot of time ;)

Re:Short lifespan? I don't think so.

[Azar]

I was wondering why it was omitted from this article as well. I believe that it's because JFS just doesn't seem to have the mindshare of ext3, XFS, or even ReiserFS. After reading various filesystem comparisons, I chose it as my FS and I have been using it for over a year without a single issue. I don't have to worry about long fsck times at reboot, my CPU has less load when deleting or copying a large volume of files (virtual machines or CD/DVD isos usually), never had any file loss or corruption, and it seems to complete large file operations quicker than ext3 (what I used previously). There are some things that ext3 does better than JFS, but overall I prefer the advantages of JFS over the advantages of ext3.

I know that if I were to rebuild an older computer that had lower specs, or perhaps a set-top box like MythTV that I wanted to be more power effecient then JFS is going to be my choice for the filesystem.

Re:Short lifespan? I don't think so.

[MrNemesis]

I've used JFS extensively for several years now, including power outages when the discs were under load - and I've never had anything fail to correctly fsck when the power came back up. fsck is also far, far faster than ext3 or XFS and CPU loading is *way* lower than XFS, whilst maintaining comparable throughput. I don't like XFS for the simple reason that if you don't have bulletproof power you will be restoring from backup. And when you've been in the job for as little as a few years you soon come to realise that *nothing* is bulletproof. Even triple-UPSed, six diesel generators, four seperate power lines and a five nines SLA with your data centre are no protection against a hung-over techie pulling the power cables from the wrong server.

JFS also gets ACL access "for free"; if you use it like I use it, as the backed for a Samba server (~300 users) with a complicated ACL structure, JFS is much faster. Last time I fiddled with ACL's under ext3, I found they introduced a small, but percetible, increase in filesystem latency - don't know if this has been fixed, but using ACLs under JFS incurs no penalties, at least as far as my testing goes [citation needed]

IMHO JFS is one of the hidden gems of Linux - fast under varying workloads, robust, frags up nowhere near as badly as ext2/3 when getting full, incredibly simple... the only downsides I'm aware of are that you can't shrink it and the fact that no-one seems to "support" it.

One filesystem to rule them all...

[INeededALogin]

and in the darkness... bind them.

Seriously... one of the nice things about Windows, OSX, Solaris is that they get a new filesystem once every 5-10 years. The safest thing to do for Linux is to be a generation behind. I would not run ext4 until btrfs came out. Why be the admin that gets screwed with early bugs and future incompatibilities...

Re:One filesystem to rule them all...

[fracai]

See, I already thought of that, so I run no fewer than 5 generations behind.

I started at 1, but realized that soon this practice would become widespread and then I'd be back to being an early adopter. So I moved to 2 generations. But then a friend agreed with my plan and I saw that in not too very long I'd be an early adopter again with my 2 gen old system. Not this time! I skipped the 3 and 4 generation delay and went right to the 5 generation wait time. I figured it was the only way to be sure I wouldn't get hit by any bugs.

Shoot, now the secret's out. Time to roll back my filesystem again.

Re:One filesystem to rule them all...

[The MAZZTer]

Windows hasn't had a new filesystem that recently! NTFS was introduced in 1993. Windows has been using it for the past 15 years (if you only count the NT line, the 9x/ME line never even supported it IIRC, they just used the even older FAT32).

Of course if you want to count the small extensions added on with each windows version then your claim about Windows is correct. Still I wouldn't be surprised if Windows Vista filesystems mount inside NT 3.1. I should test this with a VM...

Re:One filesystem to rule them all...

[El Lobo]

Sure the NTFS used by NT4 and the one Vista uses share the same name (and are *somehow* compatible: Vista understands the old NTFS and NT4 can use the new one in a limited way, but there is A LOT under the hood. The way security descriptors work, for instance is completely different in new versions. volume Shadow Copy, Hierarchical Storage Management, Junction Points, and other "extensions" are a HUGE step forward, and made the new NTFS in reality a new version, with the same old name.

Re:One filesystem to rule them all...

[TheRaven64]

NTFS is the result of the requirements team taking too long to decide what a filesystem should do. The filesystem team on the original NT project couldn't wait for them to decide, so they produced something very simple that just stored mappings from names to two kinds of data, big and small values. Big values are stored as one or more disk blocks, small values are stored in the Master File Table (MFT).

On top of this is a metadata layer, where certain types of name are interpreted in different ways. This was used to add ACLs, encryption, compression, copy-on-write semantics, and so on.

The low-level format for a Vista NTFS partition is the same as for NT3.51, but each version of NT has added a few more types of metadata and interpretations of them, meaning that writing to a disk with an older version is likely to cause data corruption.

Re:One filesystem to rule them all...

[compro01]

Pointy stick and a large beach.

Just be aware of occasional data loss when the tide_in function gets called.

Re:One filesystem to rule them all...

[Vancorps]

You think validating the integrity of a filesystem is easy??!?!?

That's insane, first of all, you won't know how it performs unless you give it real world usage complete with disk failures. There are hundreds of file systems which can store data but how they handle problems are what separates most of them. Of course there are other distinctions but the failure mode scenarios are what most interest an admin as failure is never a question of it, only a question of when. Simulating certain failure modes is exceedingly difficult to do.

Re:butterfs

[IceCreamGuy]

It's actually B-Tree Filesystem, and according to Wikipedia it really is pronounced "Butter FS," however I really was just referring to the last time it was mentioned on /. and there were like 200 comments all on how they thought btrfs would be pronounced.

Re:butterfs

[CRCulver]

That's the new name for reiserfs.

Re:short lifespan? The big distros will decide.

[INeededALogin]

it just doesn't matter THAT much for most applications

well... run an fsck against ext2 and ext3 and tell me it doesn't matter. For an admin, speed, reliability, recoverability... are all major concerns. On Solaris, I love ZFS because of the functionality like snapshots and exports. I also got burned by the IDE/100% CPU driver bug on Sparc hardware. Admins need to be aware of what they are running and what limitations exist. I honest don't give a damn about mp3 encoding speed, but the capabilities and maturity of a filesystem have to be considered.

ReiserFS

[thomasj]

ReiserFS used to be the killer FS, but now it seems like it is stuck. But I shall not be the judge of that, though there seems to be some truth buried in it somehow. And not to mention, the next release is probably more than a few years down the road.

Re:ReiserFS

[lwsimon]

I'd guess the next release of ReiserFS is 25 years to life down the road, actually.

Dumbest benchmarks ever

[Nick Ives]

I see no analysis as to why the filesystems perform the way they do. Why does XFS perform so well on a 4GB sequential read and so badly on an 8GB read? Why did they include cpu / gfx bound frame/sec benchmarks? In the few application benchmarks where there was more than a tiny fraction of percent difference there's no discussion as to whether that difference is actually significant.

Not at all enlightening.

Re:Dumbest benchmarks ever

[quintesse]

You must be new here

Next time check his /. ID first ;)

Stupid testing methodology

[afidel]

WTF who measures things like MP3 compression time when testing a filesystem?!? As far as I can tell they only ran one real I/O test and that was the Intel IOMeter based fileserver test which showed EXT4 is really fast for that profile. I would have loved to have seen the DB profile run. Their other artificial tests could have been summed up by running the streaming media profile since they were just large contiguous reads and writes.

Re:Stupid testing methodology

[ChrisA90278]

who measures things like MP3 compression time when testing a filesystem?!?

They were measuring what matters: "System performance" It very well could have been the case that the fastest file system, measured on a simple bench mark gives the worst MP3 compression time. Let's say the reason the filesystem is fast is because it uses a huge amount of CPU time and RAM. So a RAM based encrypted file system might be very fast, until you run an application on it.

It's a reasonable test and what it showed is that in the real-world performance is about the same

Re:butterfs

[MightyYar]

My nly prblm wth btrfs s tht t dsn't sv any vwls.

Useless.

[Steve Baker]

What's with the CPU/Video tests? How about some more random access pattern tests, DB/web/streaming media tests? How about showing CPU utilization in addition to I/O performance?

Re:butterfs

[Chris Mattern]

f u cn rd ths, u cn gt a gd jb n cmptr prgmmng.

Phoronix is worthless

[Rendus]

Remember, this is the site that's decided that Ubuntu 7.04 is twice as fast as any other version of Ubuntu. Take what they say with a good healthy dose of skepticism.

Re:butterfs

[noob749]

hrm, i prefer ZFS (or as it is affectionately known - 'icantbelieveitsnotbttrfs')

Re:butterfs

[MightyYar]

It's still in development... obviously not ready for production use.

The real problem with reiserfs

[TheLink]

The real problem with reiserfs: vendor lock-in.

Re:butterfs

[Just Some Guy]

I like the fact that they are naming filesystems after characters in South Park.

I like the fact that they're naming filesystems after common household items that have existed for millenia. In South Park, "Butters" is plural.