Slashdot Mirror

← Back to Stories (view on slashdot.org)

Real-World Benchmarks of Ext4

Posted by CmdrTaco on from the but-i-live-in-a-fake-world dept.

Ashmash writes "Phoronix has put out a fresh series of benchmarks that show the real world performance of the Ext4 file-system. They ran 19 tests on Fedora 10 with changing out their primary partition to test Ext3, Ext4, Xfs, and ReiserFS. The Linux 2.6.27 kernel was used with the latest file-system support. In the disk benchmarks like Bonnie++ Ext4 was a clear winner but with the real world tests the results were much tighter and Xfs also possessed many wins. They conclude though that Ext4 is a nice upgrade over Ext3 due to the new features and just not improved performance in a few areas, but its lifespan may be short with btrfs coming soon."

195 of 249 comments (clear)

  1. ext2? by jon3k · · Score: 5, Funny

    What, no ext2 comparison? seems like a pretty glaring omission.

    1. Re:ext2? by Neotrantor · · Score: 3, Funny

      hey, why not a fat16 bench as well?

    2. Re:ext2? by Hyppy · · Score: 2, Insightful

      Modded funny, but ext2 still has its uses. Journalling can be a bit of a performance breaker in the right (wrong) circumstances.

    3. Re:ext2? by Hal_Porter · · Score: 5, Informative

      You joke but fat with big clusters is pretty efficient for media applications. It's easy to get a good cache hit rate on the FAT cache.

      E.g consider a FAT32 filesystem reading from a contiguous file. You have 512 bytes sectors 32K clusters. You have a one sector buffer for FAT data in the filesystem and a cluster sized buffer to read ahead data in the application. Each read of the FAT tells you where 128 clusters are. So you read a sector of FAT and then you can read 128 data clusters (4MB) before you need to do any metadata access. That's a very low overhead. There are no inodes to be updated, no atime and no bitmap, just the FAT, data clusters and directory entries. You need to update the time in the directory entry only once when you close the file if it has been written.

      A small amount of code can get read speeds that are close to the raw read speed of the device for a contiguous file because you spend a small amount of time on bookkeeping.

      Of course directories aren't indexed but lots of applications don't have vast numbers of files in a directory. In any case reading ahead as far as possible and doing a linear search is probably quicker than doing a B tree access which involves moving the hard disk head around. Even fragmentation doesn't introduce much overhead, you'd just have to reload you FAT buffer from a different place in the FAT each time you cross into a new fragment. Traditionally inode based fiesystems are worse because you have multiple levels of indirection to find the data blocks. Ext4 uses extents instead but a FAT implentation could keep track of extents in Ram so reading a big contiguous file would require FAT reads only until the filesystem works out that the file is one big extent.

      If you have large directories it slows down a bit but you can always cache the mapping from files to clusters.

      Most people running inode based filesystems with atime updates on, the default, probably have a filesytem which is less efficient than FAT. On Windows for example NTFS is slower than FAT in the default config with atime updates on. Kind of embarassing given that NTFS is much more high tech filesystem with Btrees fo directories and extent lists in the inodes to track data blocks.

      Of course NTFS has a logfile but that only protects metadata. FAT has bit in the first FAT entry to mark the volume dirty, the idea is that you set it before you write and clear it when you're done. If you mount a filesystem with the bit set you need to do a chkdsk. Chkdsking means reading all the directories and making sure that the allocations match the FAT. It's slower than a logfile based chkdsk but it will fix corrupt metadata, mostly by freeing lost clusters. You wouldn't want to boot your OS from it, but it's good for play MP3s or AVIs from. It's also incredibly widely supported - pretty much any PC OS, games console or media player will handle it. And it works for drives of up to 2TB. Really the only problem is 4GB max file size.

      --
      echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
    4. Re:ext2? by jgtg32a · · Score: 2, Insightful

      Yup, I use it for the /boot partition

    5. Re:ext2? by Microlith · · Score: 2, Interesting

      And it works for drives of up to 2TB. Really the only problem is 4GB max file size.

      I had forgotten about those two particular limitations of Fat32 and it struck me when you mentioned them that we're about to run across the first. I had personally hit the 4GB limit a while back but at the rate things are going, before 2009 is out we'll see disks released that a single Fat32 partition doesn't support.

      To think that when it was released with Windows95, people saw the 2TB limit and committed the first sin of computing technology: assuming an upper bound is beyond reach.

    6. Re:ext2? by Kent+Recal · · Score: 1

      In fact, journaling is a performance breaker in pretty much every i/o intensive scenario, such as database servers.
      Ext2 is still the preferred choice on servers here.

    7. Re:ext2? by Hal_Porter · · Score: 4, Interesting

      You could extend FAT32 to handle bigger drives. The 2TB limit comes from the fact the the volume size is limited to 2^32 sectors. With a hard disk that's 2TB.

      One possibility would be to add a 32 bit volume size in clusters field to the FSInfo sector which already contains the first free cluster and the free cluster count. FAT32 has the upper 4 bits in a FAT entry marked as reserved which limits you to 2^28 clusters. You bump the filesystem version in the bootsector and use all of the bits in a FAT entry. With those two changges you could have 2^32 clusters which is 256TB with 64K clusters.

      It's the same with the 4GB limit. You could use one of the spare bytes in the directory entry to have more bits of filesize. Some Doses do this and call it FAT+

      Mind you, the reason people use FAT32 is because it is supported by everything. If you did either of these things you'd end up with a file system which wasn't supported by anyone. Old implementations would either corrupt volumes which were more than 2TB or had files bigger than 4GB or fail to mount them.

      Now Microsoft have something called exFAT, a completely new filesystem which is incompatible with FAT32 and patented so it's not really in their interests to keep adding features to FAT32 which is now more or less open. At least I don't think many people paid them royalties and they haven't sued anyone to get them.

      --
      echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
    8. Re:ext2? by doom · · Score: 1

      Using ext2 for /boot is an older practice, from the days when a recovery disk (e.g. BBC) might not have any other file-system.

      An obsolete practice since knoppix, I'd say.

    9. Re:ext2? by Anonymous Coward · · Score: 3, Interesting

      WTF man I can't believe I assembled and executed your signature ... damnit

    10. Re:ext2? by Atti+K. · · Score: 2, Interesting

      In fact, journaling is a performance breaker in pretty much every i/o intensive scenario, such as database servers. Ext2 is still the preferred choice on servers here.

      Ext2 itself is kind of a performance breaker :)

      Don't get me wrong, I like ext2/3, I use only ext3 on all my machines and other machines I install, it's the only Linux fs I really trust. (back then when Suse defaulted to install reiserfs, I always changed that :) But we have to admit that it's not the best-performing fs on Linux.

      --
      .sig: No such file or directory
    11. Re:ext2? by Samah · · Score: 1

      At least it's not FATX (Xbox filesystem).
      Given the 64-byte directory entry size, the file PATH must be <= 42 characters.
      (Feel free to correct/augment these details, I just know that if you want to upload a file with a long name, you need to move it up one or more subdirectories).

      --
      Homonyms are fun!
      You're driving your car, but they're riding their bikes there.
    12. Re:ext2? by dbIII · · Score: 1

      It's a 2GB limit. I hit it every week since external disk drives are usually still formatted as fat32 by default.

    13. Re:ext2? by collinstocks · · Score: 3, Informative

      Can't you tell by reading it that it's a fork bomb? I mean, any idiot should be able to figure that out! :-P

      global _start
      _start:
      mov eax, 2 ; sys_fork
      int 80h ; execute call
      jmp _start ; unconditional jump to _start

      Get it???

      (Disclaimer: I didn't really expect you to know what it did just by looking at it. I didn't. I had to look up the system call on Google. Just don't go executing code if you don't know what it does.)

    14. Re:ext2? by Hal_Porter · · Score: 1

      Depends what you mean by heavily used. I have a bunch of hard disks with MP3s ripped from CDs as well as zipped file backups. The one here looks like this

      Total files = 16,784
      Average file size = 4,555 KB
      Total fragmented files = 3
      Total excess fragments = 19
      Average fragments per file = 1.00

      Fragments File Size
      2 960 KB
      2 10,878 KB
      18 297 MB

      The 297MB file is the only heavily fragmented one, it has 18 fragments of ~16MB each, the fragmentation on the other files is not going to be noticable. And actually those files would be fragmented on any filesystem - they are fragmented because they have been extended multiple times.

      Still with big clusters you're guarantee that cluster size bytes at least are contiguous. In this case the fragments are much bigger than that.

      Plus think of what would happen in a traditional inode filesystem to a 297MB file. That's big enough to take you into using triple indirect blocks

      http://www.rwalker.nildram.co.uk/work/os/inodes.html

      Triple indirect blocks will cause horribly non local disk access.

      --
      echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
    15. Re:ext2? by dmsuperman · · Score: 1

      I know, what was that about?

      On my flash-based N800, I use an Ext2 partition. Journalling just isn't as important to me, and in fact the extra read/writes will shorten the life of the flash memory.

      What other advantages does ext3 have? I've always been told _just_ journalling.

      --
      :(){ :|:& };: Go!
  2. butterfs by IceCreamGuy · · Score: 1

    Begin discussion revolving around what you think btrfs sounds like... again
    butter file system
    butterface
    butt file system
    etc...

    1. Re:butterfs by Rinisari · · Score: 2, Funny

      u = e?

      better filesystem?

      I was being facetious, but I may actually be correct.

      --
      Colin Dean Go a year without DRM
    2. Re:butterfs by IceCreamGuy · · Score: 5, Informative

      It's actually B-Tree Filesystem, and according to Wikipedia it really is pronounced "Butter FS," however I really was just referring to the last time it was mentioned on /. and there were like 200 comments all on how they thought btrfs would be pronounced.

    3. Re:butterfs by Directrix1 · · Score: 3, Funny

      I vote for butt rape filesystem.

      --
      Occam's razor is the blind faith in the natural selection of least resistance and in universal oversimplification. -- EF
    4. Re:butterfs by CRCulver · · Score: 4, Funny

      That's the new name for reiserfs.

    5. Re:butterfs by Windows_NT · · Score: 1

      Ever met a butterface?
      Everything was nice about her ... butterface ;)

      --
      Go go Gadget Nailgun!
    6. Re:butterfs by MightyYar · · Score: 5, Funny

      My nly prblm wth btrfs s tht t dsn't sv any vwls.

      --
      W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
    7. Re:butterfs by Daimanta · · Score: 1

      "butter file system"

      I love butter file system and I run it together with a hacked up version of "toast file system (tofsys)"

      --
      Knowledge is power. Knowledge shared is power lost.
    8. Re:butterfs by MMC+Monster · · Score: 3, Informative

      I like the fact that they are naming filesystems after characters in South Park. It's old-school programing, where you don't care what the end user thinks.

      --
      Help! I'm a slashdot refugee.
    9. Re:butterfs by sohp · · Score: 1

      At last, something to put on my flying toasters! Do you btrfs will handle the requirements of Video Toaster?

    10. Re:butterfs by Chris+Mattern · · Score: 4, Funny

      f u cn rd ths, u cn gt a gd jb n cmptr prgmmng.

    11. Re:butterfs by operagost · · Score: 1

      With current products being named "Lame" and "Gimp," sounds like old-school is the new new-school.

      --

      Gamingmuseum.com: Give your 3D accelerator a rest.
    12. Re:butterfs by Directrix1 · · Score: 1

      I'm already gainfully emloyeed thanks. Why did you use vowels for the words 'you' and 'a'

      --
      Occam's razor is the blind faith in the natural selection of least resistance and in universal oversimplification. -- EF
    13. Re:butterfs by tuxgeek · · Score: 1

      I vote for butt rape filesystem.

      What's on your mind today?

      --
      "Suppose you were an idiot...and suppose you were a member of Congress...but I repeat myself." Mark Twain
    14. Re:butterfs by dunkelfalke · · Score: 2, Funny

      i think it sounds rather solid, like a BTR file system

      --
      "It's such a fine line between stupid and clever" -- David St. Hubbins, Spinal Tap
    15. Re:butterfs by SupplyMission · · Score: 1

      Oh, thank you, but I think we all got the joke before you went ahead and explained it.

    16. Re:butterfs by Windows_NT · · Score: 1

      Thats ok, I'm only here to amuse myself.

      --
      Go go Gadget Nailgun!
    17. Re:butterfs by Ragzouken · · Score: 1

      Surely it'd either be butterfuss/bitterfuss, or Bee Tee Are Eff Ess, not pronouncing btr as it's spelt but then expanding FS to file system?

    18. Re:butterfs by dpilot · · Score: 1

      Unfortunately I tend to think of it as "Beater Eff Exx". You know, like the old car you drive in the winter, so your good car never gets near the road salt. Of course for some of us, our good summer car is a beater, too.

      --
      The living have better things to do than to continue hating the dead.
    19. Re:butterfs by Barsteward · · Score: 2, Interesting

      nah, thats R-ICE-HER-FS

      --
      "The hands that help are better far than lips that pray." - Robert Ingersoll (1833-1899)
    20. Re:butterfs by noob749 · · Score: 5, Funny

      hrm, i prefer ZFS (or as it is affectionately known - 'icantbelieveitsnotbttrfs')

    21. Re:butterfs by MightyYar · · Score: 4, Funny

      It's still in development... obviously not ready for production use.

      --
      W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
    22. Re:butterfs by Directrix1 · · Score: 2, Funny

      Porn.

      --
      Occam's razor is the blind faith in the natural selection of least resistance and in universal oversimplification. -- EF
    23. Re:butterfs by cp.tar · · Score: 2, Funny

      Unfortunately I tend to think of it as "Beater Eff Exx".

      Now waiting for wfbtrfs, i.e. the wife-beater FS.

      --
      Ignore this signature. By order.
    24. Re:butterfs by Faw · · Score: 2, Funny

      Now waiting for wfbtrfs, i.e. the wife-beater FS.

      Isn't that ReiserFS?

    25. Re:butterfs by Arthur+Grumbine · · Score: 1

      How about the more phonetic "butt orifice". Or if you're pronouncing each letter: Beady orifice. I don't know what that is, but I don't trust it...

      --
      Now that I think about it, I'm pretty sure everything I just said is completely wrong.
    26. Re:butterfs by SillyWilly · · Score: 1

      Same r

      --
      Online & Feelin' Fine
    27. Re:butterfs by MightyYar · · Score: 1

      Seems like you need to look up a different word.

      Removing the Ys would make the text unreadable. The a is a mistake.

      --
      W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
    28. Re:butterfs by K.+S.+Kyosuke · · Score: 1

      Aleph is a consonant! ;-)

      --
      Ezekiel 23:20
    29. Re:butterfs by Just+Some+Guy · · Score: 4, Informative

      I like the fact that they are naming filesystems after characters in South Park.

      I like the fact that they're naming filesystems after common household items that have existed for millenia. In South Park, "Butters" is plural.

      --
      Dewey, what part of this looks like authorities should be involved?
    30. Re:butterfs by Just+Some+Guy · · Score: 1

      On Slashdot, "grammar nazi" is spelled "[sic]".

      --
      Dewey, what part of this looks like authorities should be involved?
    31. Re:butterfs by cp.tar · · Score: 1

      Now waiting for wfbtrfs, i.e. the wife-beater FS.

      Isn't that ReiserFS?

      Nope. The wife-beater FS should produce more consistent results. ReiserFS only has one notable performance on a single benchmark.

      --
      Ignore this signature. By order.
  3. Short lifespan? I don't think so. by Viol8 · · Score: 5, Interesting

    Admins tend to stick with what they know and ext4 is a natural progression from ext3. btrfs however hasn't even reached version 1.0 yet - and to be honest who is going to want to use a 1.0 release anyway on something as fundamental as a filesystem? Also its development is being done by an Oracle team , albeit FOSS , which may put a few people off.

    My prediction for what its worth is that ext4 will be around for a LONG time.

    1. Re:Short lifespan? I don't think so. by IceCreamGuy · · Score: 5, Funny

      My prediction for what its worth is that ext4 will be around for a LONG time.

      Like... how long? Longer than it takes to fsck an 80GB ext2 filesystem? Because that's a pretty long time.

    2. Re:Short lifespan? I don't think so. by statusbar · · Score: 3, Insightful

      I wonder if ext4 performs better with SSD's? Or if ext4 doesn't need an occasional fsck like ext3 does?

      --jeffk++

      --
      ipv6 is my vpn
    3. Re:Short lifespan? I don't think so. by LWATCDR · · Score: 2, Informative

      What I don't understand is why more people are not using JFS?
      It has good performance, it is stable, and it supports very large file systems.

      Seems like a good choice for many people.

      --
      See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
    4. Re:Short lifespan? I don't think so. by juan2074 · · Score: 5, Funny

      You don't have to fsck btrfs (Butterface)?

    5. Re:Short lifespan? I don't think so. by diegocgteleline.es · · Score: 4, Insightful

      Hey, ext4 actually speeds up fsck quite a lot, so maybe ext4 won't be around a lot of time ;)

    6. Re:Short lifespan? I don't think so. by Azar · · Score: 4, Interesting

      I was wondering why it was omitted from this article as well. I believe that it's because JFS just doesn't seem to have the mindshare of ext3, XFS, or even ReiserFS. After reading various filesystem comparisons, I chose it as my FS and I have been using it for over a year without a single issue. I don't have to worry about long fsck times at reboot, my CPU has less load when deleting or copying a large volume of files (virtual machines or CD/DVD isos usually), never had any file loss or corruption, and it seems to complete large file operations quicker than ext3 (what I used previously). There are some things that ext3 does better than JFS, but overall I prefer the advantages of JFS over the advantages of ext3.

      I know that if I were to rebuild an older computer that had lower specs, or perhaps a set-top box like MythTV that I wanted to be more power effecient then JFS is going to be my choice for the filesystem.

    7. Re:Short lifespan? I don't think so. by larry+bagina · · Score: 1

      you should try fscking analfs.

      --
      Do you even lift?

      These aren't the 'roids you're looking for.

    8. Re:Short lifespan? I don't think so. by timeOday · · Score: 1

      Longer than it takes to fsck an 80GB ext2 filesystem? Because that's a pretty long time.

      Oh, are we giving ext3 a pass on this? Distros typically e2fsck ext3 filesystems every so often. Boy is it horrible when you're trying to get work done and your computer decides (without asking) to take 20 minutes to boot up.

    9. Re:Short lifespan? I don't think so. by machine321 · · Score: 1

      Also its development is being done by an Oracle team

      Oh, so it's Unbreakable, then.

    10. Re:Short lifespan? I don't think so. by level_headed_midwest · · Score: 1

      So you're saying your computer spends too much time fscking around?

      --
      Just "gittin-r-done," day after day.
    11. Re:Short lifespan? I don't think so. by Anonymous Coward · · Score: 1, Insightful

      You leave yours running 24/7 wasting electricity when you're not using it?

    12. Re:Short lifespan? I don't think so. by RiotingPacifist · · Score: 1

      How does JFS handle crashes (on desktop hardware)? I gave XFS a shot to find out it doesn't handle crashes well at all, since then im on reiserfs but with support lacking my next install will probably be using JFS

      --
      IranAir Flight 655 never forget!
    13. Re:Short lifespan? I don't think so. by Anonymous Coward · · Score: 1, Insightful

      But I *do* use it 24/7, I don't have to be in front of it to use it (my screen does go off quite often).

    14. Re:Short lifespan? I don't think so. by MrNemesis · · Score: 4, Interesting

      I've used JFS extensively for several years now, including power outages when the discs were under load - and I've never had anything fail to correctly fsck when the power came back up. fsck is also far, far faster than ext3 or XFS and CPU loading is *way* lower than XFS, whilst maintaining comparable throughput. I don't like XFS for the simple reason that if you don't have bulletproof power you will be restoring from backup. And when you've been in the job for as little as a few years you soon come to realise that *nothing* is bulletproof. Even triple-UPSed, six diesel generators, four seperate power lines and a five nines SLA with your data centre are no protection against a hung-over techie pulling the power cables from the wrong server.

      JFS also gets ACL access "for free"; if you use it like I use it, as the backed for a Samba server (~300 users) with a complicated ACL structure, JFS is much faster. Last time I fiddled with ACL's under ext3, I found they introduced a small, but percetible, increase in filesystem latency - don't know if this has been fixed, but using ACLs under JFS incurs no penalties, at least as far as my testing goes [citation needed]

      IMHO JFS is one of the hidden gems of Linux - fast under varying workloads, robust, frags up nowhere near as badly as ext2/3 when getting full, incredibly simple... the only downsides I'm aware of are that you can't shrink it and the fact that no-one seems to "support" it.

      --
      Moderation Total: -1 Troll, +3 Goat
    15. Re:Short lifespan? I don't think so. by kriston · · Score: 1

      My real-world experiences with SGI's XFS is that it takes entirely too much time to delete many files.
      In addition, rsync has had a tendency to corrupt directory files on XFS leaving me with thousands of indistinguishably-named files in the /lost+found directory.
      I'm giving IBM's JFS a chance after I recover this XFS filesystem.

      --

      Kriston

    16. Re:Short lifespan? I don't think so. by Macka · · Score: 1

      i don't think so either. I'm looking forward to Btrfs as much as the next guy. But my experience with the birth of new filesystems is that it takes them at least 2 years to mature to the point where you would trust production data on them. AdvFS on tru64 was a dog on Tru64 V4, and quite flaky. It wasn't until V5 when it got a major overhaul that the wrinkles got ironed out and it became something you could bet your business on.

    17. Re:Short lifespan? I don't think so. by LWATCDR · · Score: 1

      Well I found this http://www.linux.com/feature/119025
      If the information is accurate then I would say pretty good.

      --
      See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
    18. Re:Short lifespan? I don't think so. by etnoy · · Score: 1

      You leave yours running 24/7 wasting electricity when you're not using it?

      I, for one, never reboot my Thinkpad since the Linux system keeps working day after day. When I'm not using the computer I either suspend or hibernate, no need for shutdown.

      --
      Quantum hacker.
    19. Re:Short lifespan? I don't think so. by amorsen · · Score: 1

      Or if ext4 doesn't need an occasional fsck like ext3 does?

      I haven't fsck'd the ext3 on my laptop since it was installed, 3 years ago.

      --
      Finally! A year of moderation! Ready for 2019?
    20. Re:Short lifespan? I don't think so. by Solra+Bizna · · Score: 1

      I've used JFS extensively for several years now, including power outages when the discs were under load - and I've never had anything fail to correctly fsck when the power came back up.

      I've had JFS fail me three times. Twice it was due to an application issue (not fsyncing properly), and once it was due to a total fluke. The filesystem had been corrupted in a way that a manual fsck easily fixed... but the node that was corrupted was the directory that Gentoo's /sbin/rc keeps its state in, so it didn't get far enough in the boot process to fsck it automatically. (Not being able to read this directory when the filesystem isn't even mounted read-write shouldn't be a problem, people!)

      Other than that, it performs well and is pretty nice.

      -:sigma.SB

      --
      WARN
      THERE IS ANOTHER SYSTEM
    21. Re:Short lifespan? I don't think so. by etnoy · · Score: 1

      JFS is a great filesystem! Just one big drawback: You cant shrink filesystems. This bit me once, hard, since you can grow but not shrink them. Long story short: Trying to replace a disk with a bigger one in a LVM setup almost nuked the data.
      Of course, this wasn't too precious data and there was no need for backups, but if this had been in a real situation I would have had a problem.
      Since that day I exclusively use EXT3.

      --
      Quantum hacker.
    22. Re:Short lifespan? I don't think so. by Deagol · · Score: 1

      I assume this means XFS under Linux? I administered some large SGI boxes for several years, using XFS. Sadly enough, the data center they were housed in would occasionally go tits-up and all power would go down. Never once did I have any of these machines get hung on a reboot for fsck reasons, nor was any data loss ever reported. Is there some reason the version under Linux would be so comparatively fragile?

      Any pointers to your benchmarking methods? By "perceptible" did you mean "measurable"? Not that I doubt your claims -- I'm just curious how to accurately measure such a small performance degradation.

      I had the pleasure of working with AIX, and I, too, loved working with JFS. It's a really nice file system.

      --
      Method of processing duck feet
    23. Re:Short lifespan? I don't think so. by Azar · · Score: 1

      Is there some reason the version under Linux would be so comparatively fragile?

      My understanding is that Silicon Graphics (now SGI) wrote XFS specifically for their hardware which was designed to handle power failures, and would maintain enough power to finish it's current I/O operations. Since almost none (if any) x86 hardware has this built-in feature, XFS isn't as robust as it was on native SGI hardware. I can't find the references for this tibit, though. So take it with a grain of salt. It's just what I remember being told when XFS was first appearing on the Linux scene.

      Although, some of the issues that people see on XFS may be due to modified files that haven't been flushed to disk before the system loses power. XFS intentially zeros any unwritten data blocks to avoid possible security issues arising from residual data [1]. I believe XFS also uses out-of-order writes for both meta-data and data so a loss of power could mangle some data.

      There are a couple of slightly older, but still well-written, roundups about file system comparions. One here and one here.

    24. Re:Short lifespan? I don't think so. by Archwyrm · · Score: 1

      We use XFS quite successfully on our production servers and also more notably on our compute cluster. Due to some bad hardware from a defunct vendor, these nodes reboot pretty often and users always want to recover their data or restart their jobs, and most of our stuff uses rather volatile text-based file formats. So, if data were getting corrupted even occasionally with XFS losing power, I think we would have noticed by now. Perhaps power loss issues have been fixed?

      --
      Fascism should more properly be called corporatism because it is the merger of state and corporate power. -- Mussolini
    25. Re:Short lifespan? I don't think so. by Archwyrm · · Score: 1

      Clearly what you need is redundant hung-over techies.

      If you don't already have them, that is.

      --
      Fascism should more properly be called corporatism because it is the merger of state and corporate power. -- Mussolini
    26. Re:Short lifespan? I don't think so. by MrNemesis · · Score: 1

      Indeed, XFS under Linux - never lucky enough to get my hands on any IRIX kit but a bit before my time in any case.

      As t'other poster has said, XFS was designed with bulletproof power in mind - whether IRIX boxes had battery-backed everything I don't know, but XFS has problems with any power outage since it uses system RAM extensively as a write-through cache; even a battery-backed RAID card won't help you there.

      XFS is a pretty good filesystem and no mistake, I just think JFS is a better one. ext4 looks to brings most of the advantages of both into the mainstream though, which is a good thing (if several years late IMHO...!).

      --
      Moderation Total: -1 Troll, +3 Goat
    27. Re:Short lifespan? I don't think so. by MrNemesis · · Score: 1

      Damnit, forgot to reply to the rest of your comment :)

      The test I did was just a random snapshot of part of our file server copied onto our backup system; tested it with XFS, JFS and ext3 with ACL's enabled and disabled. Basically wrote a script that did repeated read/write requests on random files within the snapshot and found that the aggregate time for the ext3 + ACL's was consistently longer than JFS + ACL's - not by a huge amount (think it was only something like generally 1-2% longer than the same test without ACL's) but as I was already leaning to JFS it made my mind up.

      We also use AIX at work, and JFS on our shiny POWER6 boxes is an absolute joy - much like XFS on x86 hardware perhaps, JFS feels gimped on Linux once you've seen what it can do in AIX. Lovely stuff.

      --
      Moderation Total: -1 Troll, +3 Goat
  4. One filesystem to rule them all... by INeededALogin · · Score: 4, Insightful

    and in the darkness... bind them.

    Seriously... one of the nice things about Windows, OSX, Solaris is that they get a new filesystem once every 5-10 years. The safest thing to do for Linux is to be a generation behind. I would not run ext4 until btrfs came out. Why be the admin that gets screwed with early bugs and future incompatibilities...

    1. Re:One filesystem to rule them all... by fracai · · Score: 5, Funny

      See, I already thought of that, so I run no fewer than 5 generations behind.

      I started at 1, but realized that soon this practice would become widespread and then I'd be back to being an early adopter. So I moved to 2 generations. But then a friend agreed with my plan and I saw that in not too very long I'd be an early adopter again with my 2 gen old system. Not this time! I skipped the 3 and 4 generation delay and went right to the 5 generation wait time. I figured it was the only way to be sure I wouldn't get hit by any bugs.

      Shoot, now the secret's out. Time to roll back my filesystem again.

      --
      -- i am jack's amusing sig file
    2. Re:One filesystem to rule them all... by The+MAZZTer · · Score: 4, Informative

      Windows hasn't had a new filesystem that recently! NTFS was introduced in 1993. Windows has been using it for the past 15 years (if you only count the NT line, the 9x/ME line never even supported it IIRC, they just used the even older FAT32).

      Of course if you want to count the small extensions added on with each windows version then your claim about Windows is correct. Still I wouldn't be surprised if Windows Vista filesystems mount inside NT 3.1. I should test this with a VM...

    3. Re:One filesystem to rule them all... by Beat+The+Odds · · Score: 3, Insightful

      Seriously... one of the nice things about Windows, OSX, Solaris is that they get a new filesystem once every 5-10 years. The safest thing to do for Linux is to be a generation behind. I would not run ext4 until btrfs came out. Why be the admin that gets screwed with early bugs and future incompatibilities...

      I love your sense of panic.

      Anyone taking the use of ext4 seriously will setup test systems where the ONLY different is the file system. And, then, beat the crap out of it and see how it performs.

      It's really pretty simple to validate this type of thing.

    4. Re:One filesystem to rule them all... by El+Lobo · · Score: 5, Informative

      Sure the NTFS used by NT4 and the one Vista uses share the same name (and are *somehow* compatible: Vista understands the old NTFS and NT4 can use the new one in a limited way, but there is A LOT under the hood. The way security descriptors work, for instance is completely different in new versions. volume Shadow Copy, Hierarchical Storage Management, Junction Points, and other "extensions" are a HUGE step forward, and made the new NTFS in reality a new version, with the same old name.

      --
      It's time to realise that Abble's products are the biggest abomination these days. Just say NO to the dumb iAbble way!!
    5. Re:One filesystem to rule them all... by ookabooka · · Score: 3, Funny

      Shoot, now the secret's out. Time to roll back my filesystem again.

      To what? Pencil and paper?

      --
      If you are about to mod me down, keep in mind that this post was most likely sarcastic.
    6. Re:One filesystem to rule them all... by TheRaven64 · · Score: 4, Interesting

      NTFS is the result of the requirements team taking too long to decide what a filesystem should do. The filesystem team on the original NT project couldn't wait for them to decide, so they produced something very simple that just stored mappings from names to two kinds of data, big and small values. Big values are stored as one or more disk blocks, small values are stored in the Master File Table (MFT).

      On top of this is a metadata layer, where certain types of name are interpreted in different ways. This was used to add ACLs, encryption, compression, copy-on-write semantics, and so on.

      The low-level format for a Vista NTFS partition is the same as for NT3.51, but each version of NT has added a few more types of metadata and interpretations of them, meaning that writing to a disk with an older version is likely to cause data corruption.

      --
      I am TheRaven on Soylent News
    7. Re:One filesystem to rule them all... by PitaBred · · Score: 3, Informative

      Junction points have been around since at least Win2K, possibly NT4. I know I used them on 2K personally, can't speak to NT4 though. And Volume Shadow Copy requires application cooperation, so it's not really that much of an improvement over standard mirroring unless you use copy-on-write, which is still not filesystem level. You need to have the apps aware of it.

      --
      My blog. Good stuff (when I remember to update it). Read it.
    8. Re:One filesystem to rule them all... by compro01 · · Score: 4, Funny

      Pointy stick and a large beach.

      Just be aware of occasional data loss when the tide_in function gets called.

      --
      upon the advice of my lawyer, i have no sig at this time
    9. Re:One filesystem to rule them all... by macraig · · Score: 1

      I'm using junctions in this Windows 2000 system right now this very moment, thanks to a third-party shell extension or two I found that makes using them practical. One of the things I use them for is to shift some of the "default" Windows file structure locations somewhere else, without having to tweak all that stuff in the Registry. Because junctions operate under the OS radar, Windows is none the wiser that C:\Documents and Settings\username\My Documents is really just a hard link to a directory in another volume entirely.

    10. Re:One filesystem to rule them all... by Vancorps · · Score: 5, Insightful

      You think validating the integrity of a filesystem is easy??!?!?

      That's insane, first of all, you won't know how it performs unless you give it real world usage complete with disk failures. There are hundreds of file systems which can store data but how they handle problems are what separates most of them. Of course there are other distinctions but the failure mode scenarios are what most interest an admin as failure is never a question of it, only a question of when. Simulating certain failure modes is exceedingly difficult to do.

    11. Re:One filesystem to rule them all... by Kadagan+AU · · Score: 2, Funny

      Obligatory xkcd about computing using rocks and sand..

      --
      This space for rent, inquire within.
    12. Re:One filesystem to rule them all... by value_added · · Score: 1

      I'm using junctions in this Windows 2000 system right now this very moment, thanks to a third-party shell extension or two I found that makes using them practical.

      To the extent they work and to the extent they aren't a half-assed implementation of what non-Windows users take for granted, junctions created using different methods (Windows tools, Sysinternals, etc.) all behave differently, so expect to be bitten soon enough once you step outside your Explorer window.

    13. Re:One filesystem to rule them all... by Orestesx · · Score: 1

      I read the first line of your post and then stopped because I couldn't find the close parentheses. I think my brain was trying to compile your post.

    14. Re:One filesystem to rule them all... by evilbessie · · Score: 1

      They do actually implement the version number, but don't generally publish it as numbers confuse people. http://en.wikipedia.org/wiki/NTFS#Versions so it about the same (ish) time line as ext2->3->4

    15. Re:One filesystem to rule them all... by setagllib · · Score: 1

      It's kind of scary that even otherwise technically capable users still don't know you can permanently and reliably move the My Documents directory without any third party extensions, and yes, have it come up correctly in file dialogs and user32 calls.

      Also, hard link doesn't mean what you think it means. A hard link can only add a reference to a file within the same volume. If it's across volumes, it's a soft link.

      --
      Sam ty sig.
    16. Re:One filesystem to rule them all... by Alpha830RulZ · · Score: 1

      Simulating certain failure modes is exceedingly difficult to do.

      And especially difficult when you have a day job, and no time to invent and implement test plans that bring out these scenarios. Trying to explain the necessity of this to a PHB is fun, too.

      --
      I was taught to respect my elders. The trouble is, it's getting harder and harder to find some.
    17. Re:One filesystem to rule them all... by Beat+The+Odds · · Score: 1
      I didn't say that you'd do it in a day.

      You setup a system and let it run for a couple of months if you want to be safe. You put some load on it. Like I said, you beat the crap out of it.

      I didn't say is was easy, just simple.

    18. Re:One filesystem to rule them all... by HiThere · · Score: 1

      A huge change from would possibly be correct. I haven't heard anything from anyone except you that would make me think it an improvement...though doubtless there is some group of people who do. Change is less arguable.

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    19. Re:One filesystem to rule them all... by Zaiff+Urgulbunger · · Score: 1

      To the extent they work and to the extent they aren't a half-assed implementation of what non-Windows users take for granted, junctions created using different methods (Windows tools, Sysinternals, etc.) all behave differently, so expect to be bitten soon enough once you step outside your Explorer window.

      I no longer use Windows as my primary OS, but I did used to use it and did seem to work fine for me. I wouldn't have used it if it involved any third-party software though since that could be a World-Of-Pain... but since you can create Junctions fairly easily with the default Windows GUI tools (and command line if you're happy playing with GUIDs), I decided it _should_ be safe!

    20. Re:One filesystem to rule them all... by macraig · · Score: 1

      Nope. What I'm using hooks in and handles those events.

    21. Re:One filesystem to rule them all... by Zaiff+Urgulbunger · · Score: 1

      I remember marveling at how easy it was when I discovered (okay, "read" about) it.... but I must admit I've forgotten now. Can you remind me?

    22. Re:One filesystem to rule them all... by macraig · · Score: 1

      Sorry, didn't mean hard link. I know there are other means to relocate such directories, but they offend my sensibilities. Truth be told it's really the fact that You-Know-Who took the heavy-handed this-is-for-your-own-good route in the first place. I could have created an automated setup file to force my own preferences, but I expect to be migrating back to Linux before that would become useful or necessary.

    23. Re:One filesystem to rule them all... by macraig · · Score: 1

      That was one of the reasons why I chose this method; I have encountered some of that foolishly hard-coded software.

    24. Re:One filesystem to rule them all... by bertok · · Score: 1

      I think you completely and totally misunderstood what Shadow Copy does.

      It's not a mirroring technology, and it does not require application cooperation.

      It is a backup/snapshot technology that has the capability to quiesce applications using a plugin framework built into the OS, so that backup vendors don't all have to reinvent the wheel for every application.

    25. Re:One filesystem to rule them all... by dbIII · · Score: 1
      Volume Shadow Copy is one of the coolest features. It allows you to get around one of the major limitations of the OS that is using the filesystem and actually do a backup of any file even if it is in use just like you can with every other non-MS system on the planet.

      IMHO Volume Shadow Copy is where they moved from being a hobby operating system to something good enough for the workplace. So why don't other filesystems have it? They don't have it becuase it's a clever hack to get around an MS Windows design flaw and they wouldn't need it unless the filesystem is run under MS Windows.

      I use a lot of MS systems because they are useful in many cases but people should remember that it is the cheap and nasty end of computing.

    26. Re:One filesystem to rule them all... by Vancorps · · Score: 1

      How do you propose to "put some load" on it?

      The whole point is that if it's a production machine that hundreds or perhaps thousands use then you won't want to use an experimental file system otherwise it's a dev envrionment in which case it won't see anywhere near real world work loads plus it can take years for disk failures which to my knowledge are almost impossible to simulate.

      There's nothing simple or easy about it.

    27. Re:One filesystem to rule them all... by Archwyrm · · Score: 1

      ..failure is never a question of IT, only a question of when.

      There, fixed it for you.

      --
      Fascism should more properly be called corporatism because it is the merger of state and corporate power. -- Mussolini
    28. Re:One filesystem to rule them all... by svank · · Score: 1

      Pointy stick and a large beach.

      Just be aware of occasional data loss when the tide_in function gets called.

      Simple work-around: recompile your kernel with the NO_MOON flag set. Hopefully this bug is fixed by the 2.6.30 release.

    29. Re:One filesystem to rule them all... by Cyner · · Score: 1

      Vista NTFS is compatible with WinXP/2K to a limited degree (Most of the non-mainstream functions will not work). Vista NTFS will not mount on NT 4.0 or 3.x
      XP and 2K's NTFS are largely compatible, neither are stable under NT 3.x; and NT 4.0 can cause corruption still.
      All versions are forward compatible, which is a fricking miracle considering it's MS, and they changed stuff in every version...

      --
      FreeBSD.org - The power to serve
    30. Re:One filesystem to rule them all... by drunkahol · · Score: 1

      Doesn't avoid the random_tsunami procedure kicking in once every now and again!

    31. Re:One filesystem to rule them all... by Beat+The+Odds · · Score: 1

      Ok, I guess it's just not possible to put a new system of any kind into production.

      What if it's not a new file system? What if it's just a new version of the existing system?

      I guess the terms easy and simple are not good. But, you have to be able to test your system with simulated load to do some performance and reliability testing. What that means depends on the type of system.

      For example, if it's a web server, you might have to write some simulated "user" programs to access the server. You might have to deploy this simulated "user" program on many machines (perhaps some virtual machines). You might have to run this for days, weeks or months. Whatever.

      It's work, but it's not like inventing a new molecule or something.

    32. Re:One filesystem to rule them all... by compro01 · · Score: 1

      Nor does it help much with typhoon() and hurricane().

      --
      upon the advice of my lawyer, i have no sig at this time
  5. short lifespan? The big distros will decide. by Ritz_Just_Ritz · · Score: 3, Insightful

    It really depends on what the larger distros choose to stick with as their default. To be honest, I'd still be using ext2 if Redhat hadn't made ext3 the default. While I'm sure that some applications depend on wringing that last few % of performance out of the spindles, it just doesn't matter THAT much for most applications.

  6. Re:short lifespan? The big distros will decide. by INeededALogin · · Score: 5, Insightful

    it just doesn't matter THAT much for most applications

    well... run an fsck against ext2 and ext3 and tell me it doesn't matter. For an admin, speed, reliability, recoverability... are all major concerns. On Solaris, I love ZFS because of the functionality like snapshots and exports. I also got burned by the IDE/100% CPU driver bug on Sparc hardware. Admins need to be aware of what they are running and what limitations exist. I honest don't give a damn about mp3 encoding speed, but the capabilities and maturity of a filesystem have to be considered.

  7. ReiserFS by thomasj · · Score: 5, Funny

    ReiserFS used to be the killer FS, but now it seems like it is stuck. But I shall not be the judge of that, though there seems to be some truth buried in it somehow. And not to mention, the next release is probably more than a few years down the road.

    --
    :-) = I am happy
    :^) = I am happy with my big nose
    C:\> = I am happy with my OS
    1. Re:ReiserFS by lwsimon · · Score: 5, Funny

      I'd guess the next release of ReiserFS is 25 years to life down the road, actually.

      --
      Learn about Photography Basics.
    2. Re:ReiserFS by santix · · Score: 1

      ReiserFS used to be the killer FS, but now it seems like it is stuck.

      You're just confused. ReiserFS isn't the killer...its creator was.

      And he's not stucked... he is in jail.

    3. Re:ReiserFS by Hal_Porter · · Score: 1

      ReiserFS used to be the killer FS, but now it seems like it is stuck. But I shall not be the judge of that, though there seems to be some truth buried in it somehow. And not to mention, the next release is probably more than a few years down the road.

      BeaterFS will absolutely slaughter ResiserFS. Bury it.

      --
      echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
    4. Re:ReiserFS by operagost · · Score: 1

      Whoosh.

      --

      Gamingmuseum.com: Give your 3D accelerator a rest.
    5. Re:ReiserFS by cp.tar · · Score: 1

      BeaterFS will absolutely slaughter ResiserFS. Bury it.

      And BallmerFS will fucking kill GoogleFS... no, wait...

      --
      Ignore this signature. By order.
    6. Re:ReiserFS by bioglaze · · Score: 1

      And BallmerFS will fucking kill GoogleFS... no, wait...

      I think BallmerFS will fsck chairFS.

      --
      Who is John Galt?
  8. Dumbest benchmarks ever by Nick+Ives · · Score: 5, Insightful

    I see no analysis as to why the filesystems perform the way they do. Why does XFS perform so well on a 4GB sequential read and so badly on an 8GB read? Why did they include cpu / gfx bound frame/sec benchmarks? In the few application benchmarks where there was more than a tiny fraction of percent difference there's no discussion as to whether that difference is actually significant.

    Not at all enlightening.

    --
    Nick
    1. Re:Dumbest benchmarks ever by quintesse · · Score: 5, Funny

      You must be new here

      Next time check his /. ID first ;)

    2. Re:Dumbest benchmarks ever by LizardKing · · Score: 1

      Why does XFS perform so well on a 4GB sequential read and so badly on an 8GB read?

      Because the cronjob that indexes the authors pr0n collection kicked in between the test runs.

    3. Re:Dumbest benchmarks ever by Tubal-Cain · · Score: 1

      Obviously bought on ebay.

    4. Re:Dumbest benchmarks ever by chrb · · Score: 1

      These benchmarks never do any kind of statistically valid analysis. Even when testing CPUs, where variance may be close to 0 on a single platform, there will be variance between different mainboards, different RAM setups, etc. It would be nice to see a benchmarking site actually do proper valid benchmarking for a change.

    5. Re:Dumbest benchmarks ever by chrb · · Score: 1

      We might as well link to the previous discussions of Phoronix testing: Ubuntu 8.10 vs. Mac OS X 10.5.5 Benchmarks and Is Ubuntu Getting Slower?

    6. Re:Dumbest benchmarks ever by Shinobi · · Score: 1

      Yeah...

      I'd have liked to see the test done properly, with several tuning profiles for each filesystem. And workloads such as search and replace in 10000 different files in 30 different directories(Bet on ReiserFS in that one). Or handling a directory with 200 3GB files being processed by an image or compositing package(XFS for that one).

    7. Re:Dumbest benchmarks ever by drspliff · · Score: 3, Funny

      The /. ID key overflowed, he's only been here a few days.

    8. Re:Dumbest benchmarks ever by Nick+Ives · · Score: 1

      Jealousy is a lovely thing, isn't it?

      For the record, I really did sign up as soon as user accounts were created - can you believe this site used to run on the honour system? You just put your name in and that's what appeared. It was only when trolls started posting as other users that user accounts were introduced.

      I dropped offline for a few years (I think I may have spent a whole year continuously offline!) and had a taste of real life (it's grim and amazing, I recommend it to anyone who feels they've grown up in front of a screen) but eventually ended up back here. I comment far more these days than I ever did when /. was first about, I think it's because the quality of the users has fallen so much (that's the real decline of /. btw, the stories have always been old or dupes or silly) that even a dumbass like me can seem insightful.

      And yes, I did read your sig. Your comment didn't seem particularly witty though, so I took it seriously.

      --
      Nick
    9. Re:Dumbest benchmarks ever by Nick+Ives · · Score: 1

      I found this comment to be amusing. I recalled reading their 4870 X2 review not too long ago. You'll notice cards of vastly different performance levels showing similar speeds until AA & AF get cranked up, a clear indication of the benchmarks being CPU bound. This isn't commented on at all in their summary!

      Admittedly there aren't any games in Linux that could stress a card like that but things like that should at least be acknowledged. Whilst it's great that they're testing these cards in Linux they really do need to improve their methodology.

      --
      Nick
    10. Re:Dumbest benchmarks ever by kantier · · Score: 1

      You must be new here

      Next time check his /. ID first ;)

      Then he must be new in the new here

  9. Stupid testing methodology by afidel · · Score: 5, Informative

    WTF who measures things like MP3 compression time when testing a filesystem?!? As far as I can tell they only ran one real I/O test and that was the Intel IOMeter based fileserver test which showed EXT4 is really fast for that profile. I would have loved to have seen the DB profile run. Their other artificial tests could have been summed up by running the streaming media profile since they were just large contiguous reads and writes.

    --
    There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    1. Re:Stupid testing methodology by ChrisA90278 · · Score: 4, Insightful

      who measures things like MP3 compression time when testing a filesystem?!?

      They were measuring what matters: "System performance" It very well could have been the case that the fastest file system, measured on a simple bench mark gives the worst MP3 compression time. Let's say the reason the filesystem is fast is because it uses a huge amount of CPU time and RAM. So a RAM based encrypted file system might be very fast, until you run an application on it.

      It's a reasonable test and what it showed is that in the real-world performance is about the same

    2. Re:Stupid testing methodology by MrNemesis · · Score: 1

      Agreed, there's not really much point to many of the benches.

      True, all other things being equal if you have two identically performing filesystems but one ramps the CPU higher than the other for a given throughput, you'd expect the more conservative filesystem to finish the CPU-bound benchmark the quickest.

      However, this is no reason to make approx. half of your tests mostly CPU/memory/graphics bound where I/O performs a minimal role. Where are the rest of the IOMeter profiles? Where's the time to do 10,000 inserts in postgres? Where's the time to create, cat and delete 1000 4kB files? How long to do an `ls` on a filesystem containing a million directories? Heck, where's even `time cp /mnt/ramdisk/testfile /mnt/ext4testdisc`? Please remember that hard disc benchmarks will be affected in normal use by the speed of the filesystem running on them.

      Don't mean to rant, but Phoronix is practically one of the only sites still doing regular hardware and software reviews under Linux, just that sometimes I feel like their methodologies are totally flawed or included just for the hell of it. Either way, valid tests tend to get lost in the noise.

      Would have liked to see some JFS benchmarks too, but I do realise that even mentioning JFS makes me evidently psychotic ;) I've generally found it the best performing filesystem all-round (fast as XFS where it counts but with considerably better resiliency IME), it just feels a bit like the forgotten bastard stepchild...

      --
      Moderation Total: -1 Troll, +3 Goat
    3. Re:Stupid testing methodology by tknd · · Score: 2, Informative

      What about office apps? Design tools (like engineering design software)? Compilers, linkers, and interpreters? Package management (since this is linux)? File searches with the "find" command? Recursive directory copies?

      The benchmark suite chosen is not representative of the "real world" usage. In the real world there are a variety apps that work on a variety of data. Not just cpu bound applications.

    4. Re:Stupid testing methodology by syousef · · Score: 2, Insightful

      WTF who measures things like MP3 compression time when testing a filesystem?!?

      Anyone interested in real world usage. Resource usage of the file system drivers while doing something processor intensive such as encoding is certainly of interest.

      --
      These posts express my own personal views, not those of my employer
  10. XFS by conspirator57 · · Score: 3, Insightful

    Given that XFS, a 15+ year old file system, is still a serious contender, one would think enough blood has been squeezed from this stone. What is left to us is application tuning and hardware improvements, possibly including filesystem management hardware. It seems to me that teaching application developers how to write their programs to best utilize the filesystem is more likely to yield better performance gains for the effort expended than trying to make a general purpose filesystem good at any flavor of IO that application developers naively throw at it. Simple rules: buffer your IO yourself, perform raw accesses in multiples of the sector/stripe size size.

    --
    "If still these truths be held to be
    Self evident."
    -Edna St. Vincent Millay
    1. Re:XFS by UnknowingFool · · Score: 1

      Well I think these tests play to XFS' advantages. XFS does well with larger files and when XFS was new there were not many applications for large files except maybe in Big Iron. These days larger and larger files like movies are more the norm.

      --
      Well, there's spam egg sausage and spam, that's not got much spam in it.
    2. Re:XFS by sohp · · Score: 3, Interesting

      Blood to squeeze? How about a new stone? Solid-state drives.

      SSDs. Yep, they will completely change the rules for filesystems. Decades of tricks and tweaking to deal with rotational latency and head movement have virtually zero application in SSDs. All the code for that will become worse than useless. It will have to be removed or at least turned off. Leaving it on will actually result in worse performance on SSDs.

    3. Re:XFS by lysergic.acid · · Score: 2, Interesting

      hrm, i don't know that SSD has gained enough widespread adoption for a mainstream filesystem to be optimized for solid state rather than mechanical rotational storage. however, you do raise an interesting point. perhaps a new filesystem can be designed from the ground up optimized for SSD. whoever gets into this area of development right now will have a huge lead on competitors when SSD storage solutions finally achieve price parity with spinning media.

    4. Re:XFS by sohp · · Score: 2, Insightful

      Agreed, adoption is still pretty low, but it's definitely a game-changer. Surprisingly, SSDs might show up first en masse at the low end in netbooks like the Eee. Consider the lower power requirements and instant-on possibilities.

    5. Re:XFS by Tubal-Cain · · Score: 3, Informative

      Perhaps a new filesystem can be designed from the ground up optimized for SSD.

      JFFS2
      UBIFS
      YAFFS

    6. Re:XFS by amorsen · · Score: 1

      Decades of tricks and tweaking to deal with rotational latency and head movement have virtually zero application in SSDs.

      Yes, that happened when we switched from drums and tape to hard drives. Then hard drives seeks became (relatively) as slow as tapes were. Wait and see, we may have a few years of bliss, but in ten years flash will have seek latency -- RAM does already.

      --
      Finally! A year of moderation! Ready for 2019?
    7. Re:XFS by lysergic.acid · · Score: 1

      interesting. i'd never heard of these flash media filesystems before. i wonder how they compare to conventional filesystems, or why they weren't used in the Linux.com SSD benchmarks.

      from these graphs it seems like ext3 performs quite well on SSD. but i can't find any benchmarks comparing any of the filesystems you listed. i did, however, come across this PDF released by Samsung, which shows some pretty interesting benchmark results for ext2, ext3, ext4, reiserfs, xfs, btrfs and nilfs. and in these tests nilfs comes out on top in almost all of the tests.

    8. Re:XFS by tabrisnet · · Score: 1

      All of those filesystems are designed to work on raw flash, not the Flash to IDE products that typically come as 'SSDs'. So sorry, but that's not a useful answer.

    9. Re:XFS by conspirator57 · · Score: 1

      i think i'd file that under hardware improvements. i'd also file the cheapening of RAM and its use as primary storage as a hardware improvement. My point was that filesystems as a whole are as good as they are going to get barring something truly novel like neural-associative storage. of course then the storage would get CRS disease just like us. SSD doesn't change the fact that the storage system is a scarce (presumptively) shared resource with a fixed theoretical bandwidth and many ways to avoid achieving anything like the max theoretical. Touting SSDs as anything but an incremental improvement is just regurgitation of marketing drone rubbish. The same crap was rampant when fibre channel drives hit the street. yes they were better. no they weren't that different.

      --
      "If still these truths be held to be
      Self evident."
      -Edna St. Vincent Millay
    10. Re:XFS by Tubal-Cain · · Score: 3, Interesting

      All of those filesystems are designed to work on raw flash

      Exactly. Theoretically, manufacturers could do away with the Flash-to-IDE interfaces.

    11. Re:XFS by owlstead · · Score: 1

      Yeah, I'll start teaching application developers right away. If history has learned us anything, it is that you cannot learn things to people that way.

      The reason that you want a well performing file system is that this is the layer where you can most easily address this issue. These people care about the performance. You'll just have to figure in some dopey application programmers while you are at it.

      Of course, you may sway one or two developers with your comments here, so keep at it. As long as you don't expect the same from file system developers.

    12. Re:XFS by wvmarle · · Score: 1

      Price parity isn't there yet but it won't take long. Devices like the iPod and EEE-PC really have brought critical mass to the SSD. Look for example at the price points of the first and second generation EEE: ten times the storage, larger screen, same or even lower overall price.

      SSD prices are falling, and falling fast. Critical mass has been reached. Fs vendors should be about ready now for the SSD specific file systems, now is the time for them to build up mindshare and critical mass. The SSD is taking over the laptop market, a few % market share now though it's growing. Very fast.

    13. Re:XFS by conspirator57 · · Score: 1

      my point is that we *have* well performing filesystems. So much so that investing money in developing improvements or new filesystems is producing little or no return in the form of greater performance. So why spend money you're not getting anything extra for? Spend it tuning the apps and improving the system bandwidth from hardware. Occasionally take a look to see whether there is a scaling issue with the filesystems against the new hardware and bandwidths. Meanwhile, perhaps push the filesystem into hardware. You might see general case performance gains from that. In any case, repeating the same previous behavior of investing in filesystems expecting different results than prior investments is insanity.

      --
      "If still these truths be held to be
      Self evident."
      -Edna St. Vincent Millay
  11. Useless. by Steve+Baker · · Score: 4, Insightful

    What's with the CPU/Video tests? How about some more random access pattern tests, DB/web/streaming media tests? How about showing CPU utilization in addition to I/O performance?

    1. Re:Useless. by TheLink · · Score: 1

      I also would like to see how well they handle and avoid fragmentation.

      --
  12. oops by conspirator57 · · Score: 1

    size size.

    pizza pizza.

    --
    "If still these truths be held to be
    Self evident."
    -Edna St. Vincent Millay
  13. Distro by Enderandrew · · Score: 1

    Which distros will include this as an option at install time? That is what I want to know.

    --
    http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
  14. aah Ext4 by noppy · · Score: 2, Funny

    Time to get a new wife

  15. oblig by russlar · · Score: 1

    I can't believe it's not butterfs!

    --
    Anybody want my mod points?
    1. Re:oblig by ZERO1ZERO · · Score: 1
      I never knew I had to have an opinion on how it is pronounced.. until this story.

      I propose Butt-Riffsss.

  16. Re:short lifespan? The big distros will decide. by quintesse · · Score: 2, Informative

    No need to wonder, Fedora 9 had it already back in may: http://fedoraproject.org/wiki/Releases/9/FeatureList

  17. bad testing by ianare · · Score: 1

    This has got to be one of the worst tests I have ever seen. Their 'real world' tests were made on operations that take mostly processor power. Without getting into the completely retarded game testing, they looked at encoding, compression, and file encryption. Sorry but for me this tells practically nothing of the speed of a file system.
    I would have like to see :
    creating 4gb of large files
    deleting 4gb of large files
    copying 4gb of large files
    creating 4gb of small files
    deleting 4gb of small files
    copying 4gb of small files
    what happens if you yank the power plug during a FS operation ?
    Start up times for OpenOffice, Eclipse, and GIMP

    And then, sure do the tests they ran, but not just

    1. Re:bad testing by bigredradio · · Score: 1

      I was wondering if I was the only one who saw that. Thank you. I totally agree.

      You can add to your list:
      Grow the filesystem online by 32GB
      Put oracle or some other database on it
      Use it for /var and run a real mail server.

      --
      Flexible bare-metal recovery for Linux/UNIX
  18. Fedora by Booker · · Score: 1

    In since F9; in F10 boot the installer with "ext4" on the commandline to get it enabled.

  19. Phoronix is worthless by Rendus · · Score: 5, Informative

    Remember, this is the site that's decided that Ubuntu 7.04 is twice as fast as any other version of Ubuntu. Take what they say with a good healthy dose of skepticism.

  20. No blogbench bench? by chrysalis · · Score: 1

    Ehm, still those benchmarks filesystems are optimized for. Please try blogbench in order to make filesystem really hurt like they would do for a file server.

    --
    {{.sig}}
  21. Interesting choice of hardware by pz · · Score: 1

    Their test system is a monster 8-core, dual CPU setup, with only 2 GB of RAM. (Hell, I've got 2 GB of RAM in my dinky single-core Athlon 2800+ desktop.)

    RAM is cheap and CPUs are expensive. Their system is not particularly representative since it seems to be biased in the other direction. Further, when the tests include reads and writes that are guaranteed to fill up all available RAM (sequential ops of 4 and 8 GB), the design is flawed because I/O to swap may contaminate the results.

    Maybe they can fill up the four open banks on their motherboard (nice of them to show us the photo) and re-run the tests with a more realistic setup, or at least more balanced on the RAM/CPU ratio. Even better, ditch the dual quad-core Xeon (how many Linux users have that?!?) and use a more common format. The ostensible purpose is real-world testing, after all.

    --

    Put my fist through my alarm clock with its ding-dong death inside my ear. - The Blackjacks.
    1. Re:Interesting choice of hardware by Gothmolly · · Score: 1

      Sounds like a classic gamer/fanboi computer. ZOMG I HAVE 8 CPUS!!1! Show me benchmarks on a 2way, 8GB machine, with more than 1 dinky ATA drive, and I might be more interested.

      --
      I want to delete my account but Slashdot doesn't allow it.
    2. Re:Interesting choice of hardware by Alpha830RulZ · · Score: 1

      Actually, that's our production configuration, (other than the RAM, which I agree is low).

      I think the deeper criticism is that the loads and scenarios are simplistic and not representative of at least the loads I use. Give me a day's processing action against a database, and show me what happens there. How about a variety of machine to machine small and large file transfers? I know, these should be limited by the network, but then that would should that the speed of the FS doesn't matter, and that other features should govern. How about a restore from backup? How about an application that opens, reads, writes, and closes lots of small files?

      --
      I was taught to respect my elders. The trouble is, it's getting harder and harder to find some.
  22. Re:short lifespan? The big distros will decide. by Chris+Burke · · Score: 2, Insightful

    To be honest, I'd still be using ext2 if Redhat hadn't made ext3 the default.

    Well thank goodness RedHat saved you from yourself, then!

    It's not about performance, it's about journaling. Ext3 has it, ext2 doesn't, ergo by modern standards ext2 is crap. The only justification for using it was when the only journaling file systems for linux were unstable.

    --

    The enemies of Democracy are
  23. Does Phoronix even know what "Real World" means? by Ant+P. · · Score: 1

    Here's a few pulled-from-my-ass tests for the real world, as opposed to their real-world tests for an ad-revenue-driven world.

    1. Fill the disk with data, then write random bytes to random sectors of the disk (using a repeatable PRNG). See what percentage of the filesystem is still recoverable afterwards, and how long it takes to do a fsck on it.
    2. Time this.
    3. Count the number of times each FS's developers have murdered someone (hey, it's no less meaningless than measuring a FPS timedemo in terms of seconds per filesystem).
  24. ZFS for the win by MsGeek · · Score: 1, Insightful

    Ha! B-Trees! And Linus Torvalds calls Macintosh HFS+ "retarded!" Soooooo funny! That's the newest and kewlest? Come on, HFS has been around since Macs had hard drives, and it used B-Trees! Seriously, nobody benchmarked ZFS, and that's where the real action is going to be headed. Lose two hard drives, your RAID is still AOK, and you can rebuild it on the fly!!! ZFS is the future.

    --
    Knowledge is power. Knowledge shared is power multiplied.
    1. Re:ZFS for the win by pyite · · Score: 2, Informative

      Lose two hard drives, your RAID is still AOK

      That's an awfully simplistic statement, and only applicable for RAID-Z2. A simple mirror or RAID-Z can (understandably) only sustain a single failure.

      --

      "Nature doesn't care how smart you are. You can still be wrong." - Richard Feynman

    2. Re:ZFS for the win by jon3k · · Score: 1

      A filesystem can use B-Trees and still be retarded. There's _slightly_ more to an operating system than it's searching algorithm.

      Also, he actually used the phrase "utter crap."

  25. Is this the best you can do? by Anonymous Coward · · Score: 1, Funny

    Linux developers are PATHETIC.

    OSX + ZFS *FOR THE WIN*!

    Think different.
    Think better.
    THINK APPLE.

  26. a HUGE step forward .. by rs232 · · Score: 1

    Volume Shadow Copy: sounds like LVM2 snapshots under Linux and VMS had real versioning file restoraton a long time ago.

    Hierarchical Storage Management: HSM as implemented by IBM and then ported to Linux

    Junction Points: sounds like hard links under Linux

    --
    davecb5620@gmail.com
  27. The real problem with reiserfs by TheLink · · Score: 5, Funny

    The real problem with reiserfs: vendor lock-in.

    --
    1. Re:The real problem with reiserfs by Lisandro · · Score: 2, Funny

      They'll end up burying the competition, you'll see!

  28. Re:short lifespan? The big distros will decide. by BitZtream · · Score: 1

    Well, I'm going to assume the ext2/3 drivers are not buggy and then ask, why not just STOP fscking it if it bothers you that much?

    Back when I ran linux you had automatic fsck's at regular (reboot) intervals because no one actually trusted the code, but that was 12-13 years ago, I can not possibly believe that by now its not stable and safe enough to avoid checking it a regular interval from clean shutdowns. And even then, its not like its going to happen that often on a server unless you reboot server nightly for some weird reason?

    So okay, on a desktop it can get annoying since you may shutdown and restart often, but this is the perfect example of where to just turn off the scheduled fsck on boot. My FreeBSD boxes have never fsck'd on boot after a clean shutdown, I've only seen Windows do it after some sort of change to the FS that made it seem like a good idea, just in case. I can't remember a time when I've seen a Solaris box fsck on boot after a clean shutdown.

    So really, WHY is it running so often that it bothers you in the first place?

    --
    Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
  29. Re:third-party shell extension .. by karnal · · Score: 1

    C:\Documents and Settings\karnal\Desktop>ln
    'ln' is not recognized as an internal or external command,
    operable program or batch file.

    C:\Documents and Settings\karnal\Desktop>mount
    'mount' is not recognized as an internal or external command,
    operable program or batch file.

    C:\Documents and Settings\karnal\Desktop>

    :(

    --
    Karnal
  30. Re:Quit Benchmarking against Reiserfs, no one care by Dogun · · Score: 1

    If this test had included small-file tests, you would see why reiserfs is relevant.

  31. Small files? by Dogun · · Score: 1

    That's odd. I didn't see any small-file benchmarks in this article.

  32. Re:ext4/XFS by Anonymous Coward · · Score: 1, Insightful

    XFS has been around for 15+ years and still makes a pretty good appearance on benchmarks. Which means that ext,ext2 and ext3 filesystems were late and have little to offer more, with ext4 just marginally (15 years to make something like ext4 with such minimally more performance?)

    âoeFoolsâ to the Phoronix guys who thought that filesystems relate to in-game frames per second or compression. Most games preload all their stuff into RAM anyway. And compression is CPU-bound, your disk writes are done in the background anyway.

  33. How do you take a _consistent_ snapshot with LVM? by ToasterMonkey · · Score: 1

    Volume Shadow Copy: sounds like LVM2 snapshots under Linux

    VSC isn't perfect for sure, but how exactly do you take a _consistent_ snapshot with LVM? Snap and pray?

    VSC has hooks for applications to detect a snapshot and quiesce themselves, in a cooperative manner.
    Not perfect, but at least Microsoft's big apps mostly support it to enable consistent backups.

    I don't know how popular it is though. The most common way of doing consistent backups on both UNIX and Windows is for the backup software to either tell the app to quiesce itself then back it up, or a combo quiesce & snapshot, exit quiesced mode soon as snap is done, then take your time backing up the snapshot. The problem is that you need app specific modules to quiesce them, and the need to deal with a multitude of snapshot techniques. Snapshot might be software like LVM, or SVM, third party software like VxVM, hardware like Symmetrix BCV, or Clariion snap, God only knows. You can imagine that this gets VERY complex and expensive.

    VSC allows applications to plug into it so that backup software doesn't need to be app specific, just VSC aware to start/stop a snap. The app detects a snap and quiesces automatically. VSC also allows storage vendors to integrate their platforms with it. Like I said, the most common was is still the app/vendor specific route, I'm not sure just how many storage vendors have integrated with VSC, but it could be more than I think.

    Backup software calls for a snap, the app quiesces by itself, the backend storage system does a hardware snap or mirror split, backup software gets the OK to backup. Cool, huh?

    I dig on Microsoft a lot too guys, but you should really get your heads out of the sand and pay attention to the many good things they (and other vendors) do. Reading linux.com all day doesn't give a realistic representation of where Linux stands technologically.

    Hierarchical Storage Management: HSM as implemented by IBM and then ported to Linux

    News to me, where does Linux have HSM built in? Is it free? Do you have any examples of it being used in production?

    Junction Points: sounds like hard links under Linux

    Don't nitpick, there is obviously a lot you don't understand about NTFS and it's associated frameworks. The GP is right, NTFS has changed a lot over the years but kept the same name.

    Ext has... well, it got journaling at some point. And then.. well, here we are. If Ext3 was more than just your run of the mill textbook filesystem, there wouldn't be so many damned new ones in the works.

  34. Speaking of.... by cayenne8 · · Score: 1
    "We already have ReiserFS to fill this niche"

    Speaking of ReiserFS...what is the deal with it? I always liked it, many of my boxes use it.

    Is it dead now with him in prison, or is someone else taking it over?

    --
    Light travels faster than sound. This is why some people appear bright until you hear them speak.........
    1. Re:Speaking of.... by doom · · Score: 1

      Is it dead now with him in prison, or is someone else taking it over?

      There's one or two guys still working on it, but if all anyone sees are lame benchmark sets like this, interest in it is definitely going to trickle away.

      Myself, I continue to use Reiser 3. It's essentially mature code at this point that isn't likely to need a lot of attention.

      Reiser 4 isn't quite "dead", but it's certainly limping. Someone with some serious balls is going to have to get involved with it for it to keep going -- it's going to take work at the distro level, possibly with a forked linux kernel.

      My usual editorial: unix file systems have been crippled for so long, people have gotten used to dancing around the limitations ("don't treat the file system as a database!"), and have trouble grasping why you would want to remove the limitations. The concentrate on optimizing for "real world cases", and think it's weird if you want to add capabilities to do anything else.

    2. Re:Speaking of.... by berend+botje · · Score: 1

      Yes. Yes, it is.

    3. Re:Speaking of.... by kv9 · · Score: 1

      Speaking of ReiserFS...what is the deal with it?

      you could say it's a killer file system.

      --
      Stop Computers/Cars Analogies on S
  35. Re:Quit Benchmarking against Reiserfs, no one care by theendlessnow · · Score: 1

    What makes you think the JFS inside of Linux has a future? It's not the same as the one in AIX. It's an old crummy (bad, buggy) version from the OS/2 days. From what I can tell, JFS (in Linux) isn't going to make it. Too many flaws, and no interest by IBM or others in fixing it.

  36. Re:How do you take a _consistent_ snapshot with LV by ToasterMonkey · · Score: 2, Interesting

    I remembered another little known built in NTFS feature few people know about. Maybe those of you genuinely interested in filesystem features & backups, not OS religion will appreciate it.

    Change journaling.

    I don't think there is a native GUI for this, instead the backup vendor provides one, or handles it for you.
    It is specifically meant for backups. Instead of scrubbing the entire disk and comparing to previous records to find changed files, with the change journal enabled, incremental backups consisting of millions of files are pretty quick. Well, discovering the list is MUCH faster anyway, backing up a million 1k files sucks regardless. The beginning of a backup can actually be very resource intensive, and time consuming without this feature. It has a small overhead, but each time you perform a backup operation the journal resets, releasing that overhead.

    Anyone know where this feature might have originated from? AFAIK Solaris and Linux don't have this ability, and being backup specific, I can't imaging the FOSS world even dream of it. Well, now you know, copy away :)

  37. Dodgy results? by Roger+W+Moore · · Score: 1

    Those results look dodgy to me. It looks like the total time to check the filesystem decreases as the number of inodes increases. I could perhaps understand time per inode decreasing but not the total time.

  38. The upside of no vowels by Space_Pirate_Arrr · · Score: 1

    Th upside is an ~20% increase in storage capacity and transfer speeds due to no vowel overhead.

  39. Large files only? by MikeBabcock · · Score: 1

    How come all these 'real world' benchmarks they did are for 2GB files only? Why not thousands of small files instead? I'd like to know how the filesystem works for Maildir E-mail and web hosting, or for BitTorrent usage. BitTorrent especially loves to trash whole chunks of the drive all at once, although I understand that a test-bed BitTorrent network would take a bit more work.

    What about browsing a directory full of downloaded photos (and the resulting thumbnail generation), or re-compressing a music collection?

    That's a few benchmarks I think might be realistic.

    --
    - Michael T. Babcock (Yes, I blog)
    1. Re:Large files only? by doom · · Score: 1

      How come all these 'real world' benchmarks they did are for 2GB files only? Why not thousands of small files instead?

      Because then ReiserFS 3 would kick their asses, since it's the only one that was designed to scale down as well as up.

      And if you actually look at the results on these, with the exception of a few of the tests it was a dead-heat between all the file systems -- and if we're talking "real world" usage, how many applications are actually bound by disk performance rather than network?

      But then, "it doesn't much matter what you use, get a life" doesn't make a very exciting head line.

    2. Re:Large files only? by MikeBabcock · · Score: 1

      The fact that ReiserFS 3 kicks ass in the situations I described is exactly why I use it in those situations :-)

      That said, it would've been nice to compare.

      --
      - Michael T. Babcock (Yes, I blog)
  40. Re:How do you take a _consistent_ snapshot with LV by amorsen · · Score: 1

    VSC isn't perfect for sure, but how exactly do you take a _consistent_ snapshot with LVM? Snap and pray?

    If your disk isn't consistent at all times, how do you deal with crashes? Restore from backup?

    --
    Finally! A year of moderation! Ready for 2019?
  41. Re:How do you take a _consistent_ snapshot with LV by Atti+K. · · Score: 1
    You just didn't mention the magic word MS calls this feature: USN journal

    Try the command fsutil usn to play around with it. Of course, it's pretty useless if there's no app relying on it. (but very useful if there is, like for backups as parent said).

    I'm not sure, but OS X might have something similar in HFS+, and use it for Time Machine and also Spotlight (know quickly which files were changed and reindex them).

    --
    .sig: No such file or directory
  42. Re:third-party shell extension .. by rs232 · · Score: 1

    You need to totally remove the Windows partition from the harddrive and install one of these distros ..

    --
    davecb5620@gmail.com
  43. Re:short lifespan? The big distros will decide. by INeededALogin · · Score: 1

    Gotta reply even though its late.

    Well, I'm going to assume the ext2/3 drivers are not buggy and then ask, why not just STOP fscking it if it bothers you that much?

    Because in the case of a crash(yes... servers crash in the real world. Power outages happen... retards accidentally unplug it...). When a server goes down, I need it back up extremely fast. My time is valuable... I don't have 2 hours to devote to micro-manage an fsck. And don't tell me you don't have to micro-manage it. In Solaris, it will try its best to fix it, but drop you into single user mode to force fix some errors.

    So really, WHY is it running so often that it bothers you in the first place?

    Because I use to maintain a few hundred machines. At that scale, something is always broken or ready to break.

  44. Re:How do you take a _consistent_ snapshot with LV by ToasterMonkey · · Score: 1

    I'm the GP too, had to reply to myself :)

    Thanks for the info, I've only dealt with the little window Networker provided to it.
    You might be right about OS X, there's got to be something going on under the hood to make them as smooth as they are.