Slashdot Mirror

← Back to Stories (view on slashdot.org)

File Fragmentation and File System Resiliency

Posted by Cliff on from the king-of-the-platter dept.

Eric^2 asks: "We have an old NT server that we are going to replace sometime this year. It has Diskeeper on it to do disk defragmentation. I remember from DOS that this was also a BIG problem, and am curious how the EXT2FS handles file fragmentation. Whatever we put in to replace that NT box needs to be fairly resiliant, and I was thinking either FreeBSD, or maybe Linux with the XFS file system, as it's supposed to be more fault tolerant. I would appreciate any suggestions that you may have! Is there a more robust solution than XFS for Linux? FreeBSD? Should I stick with NT and Diskeeper? "

9 comments

  1. Starting points. by SEWilco · · Score: 2
    A good starting point is probably the Multi Disk System Tuning HOWTO. The various RAID HOWTOs may also be informative.

    EXT2FS does not have much problem with file fragmentation. Like most Unix filesystems, it automatically reuses all empty space and tries to keep files/directories together.

    XFS does look interesting, and it should at least reduce restart time.

    Notice that on a server you can reduce file system updates by putting files which are rarely updated on read-only filesystems which are separate from the often-updated ones, reducing the partitions which need to be checked.

    Of course, maybe you should also start by considering how much better anything else is when compared to what you're presently using. NT needs defragmentation, and crashes often enough that restart time and disk recovery are great concerns.

  2. Beware filesystem corruption by Paul+Carver · · Score: 0

    In my experience, ext2 is rather fragile if not cleanly unmounted. I don't recall my Linux desktop machine ever crashing, but it has lost power on occasion. Every time it has suffered a power outage I had files corrupted and data lost.

    My NT laptop crashes on a semi-regular basis (probably heat related, it gets very hot), but I've never had any file corruption or data loss.

    If you do switch to Linux make very sure that no one can trip over the plug. And make sure the cleaning people don't unplug your server to plug in a vacuum cleaner. (or is that an urban legend?)

    1. Re:Beware filesystem corruption by rise · · Score: 1
      If you do switch to Linux make very sure that no one can trip over the plug. And make sure the cleaning people don't unplug your server to plug in a vacuum cleaner. (or is that an urban legend?)

      Nope, most definitely not an urban legend. We had it happen a few times at my old employers. Of course that wasn't nearly as traumatic as coming in to find that the security guys had left the development lab open and somebody's kid had pryed a CPU heatsink off as a souvenir.

      Note as well that ext2 is also reasonably conservative about what it considers corruption. The data loss is definitely an issue with any non-journaling filesystem that caches disk updates, but I've seen data loss & disk corruption with some frequency on high-load NT boxes.

    2. Re:Beware filesystem corruption by dannyspanner · · Score: 1
      Nope, most definitely not an urban legend. We had it happen a few times at my old employers.

      We have little plastic labels that you insert between the plug and socket. They have "LEAVE ON" written on them in big black letters. A very cheap way to avoid a headache.

    3. Re:Beware filesystem corruption by RGRistroph · · Score: 2

      I have never had data loss with linux unless it was me inadvertently deleting something. I loose my machine to power outages at least once a month, and once when netscape was open it came back with an empty bookmarks file, but the whole bookmarks file was in /lost+found. I also regularly start huge compile runs on my laptop, put it in my backpack, and let it compile until the battery runs out. When I plug it in it fsck's on the way up, but I just restart the compile and it is fine. So I don't have your experience of loosing data on a power outage.

      I recently bought a cheap big IDE drive for my home machine. I plan to have a couple of partitions for experimenting with ext3 and other journaling filesystems. But I don't really ever stress the ext2 filesystem, so all I can test is that it is as good as ext2. How would you test a filesytem for reliability and crash resistence ?

      Another question: I have noticed that access from linux to a FAT32 partition is much slower than from windows to a FAT32 partion. Linux FAT32 access is also much slower than Linux ext2 access. I would expect some difference -- who wants to spend a lot of time making the FAT32 fs code fast -- but the difference seems rediculous, at least 2 times slower. Why is this ?

    4. Re:Beware filesystem corruption by RGRistroph · · Score: 2

      If you have problems with ext2 you might want to look at some of the links posted here:

      http://slashdot.org/askslashdot/00/01/22/1958212 .shtml

      Why is the root of this thread marked offtopic ? I'll have to start meta-moderating more regularly.

  3. You aren't ready for FreeBSD or Linux. by draziw · · Score: 2

    If you are talking about running Linux with XFS - you aren't ready to run Linux. XFS is not in the Linux kernel. Before you consider switching away from NT, gain some admin time on a FreeBSD and/or Linux box, and see how they work. Work with Samba and see how the combo meets your needs and if you are up to admining the system. If so, you may want to look at ext3 or reiserfs (journaling filesystems) that should be more robust than ext2.

    1. Re:You aren't ready for FreeBSD or Linux. by RGRistroph · · Score: 2

      I think that sounds a little overly discouraging. I would say that the asker is probably much better equiped to learn about and set up Linux than most new users.

      Rather, I would advise that you just check out a linux installation that is as standard as possible. See if that is reliable enough, and don't experiment with any new filesystems unless plain vanilla won't cut it.

      Also, you said it was an "NT Server". Could you give more details on what it is serving ? If it is just diskspace to windows client machines, it should be easy to configure.

  4. Unplugged for a vacuum by RGRistroph · · Score: 1

    "And make sure the cleaning people don't unplug your server to plug in a vacuum cleaner.(or is that an urban legend?)"

    I think what Paul is refering to here is not the fact that the late night staff occasionally unplug machines (an environmentally conscious janitor were I work used to turn my linux PC off -- surely similar things happen everywhere) but a particular old 60s or 70s anecdote about an IBM mainframe that always mysteriously and unexplicably crashed on a particular day of the week, at a certain time at night; and a support team at IBM was so baffled that they actually went to the site one night and sat down to watch it happen; and the janitor entered, unplugged the machine and plugged in the vacuum. It turned out he always vacuumed that room on that day of the week.

    I believe this story is in the New Hacker's Dictionary, the most recent edition. Unforntunately my copy is at home so I can't consult it.

    It's a neat anecdote when well told. It would be interesting to know if it were true or urban legend.