Slashdot Mirror

← Back to Stories (view on slashdot.org)

Mac OS X 10.3 Defrags Automatically

Posted by pudge on from the hot-character-adaptive-respawning dept.

EverLurking writes "There is a very interesting discussion over at Ars' Mac Forum about how Mac OS X 10.3 has implemented an on-the-fly defragmentation scheme for files on the hard drive. Apparently it uses a method known as 'Hot-File-Adaptive-Clustering' to consolidate fragmented files that are under 20 MB in size as they are accessed. Source code from the Davwin 7.0 Kernel is cited as proof that this is happening."

13 of 181 comments (clear)

  1. Amortized cost... by Ianoo · · Score: 3, Interesting

    Obviously doing this process slows down file access a little. I wonder whether any safeguards are in place, such as turning the system off after a certain I/O load is reached? If not, this may not be such a good idea.

    Also, I wonder whether if you were to calculate the extra time (perhaps 500ms) to defragment each fragmented 20MB file against doing a manual defrag every month, and whether it's actually worth it...

    Don't some Linux filesystems already do this to some extent? I could be hallucinating again, but I'm sure I read this somewhere.

    1. Re:Amortized cost... by Steve+Cowan · · Score: 2, Interesting

      Yes it would be a one-time hit, but we hard disk intensive audio and video people don't want to be streaming multiple tracks off our hard disks while they are defragging themselves!

      Self-defragging might be great on file servers but Macs are (largely) about the multimedia.

    2. Re:Amortized cost... by clifyt · · Score: 5, Interesting

      "Drives are defragged to allow the OS to access the files faster."

      Are you so sure?

      I have talked with a senior OS designer (one of the non-free ones) and his view is that these days, defragging does more damage than it saves.

      Why? Drives generally have large caches on them and multiple platters / read heads.

      Noting this, the fastest way to get data off a drive might not be a straight line. Its looks pretty when you run the different utilities and makes the home makers of whom believe everything should be put away neat and tidy, but the engineer had mentioned that being defragged means you loose a lot of advantages of those multiple readheads and cache. He claimed that it was actually better to leave your drive to its own devices, allowing for about 30% free space at all times, and you will see a speedup over a defragged drive.

      I didn't believe it at first, but his arguments did make a lot of sense even though it went against everything I had learned before. He actually mentioned if he had his choice, he'd make certain defraggers would NEVER work, but the market believes that these are necessary so its easier to have these things included as well as supporting third parties, so its there.

  2. In other news.... by m0rph3us0 · · Score: 0, Interesting

    Darwin is welcomed to 1980's filesystem technology. But instead of defraging FFS just makes better decisions as to where to put files. This also explains why the hard drive on my iBook seems alot hotter since upgrading. Seriously, if Apple got rid of HFS, none of this technology would be necessary.

    1. Re:In other news.... by anarkhos · · Score: 2, Interesting

      If Apple got rid of HFS+ they would need to replace it with something else. No other filesystem supports FileIDs for example.

      Time for HFS++

      --
      >80 column hard wrapped e-mail is not a sign of intelligent
      >life
    2. Re:In other news.... by squiggleslash · · Score: 3, Interesting
      Er, when you install OS X, you can choose UFS instead of HFS/HFS+. Then you get an old fashioned Unix-style file system. Unfortunately, doing so means that you lose metadata, forks, etc (though the Finder does a sort-of half-arsed job and creates little dot files all over the place to try to at least cover some of the metadata.)

      Me, I'd take the comparatively modern HFS+. I'm still confused as to why metadata isn't being taken seriously by the rest of the computing world.

      --
      You are not alone. This is not normal. None of this is normal.
  3. What exactly are.... by Creepy+Crawler · · Score: 2, Interesting

    MacOS FileID's?

    Are they comparible to what Reiser4FS will have? Are they better that XYZ offering in Linux?

    I'm seriously interested in what EXACTLY they are. Please spare the fanboy attitude if you do wish to answer..

    --
  4. XP has a similar feature by Doc+Squidly · · Score: 2, Interesting

    Windows XP has a similar feature that waits a until the computer is not in use for a certain amount of time. It would make sense that Apple would give users the same option.

    --
    I think I think, therefore I think I am.
  5. XP way maybe not so good. by goombah99 · · Score: 2, Interesting
    Windows XP has a similar feature that waits a until the computer is not in use for a certain amount of time. It would make sense that Apple would give users the same option.

    I'm Not sure the windows approach is really better. Notice that the apple approach is more minimalist in moving files.

    • If you aren't actively using a file it wont get moved--that's good since moving a file always entails a tiny but finite risk of corruption.
    • (notice that the apple method relies on journaling to save your butt if the computer crashes mid write.)
    • The apple program doesn't even activate unless the file is fragmented more than 8 ways-- again minimalist.
    • the windows program wont be able to move files that are currently open (I would guess). and of course those are exactly the files you will want to defrag most!
    • the windows program is probably taking a risk that some baddly written program will suddenly decide to read or write to a cached but now dead file pointer.
    • apple gets away with this at almost no cost by defragging files it was going to read in anyhow for another reason. thus the read adds no time. For many user applications probably wont be going any other disk reads right after that. (e.g. read in the word doc and start editing it).
    --
    Some drink at the fountain of knowledge. Others just gargle.
    1. Re:XP way maybe not so good. by drsmithy · · Score: 2, Interesting
      If you aren't actively using a file it wont get moved--that's good since moving a file always entails a tiny but finite risk of corruption.

      Not if you do it intelligently (copy data, compare to original, delete original as an atomic operation).

      (notice that the apple method relies on journaling to save your butt if the computer crashes mid write.)

      That mightn't save your file data. AFAIK HFS+'s journalling is metadata only.

      the windows program wont be able to move files that are currently open (I would guess). and of course those are exactly the files you will want to defrag most!

      OS X allows moving the physical data around of an open file ?

      the windows program is probably taking a risk that some baddly written program will suddenly decide to read or write to a cached but now dead file pointer.

      How do you figure that ?

      apple gets away with this at almost no cost by defragging files it was going to read in anyhow for another reason. thus the read adds no time.

      And Windows (assuming it actually does it) "gets away with it" by doing all this when the machine is idle. And while the read adds no time, the subsequent seeking, rewriting and deleting certainly does.

  6. Re:Necessarily Useless by berniecase · · Score: 3, Interesting

    Fragmentation is a very real problem for people who need lots of contiguous free space, especially those working with multitrack audio and video files. They can't have drive heads searching around a drive for free blocks of space when they could be writing linearly.

    Even with this file defragmenter built-in, a drive defragmenter is still needed for certain types of users.

  7. Re:No youre wrong, parent is right by drsmithy · · Score: 2, Interesting
    Every time you move a file you risk corrupting it.

    By the logic you seem to be applying, every time a file is accessed you "risk" corrupting it.

    It does not matter if you double check what you wrote, because that only decreases the chance of making an error. it does not eliminate it. you might make the same read error twice in a row.(e.g. to make this plausible imagine say a weak magnetization that flips after a temperature change later that night). or perhaps you may have read the file wrong in the first place. or something might go wrong when you are updating the file allocation table or there might be some bug in the code that makes an error.

    Please explain why these events are more likely to occur to a file's new location than the old one.

    the point is that if you just left the file alone inthe first place its safer.

    Why ? It's equally as vulnerable to all the same one-in-a-billion events.

    thus the apple approach of only defragging a file in active use and leaving all the other's alone may be preferred to a blanket de-fraggmenting of a disk.

    So, assuming you're correct, the Apple method is preferred because it (minutely) increases the chances of corrupting the files you access, as opposed to some random file ?

    Tell me, which file do you think you're more likely to care about - a random file X from the set of all files on the drive or a random file Y from the set of files you deliberately access ?

    furthermore, when was the last time you tried de-fraggmenting a 500GB disk? do you have any clue how many days this would take??? The apple approach of doing it just-in-time makes a lot of sense. It follows the same logic as jounaling, which is in part a response to no having to fsck a 500GB disk.

    After many years of experimenting with defragging and not defragging, I've come to the conclusion that it makes bugger all difference whether a disk is fragmented or not. If I can't tell the difference, it's not worth doing.

  8. interaction with "secure" delete by mjs · · Score: 2, Interesting

    I wonder how that interacts with the "secure" delete. Does it seeks out previous copies of the file and securely delete them too? That would be quite a feat.

    (Also, has anyone confirmed that the code snippet is actually executed?)