Slashdot Mirror
Measuring Fragmentation in HFS+
keyblob8K writes "Amit Singh takes a look at fragmentation in HFS+. The author provides numbers from his experiments on several HFS+ disks, and more interestingly he also provides the program he developed for this purpose. From his own limited testing, Apple's filesystem seems pretty solid in the fragmentation avoidance department. I gave hfsdebug a whirl on my 8-month-old iMac and the disk seems to be in good shape. I don't have much idea about ext2/3 or reiser, but I know that my NTFS disks are way more fragmented than this after similar amount of use."
I know this!
The preceding message was based on actual events. Only the names, locations and events have been changed.
but I know that my NTFS disks are way more fragmented than this after similar amount of use
Is this based off of instinct, actual data, or what?
STOP! Don't run it! It's a trojan horse. You'll be sorry!
PS - I do not work for Microsoft, so you can believe whatever I say.
As mentioned in the article, HFS+ does defragging on the fly when files are opened if they are less than 20MB. The source code for this is available here, as is a discussion about it that contains input from some Darwin developers.
what type of file system is there where there is no main allocation table just a header then the file then a header then the file so you could theoretically break a disk and still read the half that was good because all pertinent information relating to a file was in one place?
Doesn't Panther include some sort of on the fly defrag built into the system? So when files are used more often they grouped together?
Goto My Computer. Right click the drive to be analyzed. Select tools/defragment now.../Analyze.
This was my PhD Thesis.
I'm no MS fan, but the latest release of NTFS is pretty solid. The only time I've ever had a problem is with a failing HD, but even then, NTFS was able to recover enough that I could save the data to a new disk.
See on a windows machine there are so many great games and apps to install that you end up installing and uninstalling stuff all the time leading to fragmentation. But on Mac's, well you tend to keep those precious few fun games and useful apps because you have no choice, thus the HDD tends to stay the same. :P
It must be pretty damn good if it can outdo NTFS. I have three computers with WinXP (NTFS 5.1) that I run quite a bit of data through on a daily basis, and neither needs to be defragmented very often at all (two of them have never needed defragmentation in more than a year of use). Mind you, I might fall into some special category of people who don't fall victim to fragmentation for some reason. Anyway, my point is, before you make remarks regarding how well this compares to NTFS, and/or how much "Microsoft sucks", consider how well NTFS still holds up considering its age. Another bonus is, I don't risk losing file system integrity if there's a power failure. ;)
Quality, performance, value; you get only two, and you don't always get to pick.
I've had a continued problem on my iBook for the past year or so.
Under HFS+ in Mac OS X Jaguar or Panther, after about a day of having a clean install, fresh partition and format my hard drive starts making clunking noises and the system locks up (without actually freezing) -- then when reboot attempts are made they take aeons.
Under ReiserFS in Gentoo Linux for PPC: never have the problem. Same hard drive. Months of use, never once hear the hard drive being funky. No lockups.
Do I put the blame on HFS? OS X? I just can't figure out this strange problem.
d. Taylor Singletary,
reality technician techra.el
I've got a G4 with an 80 GB root drive which I use all day, every day. Well, almost. It's never had anything done to it, filesystem-maintenance-wise, since I last did an OS upgrade last fall, about eight months ago.Not too shabby, methinks.
I write in my journal
The near total lack of applications limits the creating of the pesky data files that so damage the contiguous layout of the filesystem.
I Never really had a problem under Linux until I started using P2P..
I know that Reiser does extremely well with space management on small files (CDDB database is a great example). Do any of the other Linux FSs do better than EXT2 with frag?
Comment removed based on user account deletion
Slashdot wouldn't be Slashdot without an un-backed-up dig at Microsoft, would it? Come on people, there's enough wrong with Microsoft that has evidence to back it up without cheap, snide shots like that.
Get your own free personal location tracker
it's not how fragmented your disk is, it's what you can do with your fragmented disk that counts.
CVB
free ipod and free gmail!
Comment removed based on user account deletion
I'm sure someone else will point this out as well but its worth noting. In 10.3 there is kernel level defragmentation. When a file is accessed the kernel checks to see if its fragmented, then moves it to a area of the disk where it can exist unfragmented. I think there is a limitation to file size under 20MB but it may be higher. This still gets rid of a great deal of fragmenation. Just food for thought.
But not sure how this are managed in linux filesystems, not just ext2/3 and reiserfs, but also in xfs and jfs.
I can't find any info about this on the site. Is anyone else getting this error?
Recent 2.6-mm kernels contains Chris Mason's work in order to dramatically reduce the fragmentation of ReiserFS filesystems.
It's really good on filesystems with a lot of files or on databases.
{{.sig}}
Seriously, with NTFS and HFS+ I see very little fragmentation on both my Wintel and Apple machines.
Both have 40gig HD's and both have applications installed/uninstalled quite often. My PC feels the worst of this as he gets games installed and uninstalled in addition to the apps.
For example the last time I reinstalled either of these machines was back in january(new year fresh install) and since then my pc has felt the install/uninstal of various games usually ranging from 2-5 gigs each. The Apple has been installed and with the exception of updates, plugins, video codecs and basic small apps that get added/upgraded often has done alright.
Right now Norton System Works on my PC is saying the drive is 4% fragmented. Disk Warrior on my Apple is saying the drive is 2% fragmented.
Conclusion: Fragmentation is no longer an issue for the HOME USER(note how i'm not saying your companies network doesn't need to be concerned), unless there still running a FAT32 partition >. which well they deserve to have there computer explode at that point anyway.
Ave Molech Setting
Has anyone got a similar prog to measure fragmentation for reiserfs, ext2/ext3, and NTFS? And, what the heck, FAT32 as well? Could be very useful...
-- Manik Surtani
Since NTFS disks are likely to hold more than 3 applications....
Why is it that everytime Steve Jobs farts and it smells better then Gate's a@@ that I get to read about it on the frontpage of /. I get so sick of this, it's enough to make me delete my /. bookmark
http://jayceecorder.blogspot.com
That's not quite correct. In Panther (Mac OS X 10.3, for the uninitiated), journaling is enabled by default: that is, when you first install Panther, it will add journaling to your existing HFS+ disk, and if you're reformatting, it will default to HFS+ (Journaled). However, prior to Panther, there was no journaling support in HFS+, to my knowledge.
Dan Aris
Fun. Free. Online. RPG. BattleMaster.
I just put my hard drive in my drier when it is fragmented. Since the group of unfragmented bits weighs more than the fragmented ones, The spinning action causes all of those stray bits to attach to the greater mass.
Describe, then the automatic capabilities of WinXP's implementation NTFS 5.1. What happens when you run a third party defrag tool, how often do you run it?
Mind you, I might fall into some special category of people who don't fall victim to fragmentation for some reason.
Why? How do you use your system?
You are the victim of the "click of death" this is a very well documented HD defect (not an FS defect). Tell Apple about it, or go to an Apple store. They will replace your HD free of charge.
I boycott signatures
I've often wondered if defragging and defrag utils are more of a placebo for people concernced with system performance. In my experience I've never noticed any perceivable difference after using a defrag util, on either OS8/9, OSX, 95, 98SE or XP. Then again, I've always made sure to have plenty of free space on my disks [and made sure they were fast enough] whenever I've done anything seriously disk intensive like multitrack audio...
I'll turn into a supernova and burn up everything. Well I'll turn into a black little hole and you'll turn into string.
were entirely devoid of glowing praise for all things Apple. Amen.
I trust you will not be so reckless in the future. YOU HAVE BEEN WARNED!
A lot of people simply equate disk fragmentation with slow application execution and opening of data files. While this is the most visible effect that fragmentation has on a system, it's not the only one. If you are dealing with large files (multi track audio, video, databases) then you will get a different kind of performance hit due to the non-contiguous nature of the free space you are writing to. If you want to capture video with no dropouts, you really want a drive that has all of it's free space basically in one location. This allows you to write those large files with no physical disruption in location. Please do not think that the only benefit to unfragmented space is just "my programs launch faster". If you do any real kind of work on your system with large data files, you should know that a defragmented drive is a godsend.
Who is Twirlip of the Mists?
I know the drive is hosed. But no data ever actually seems to get corrupt. For the past year I've moved stuff on and off and never have seen any major problems with the data itself, and thus to be safe I never put anything really important.
HD replacement: will they do this for free even if you aren't an Applecare customer? My warranty ran out about a year before this problem began.
d. Taylor Singletary,
reality technician techra.el
I have a program from there that at startup will check the MFT, swapfiles and other important files and will make each one contiguous collection of disk blocks. Gotta be done then, as you can't lock them once Windows is completely up.
Blar.
I'm on a WinXP PC day 15 after install (NTFS). This PC has been used for web surfing/e-mail only. -No downloads -No Multimedia content According to the Windows Disk Defragmenter: "You should Defragment this Volume" upon closer inspection: "Total Fragmentation = 12%" "File Fragmentation = 24%" It is 85% free space. Hmmmmmm. Any explanations?
I boycott signatures
Buzzsaw and Dirms -- I admit, the site looks a little seedy, but I've used both of these programs on several machines for upwards of a year and they've done a superb job of keeping my NTFS disks defragmented.
http://dictionary.reference.com/search?q=Anecdote
anecdote n.
1. A short account of an interesting or humorous incident.
2. pl. anecdotes or anecdota (-dt) Secret or hitherto undivulged particulars of history or biography.
The Farewell Tour II
This is a very arcane procedure in XP. I shall try to explain, but only a professional should attempt this.
1. Right click on drive icon, select properties
2. Select Tools tab and click on "Defragment Now"
3. Click on "Analyze"
4. When analysis finishes, click on "View Report"
This shows two list windows, one containing general properties of the disk such as volume size, free space, total fragmentation, file fragmentation and free space fragmentation. The second list shows all fragmented files and how badly they are fragmented.
Mmmm the key here would be that it is somewhere documented in their database. IE: You complained about HD noise at some point while you were under Applecare waranty (and they had a record of the complaint). You could always sign up now for Applecare, telling them that everything is working fine. Then luck have it, in a week or so, all of a sudden -the HD starts making noise!
I boycott signatures
Jaguar (10.2) has journaled support as well, but you had to enable it as it was not a default option.
Even in 10.3 it's optional, not required, but it's the new default for new disks. Probably because Apple decided that their code was solid enough to put into production. After testing it on 10.2 I agree with them.
Is this the article that is supposed to make me feel better about not having defrag in OS X?
this sig limit is too small to put anything good h
at how Microsoft still manages to convince everyone that fragmentation is "part of the computing experience".
You could enable journalling on HFS+ drives in later versions of Jaguar. It wasn't on by default, though.
I'm not one to nitpick, but for Pete's sake, couldn't you have done better than "un-backed-up"? That's just pathetic.... I don't even want to look at your posting history...you probably are one of those people who uses words like "funner," aren't you?
But there is another kind of evil that we must fear most... and that is the indifference of good men.
Could you please look that up in the Oxford English Dictionary? I don't trust so-called "American" dictionaries to tell me what is and what isn't a word.
Proper English speakers use the word "antidote," not the phony American creation "anecdote."
I'm not Seth Finkelstein. I still speak the truth.
http://docs.info.apple.com/article.html?artnum=256 68
Mac OS X: About Disk Optimization
Do I need to optimize?
You probably won't need to optimize at all if you use Mac OS X. Here's why:
BSD filesystems like UFS have a thing called fragments, which are like partial blocks. Fragments help keep the wasted space of partially used blocks to a minimum. This number is often reported by fsck.
These kind of Fragments should not be confused with 'Fragmentation', which is when the blocks composing a file are spread all over the place.
[a. Fr. anecdote, or ad. its source, med.L. anecdota (see sense 1), a. Gr. things unpublished, f. priv. + - published, f. - to give out, publish: applied by Procopius to his ‘Unpublished Memoirs’ of the Emperor Justinian, which consisted chiefly of tales of the private life of the court; whence the application of the name to short stories or particulars.]
2. a. The narrative of a detached incident, or of a single event, told as being in itself interesting or striking. (At first, an item of gossip.)
Except for FAT and similar ancient file systems, hasn't this been a dead issue for 10 or so years?
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
What kind of a word is "anecdote" anyway? Words can't have a 'c' and 'd' right next to each other like that. It's simply unpronounceable.
My new laptop, with a 60GB 7200RPM disk, is under two months old, and I'm defragmenting it now. It's been running for 5 minutes, and is 3% complete, on a disk that is 62% full.
20 minutes later, and it's on 17%. That's pretty damn fragmented, in my opinion.
There is a fairly comprehensive procedure for Defraggling your Motherdisc here. Good Luck!
I doubt that. They replaced mine, but that was only three days after the warranty ran out and after the entire harddisk had shuffled off this mortal coil. Every single byte was lost. (But luckily backed up beforehand.)
only if you are thinking they are part of the same syllybal(i can't spell today)
anecdote ~= An Ek Dote
Not bad. That's 8 months of heavy use since my last format.
I gotta bring this to work today and see what that machine's like. My co-worker has been complaining that he doesn't have a defrag utility since he got OSX. I've been telling him that I don't think it matters. Now I can prove it to him.
I remember back in the days of my Powermac 8100/80av, we would leave the 2 800mb drives defragging over the weekend because they had like 75% fragmentation.
...spike
Ewwwwww, coconut...
Straw man. HAND.
It's a perfectly valid test -- it measures how much fragmentation can be observed after a certain amount of use. According to your logic we couldn't compare any properties of NTFS/ReiserFS/FAT32/HFS+ because they work differently.
HAND.
Yes and no, it won't have any long time effect on your performance but there is a short time effect that can be usefull when dealing with audio. On a Mac, using a drive with block sizes of 64K to 256K (ideal when dealing with digital audio, as long as you set the buffer per track size of your daw to the same size as the blocks on your drive) you can gain up to 8 tracks by defraging your drive. Sometimes on large projects I have to record a file or playback the entire session in edit mode (no tracks frozen, everything real-time and not bounced), after editing for a while the daw refuses to play the project, lags, stutter or present some serious drop-outs, I defrag and this is where I get this 6-8 tracks headroom, but that will last only for a day of work and even then (Pro-Tools, Nuendo, Cubase, MOTU DP all present this caracteristic, as for the other I haven't tested them enough to provide meaningfull data).
however, defraging is not the same for every defrag utility. For example, I was working with Avid Audiovision about 5-6 years ago on a TV show, it seems that defraging a drive hosting files created or edited with Audiovision with Speed Disk by Symantec would actually corrupt the entire projects contained on the drive (the biggest mistake and the only serious one I had in my career, I didn't loose my job but my boss did loose his temper, live and learn!), audio file were not readable at all after, it was actually a documented bug of Audiovision and I even think it was affecting every OMF files not just the ones used by Audiovision (not sure about this though), thats what happens when your boss won't let you RTFM. Only Disk Express, some Avid defrager or, later, Techtool could defrag those drives.
On a side note, in the Classic mac (7-9.2), defragmenting your drive was also a way to prevent data corruption, actually its the other way around, not defraging would lead to data corruption. I don't know if its also the case with NTFS, EXT2 et al.
Mostly because they end up re-installing the OS every year or so!
Competition Good, Monopoly Bad.
Beg pardon, I meant 18% fragmentation. My question remains.
A great way to defraggle your harddrive is described here:
http://www.datadocktorn.nu/us_frag1.php
It looks like it works both with Windows and Macintosh drives.
Result: you have a bunch of large files, all very fragmented, and the free space is very fragmented.
fragmentaton is highly probable on essentially one dimentional datastorage system like a drive. If you had a two dimensional system it would be less likely. How many dimensions, I wonder would you need, to make it extremely unlikely. No I will not post pictures of my 4 dimensional harddrive ;-)
But no data ever actually seems to get corrupt.
Seriously, get a new drive. The most important thing on your entire computer is the data, if you have ANYTHING worthwhile on the computer it's worth it. If you can't afford a new drive, then consider backing up your data nearly every day. Bad enough that regular people aren't prepared when their drives crap out, but when you've been hearing the click of death and lose everything you'll be kicking yourself.
This article, Interview With the People Behind JFS, ReiserFS & XFS asks the developers directly about fragmentation. XFS, JFS, and ReiserFS are extent-based file systems, which keep fragmentation to a minimum.
I dunno, somehow back when I used Windows. Just after installing and updating on a freshly formatted drive, I'd notice it was due for a defragging.
Someone set us up the bomb, so shine we are!
I've noticed that my home file server, work file server, and workstations go CLUNK and hughalhughalahughalahughala around four in the morning ("bedtime")- when cron runs by default. This causes iTunes and VLC to skip on a G4, and brought my fileserver to its KNEES. (nothing chungs like an SCA drive! :D)
It's a unix. It has timed jobs that optimize the system. My powerbook ran like ASS until I left the thing on and running for 24 hours, after which it seemed to get its act together.
Does anyone else think that statement is a bit odd? Maybe it's just me, but I think he's being a little bit presumptuous about the programming skills of the average geek site browser.
Transistors and Beer!!
As far as disk optimizations go, defragmenting is the dumbest, least interesting thing you can do. In fact, if you don't have a defrag utility for your FS, just copy all of your stuff somewhere else (disk/tape/network) and copy it back. Tada. Unfortunetly, it is still useful for most of the filesystems today.
Defragmenting is stupid because it only works on the file level. And only helps in situations where files are read entirely in sequence. This usually tends to be media files and config files.
Executables, shared libraries are usually mapped into memory, and only read from disk when needed. Database files are rarely read sequentually.
What you actually need to do is to FRAGMENT your FS. Analyze what blocks are most commonly used, and bunch blocks together that are used at similar times. This would bunch all of your startup executables/scripts together on the disk. And, when you execute openoffice.org, only the most often used parts of the executable are cached/ loaded.
Ideally you wouldn't see your harddrive thrash when booting, or starting OO, but this is something that a standard defrag won't help.
Programs that have a lot of data files will need to seek to each file. This is why a defrag at the file level is not always helpful.
Oye.. This one bit me in the arse awhile back. Thought i'd defragment one day, and found that most of my drive (well over 70%) was fragmented. To top it, windows built in defrag could do nothing to help it.. Each pass would clean up maybe 1 percent. In the end, I basically wiped the drive and started fresh.. wasn't worth the hassle. And yes, it does feel zippier now.
... I hate cliffhangers!
Slashdot still doesnâ(TM)t support Unicode after it was added to the HTML standard in 1997.
My reccollection of the OS/2 HPFS file system from IBM was that in many cases it would purposely fragment to take advantage of the disk spin, thus using fragmentation to increase performance.
Defrag utils for OS/2 had options to only defrag if there were more than 3 extents, to avoid nullifying this effect.
funny, years after the death of OS/2, it still kicks ass on much what we use now.
That would be well and good if the problem were otherwise insurmountable. But, it turns out, we've known how to minimize, if not entirely eliminate, filesystem fragmentation for twenty years now - since the introduction of the BSD Fast File System.
It doesn't take expensive (in time, if not in money) tools. All it takes is a moderately clever block allocation algorithm - one that tries to allocate a block close in seek time to the previous one, rather than just picking one at random.
The fundamental insight that the authors of FFS had was that while there may only be one "optimal" block to pick for the next one in a file, there are tens of blocks that are "almost optimal" and hundreds that are "pretty darn good." This is because a filesystem is not a long linear row of storage bins, one after another, as it is treated by many simplistic filesystems. The bins are stacked on top of each other, and beside each other. While the bin right next to you might be "best", the one right next to that, or in another row beside the one you're on, or in another row above or below, is almost as good.
The BSD folk decided to group nearby bins into collections and try to allocate from within collections. This organization is known as "cylinder groups" because of the appearance of the group on the disk as a cylinder. Free blocks are managed within cylinder groups rather than across the whole disk.
It's a trivial concept, but very effective; fragmentation related delays on FFS systems are typically within 10% of optimum.
This kind of effectiveness is, unfortunately, difficult to achieve when the geometry of the disk is unknown -- and with many modern disk systems the actual disk geometry is falsely reported (usually to work around limits or bugs in older controller software). There has been some research into auto-detecting geometry but an acceptable alternative is to simply group some number of adjacent blocks into an allocation cluster. In any case, many modern filesystems do something like this to minimize fragmentation-related latency.
The gist of this is that Microsoft could have dramatically reduced the tendency towards fragmentation of any or all of their filesystems by doing nothing else but dropping in an improved block allocator, and done so with 100% backward compatibility (since there is no change to the on-disk format).
Maybe it was reasonable for them to not bother to so extravagantly waste a few days of their developers' time with MS-DOS and FAT, seeing as they only milked that without significant improvement for eight or nine years, but it's hard to explain the omission when it came to Windows NT. NTFS is a derivative of HPFS which is a derivative of FFS. They had to have known about cylinder group optimizations.
So the fact that, in 2004, we're still seeing problems with filesystem fragmentation absolutely pisses me off. There's no reason for it, and Microsoft in particular ought to be ashamed of themselves. It's ridiculous that I have to go and degragment my WinXP box every few months (which takes like 18 hours) when the FreeBSD box in the basement continues to run like a well-oiled machine despite the fact that it works with small files 24 hours a day, 365 days a year.
Hey Microsoft: You guys have like fifty billion bucks in the bank (well, ok, 46 or 47 billion after all the antitrust suits) and yet you can't even duplicate the efforts of some hippy Berkeleyite some twenty years after the fact? What's up with that?
(I mean "hippy Berkeleyite" in an affectionate way, Kirk. :-)
jim frost
jimf@frostbytes.com
Panther optimizes on the fly and after any installation. Fragmentation is really a non-issue.
I hate sigs.
Panther:/Users/coyote/VPCs/Windows 2000.vpc6/Windows 2000 Drive 1.vhdp/BaseDrive.vhd
Ah, so its Windows!
Self defragmenting my arse!!
Try editing digital video using Final Cut Pro for a couple months. My filesystem got so fragmented on a data drive (120GB), that FCP became unusable. I had to buy another drive and copy all the files from the fragmented disk to it. And it took *days* to do the copy. This process repeats every couple months.
All this time I thought defragging my hard drive meant removing my copy of Quake from it.
Live and learn.
If what you say is true, I'd like to know what eMule and BitTorrent (3.2 and before) do to mess with it. 'Cos they both pre-allocate files with the size of the download, and these files _always_ get freaking heavily fragmented even before being written to (and yes, with plenty contiguous free space available).
n/t means no text... duh
so why are you reading this
- Disclaimer: Information in this post deemed reliable but not guaranteed.
As a minor aside, it seems that the latest iPod update enables journalling on HFS+ iPods. This would be fine in most cases, but I found this out the hard way as I also use my iPod with my Debian box. The new hfsplus module in the 2.6 kernel doesn't seem to understand HFS + (journalled) yet, and would only mount it read-only until I disabled the journal.
here is my drive after 6 months of usage:
Out of 242682 non-zero data forks total, 242594 (99.96 %) have no fragmentation.
Out of 5378 non-zero resource forks total, 5367 (99.80 %) have no fragmentation.
Seems better then my machine at work on NTFS which shows up as a big red square in disk defragmenter.
Oh, right; that's true. However, it was not introduced until 10.2.8, which might (not sure about this) have been released after Panther was released. It was certainly released after WWDC 2003, when Panther was announced.
So there was no support for it before Panther, but there is support for it in pre-Panther versions. My nits stand picked ;-)
Dan Aris
Fun. Free. Online. RPG. BattleMaster.
One of the reasons I've stuck with Apple over the years- it's the little details like this that seriously add to the experience, and make older versions of the operating system so painless to troubleshoot. :D
There are so many comments already posted to this topic that seem to not grasp the following point, that I think the best way to deal with it is to start a completely new thread. I'm sorry if it seems more than a little obvious to some of you:
:-), such as until you would need to perform disk maintenace anyway for lots of other reasons in any 'reasonable' file system. A typical media file is probably Type 1 in 99%+ of cases.
.txt format, while it is stll being edited by its creator. (type 2). (That same document may behave as effectively Type 1 once it is finished, only to revert to type 2 when a second edition is created from it.)
.doc file, that may become longer for obvious reasons like more text, but may also become longer for less obvious reasons (such as the hidden characters created when you make some text italic or underlined). (These are reasons that are not obvious to most end users, and often not predictable in detail even to people who understand them better). The default configuration for a Windows swap file is type 2a. It is likely to be hard for an automated system to predict the final size of Type 2a files, as that would imply a software system of near human level intelligence to detect patterns that are not obvious and invariant to a normal human mind. It may be possible to predict in some cases only because many users are unlikely to make certain mistakes, (i.e. cutting and pasting an entire second copy of a text file into itself is unusual, while duplicating a single sentence or word isn't).
.bmp, which will only get larger or smaller if the user changes the color depth or size of the image, and not if he just draws something else on the existing one.). A good portion of users (not all by any means) will learn
.zip) is hopefully a 3b, but only until it is run, then the contents may be of any type. A typical Microsoft patch is a 3a (it will somehow always end up longer overall, but you never know just what parts will vary or why).
:-).
There are fundamentally only a few types of files when it comes to fragmentation.
1. There are files that simply never change size, and once written don't get overwritten. (Type 1). Most programs are actually type 1, if you use sufficiently small values of never
2. There are files that will often shorten or lengthen in use, for example a word processor document in
Of type 2, there are files of type 2a. Files that may get either longer or shorter with use, on a (relatively) random basis. (as a relatively simple case, a
Then there are files of type 2b. Files that get longer or shorter only for predictable reasons, (such as a Windows
what to expect for these files, which suggests a well-written defragger could theoretically also auto-predict the consequences of the changes a user is making).
3. Then there are type 3 files, which only get longer. These too have predictable and unpredictable subtypes. Most log files for example, are set up to keep getting longer on a predictable basis when their associated program is run (type 3b). Anything that has been compressed (i.e.
4. Type 4 would be files that always get smaller, but there are no known examples of this type
These types are basic in any system, as they are implied by fundamental physical constraints. However, many defrag programs use other types instead of starting from this model, often with poor results.
In analyizing what happens with various defrag methods, such as reserving space for predicted expansion or defragging in the background/on the fly methods, the reader should try these various types (at least 1 through 3), and see what will happen when that method is used on each type. Then consider how many of those type files will be involved in the overall process, and how often.
For example, Some versions of Microsoft Windows (tm) FAT32 defragger move files that have been accessed more than a certain number of times (typically f
Who is John Cabal?
I have to disagree with you. Back when I had NT4, I could defrag my drive and be back to >90% fragmentation in three weeks. My XP box is less than two months old, the drive is only 35% full, and my file fragmentation is 50%. In my experience, NTFS fragments like nobody's business.
In order for the Hard Drive to remain Static in relation to the point of contact on the dryer, an equal but opposite force must be applied. This causes those bits to be reshuffled. Next up Mechanics of Deformable Bodies.
Which brings us an interesting concept, the inward and outward forces on the drive would act as a form of compression, thereby freeing up even more space on the hard drive. Maybe SCO would like to patent this form of Data compression.
What?. I think you need to change your signature. If there are no WMD, then there could not have been a binary chemical artillery shell filled with nearly a gallon of Sarin in Iraq. This was not an old shell either. Iraq didn't have binary chemical capabilities before the first Gulf War.
Strange women lying in ponds distributing swords is no basis for a system of government.
Nope, you're wrong. 10.2.8 was out well before the release of Panther
Journalling has existed since 10.2.2 (at least on the Server end; I believe the consumer end too, except you had to enable it via a terminal command), so... ^_^
A word about browsers (and any thing else that requires change):
People, in general (more than 50% of them), prefer to resist change, and for that matter, extra work and/or thinking. It's just the way they are. It's what explains product loyalty. In this case, the product loyalty is browser based.
In my job, as a web server support admin, I find that 95%, or more, of the people I speak with in support situations are not even aware of the alternatives available to them. In fact, just last Sunday, a friend of mine was showing off his new Power Book to me (by the way, even though I am a complete Linux advocate, you have to give credit where credit is due: Mac has a great GUI). I had to laugh during his enthusiastic demo of Mac OS X's features when my friend opens up Safari and goes, "Check this out. It's a feature called 'tabbed browsing.'" He was a kid in a candy store and had just found new, profound flavor of buble-gum or something. But, how could I not laugh at this previously 100% Windows user's intron to me of something that I began using in Opera, back around 5.x-6.x (I really don't remember if 5.x had tabs or not. I really don't care since that browser drives me crazy. But that's just me.) Translation: it's be around for years. In my work day I begin with 12-13 of them opening in FireFOx (NT 2000 doesn't like that, even with 512MB RAM, but it gets by well enough). The number of tabs only increase from there, unless there's an accident of closing a tab. But no big deal there either, I just open another one and then drag it back between where I normally would have it in my list of tabs. You won't find any thing like that in a browser direct from MS.
Another example: my co-workers, particularly the NT techs. Most, certainly not all (thank God), of our NT techs still use IE for their work. I don't really know what they need for their work, but I've seen their desktops and their taskbars; WHAT A MESS! It's beyond me why they would waste their time with a browser (read: IE) that doesn't organize their open web pages into one taskbar entity, because they DO use other programs on the NT 2000 desktop, which we all must use at my job, regardless of the servers we admin for. (If you haven't guessed yet, I don't admin for NT servers, I get the please and ease of admining for Linux boxes. And a big THANK GOD for that!)
Back to my point: most people are not aware of features in other browsers AND if they are aware of new inovations (read: tab browsing, which is one reason I will never go back to IE) they are not in any hurry to change and think and evaluate something that, however troubling it can be at times: pop-ups, vulnerbilities, "________________" [fill in the blank], lack of inovation, etc.
So what if most of /. visitors are Windows based? There are plenty of better choices to MS products, even on their own OS platform. But, people the world over resist change; they get stuck in a rut, good or bad in it's results, and they either don't like to change, don't "need" to change, or cannot change. Thus, the end result is resistance to change; for the better or for the worst.
convert the disk to NTFS
<bigger>It is NTFS!</bigger>
Don't blame me, I didn't vote for either of them!
If you aren't in the habit of leaving your macs running 24/7, you can use macjanitor to run these tasks from a nice, friendly GUI.
Seems to be an incredible drop off in the intelligence level of right wing sympathisers in recent weeks. All you see on the boards and chatrooms are pathetic 'me-too' types, parroting back whatever pointless point the 'pundits' and scream-radio hosts are trotting out. Used to be, you had some folks making rational, reasoned, well thought out arguments. Now, all the smart people have jumped ship or shut up. All that's left are the dregs, but they are in absolute flood-mode, squatting and dropping their smelly little memes in the most inappropriate places.
1. Create a partition at the end of the drive that is the size of the paging (swap) file you wish to use.
2. Move the swap file there. (2000/XP - Control Panel>System > [Advanced]> Performance Options > Change Virtual Memory
Remember to set the pagefile on your system/data drives to blank fields (instead of 0), and that every change you make you MUST USE THE SET BUTTON.
Using this I only need to defrag every once in awhile, instead of regularly. I also find windows runs faster, and I don't have to re-install as often.
Cheers!
On any file system. You can have a very fragmented file system in some situations, but typically ANY file system is losing very little performance to fragmentation.
And fragmentation is not a bugaboo. It's a fact. When you have random allocation on a volume, it will get fragmented. You can go back after the fact and unfragment it, but doing so in any serious fashion when writing files actually degrades performance due to the extra effort required.
Also note NTFS is not a derivative of HPFS. I cannot speak as to whether HPFS is a derivative of FFS.
HFS+ does nothing to prevent fragmentation except for use super-clusters. That is, when you extend a file, you get more than one block at at time. When you close the file, some of those blocks might be scavenged, or might not. NTFS could do this, but I believe they do not. However, NTFS on servers has an allocation block size of 8K, which isn't far off of the super-cluster size on HFS+ which I believe is 16K.
Finally, note that on modern drives, you can seek all the way across the disk in only about 30-50% more time than you can seek a short seek. Thus keeping your blocks close to the current cylinder but not in it has very limited value. Note that this is not the case on optical (CD/DVD) disks.
As a user of NTFS 4(WinNT 4.0) for five years,
I noticed a huge problem went I switched to NTFS
5 (Win2K). During the D/L of big files, I would
get an OS alert that there was not enough disk
space free: the report would claim insufficient
space for a 50 MB file while I had over 2 GB free.
The disk analysis tool (MS) would report perhaps
20% fragmentation.
The only way I could proceed with the 50 MB file
D/L would be to log in as the administrator,
defrag the partition (sometimes more than once),
and reboot the system. This problem NEVER
occurred with NTFS 4. And, being the cheapskate
that I am, I never used anything but the MS-
supplied defragmentation tool.
I have never experienced these types of file-
system problems with ANY other OS, including
hpux (10.x), irix (6.x), linux (2.x), solaris
(2.5/6/7/8), or my mac (10.x). The Win2K
filesystem is worst than any of those supported
by any of these other OSes (and the defrag tool
is worst than in WinNT 4).
If I could get (native) support in Win32 for
XFS and HPFS+, I wouldn't use NTFS at all.
(Or better still, rock solid binary support
for those Win32 applications under linux or
max osx, and do away with Win2K completely).
Very cool, very cool hack.
Damn, that was my next project...
I haven't seen one of those in a loooooooong time!
(note to self, save open documents before running experimental software)
Finally, note that on modern drives, you can seek all the way across the disk in only about 30-50% more time than you can seek a short seek. Thus keeping your blocks close to the current cylinder but not in it has very limited value. Note that this is not the case on optical (CD/DVD) disks.
Bzzt. Wrong. Thanks for playing.
Seagate Barracuda 7200.7 Plus product manual. This is a modern drive. Look at page 17 of the PDF, section 2.7, "Seek Time". Track to track seek is listed as sub 1ms, while average seek is 8.5ms (for read). Latency (the rotational aspect), which is separate, is an average of 4.16 (as it should be for a 7200 RPM drive). So a short seek is 8 times faster than an average seek, much less a whole-drive seek. So keeping your blocks close to the current cylinder but not in it still has high value.
Additionally, if you can keep your data in the same track, you don't have any seek time, just rotational latency. And the size of track groups has been growing as densities have been growing. So there are lots and lots of blocks in the same track that aren't within readahead range.
And fragmentation is not a bugaboo. It's a fact. When you have random allocation on a volume, it will get fragmented.
Now you misuse the term "random allocation". When talking about disk, random allocation means that you randomly choose your next block--which certainly will cause fragmentation. I think what you are looking for is random file creation. However, under FFS, if you maintain sufficient free space, it is very unlikely that real fragmentation will occur even with random file creation/deletion. Yes, you won't be able to store all of your files contiguously. But as the grandparent points out, the old FFS block allocator finds "nearly optimal" blocks. From the original paper on FFS, you don't get serious fragmentation-related performance issues until you reach 90% disk utilization (there's an '86 FFS paper I can't find an electronic copy of which does a better analysis, but even the '84 paper has the 90% figure). At 90% utilization, nearly every file system ever starts getting severe performance problems due to fragmentation.
You can go back after the fact and unfragment it, but doing so in any serious fashion when writing files actually degrades performance due to the extra effort required.
You should read about log-structured file systems sometime. Like Sprite. The base idea of a log-structured file system (LFS) is that you don't try to keep your blocks the same. You write a log of changes, and stream that out continuously to make maximum use of your available write bandwidth. This has the severe downside of causing horrific freespace fragmentation, since every change to a file means the affected blocks are reallocated, and the old blocks are now garbage. So you have continuous freespace compaction (called segment cleaning in the paper). LFSs didn't catch on because at the time, they needed a large amount of free space compared to more traditional "overwrite" file systems to maintain performance. However, recent work with log file systems shows that the changing performance characteristics of disks (much higher bandwidth but same seek/rotational delays) have tipped the balance to log structuring. Regardless, the cleaning process is precisely going back and unfragmenting your data, and is necessary for a LFS.
HFS+ does nothing to prevent fragmentation except for use super-clusters.
As pointed out by others, Apple's later implementations do defragmentation of smaller files when they are accessed.
NTFS could do this, but I believe they do not. However, NTFS on servers has an allocation block size of 8K
Anyway, the other guy claimed that NTFS was not a derivative of HPFS. That is not my understanding; I would have learned about the etymology in either the NTFS internals book or Showstopper!, both of which were from Microsoft, but a bit of work with Google shows that the belief is widespread (for whatever reason). In any case HPFS is a very direct descendent of FFS; really the only significant thing they did was simplify the block allocator so they didn't need to know the disk geometry.
So no matter how I look at it, Microsoft knew about the technique no later than 1989. Too bad the only thing they put it in was OS/2, which nobody used.
jim frost
jimf@frostbytes.com
Sorry, slightly OT here. I always wondered why a sysadmin ran nightly defragmentation jobs on home disks on an OSF1 system. Every night from about 1am to 4am, the system would become completely useless. By "useless" I mean it might take 30 seconds to respond to "ls" in a directory with 20 files. Productive work was impossible, and overnight jobs weren't getting much time. Logging onto the file server, I would see about 10 defrag proceses running. I never talked to him much about it, but from your post I'm surprised it was necessary at all, particularly on a DAILY basis!
I don't know what filesystem he was using -- I know virtually nothing about OSF1 from a sysadmin perspective. Well, this was 5 years ago. Now I think he's still running OSF1 for a fileserver that you can't log into, because the OSF1 filesystem is "so much more advanced" than any Linux offerings. Now with LVM2 and the various mature journaling filesystems in Linux, I very much doubt that's true anymore. He's always been of the mindset that you get what you pay for. Our compute servers are all running Linux these days, and that's what I really care about.
One thing people rarely talk about is how fast HFS+ is. Or perhaps how slow UFS on the Mac with OS X is. But the difference is more than dramatic: a clean install of OS X using HFS+ can take less than half an hour - including the developers tools. The same procedure using UFS seems to never end.
It might be the way they've 'frobbed' UFS for use with OS Server, but UFS really gives high priority to disk ops with GUI ops taking the back seat, and yet HFS+ is in comparison blazingly fast.
I believe in a good clean machine like anyone, and I do see the probability DiskWarrior will be needed now and again, but the speed alone is quite a pedigree for HFS+ IMHO.
Anyone have pointers to the algorithms used for defragging HDDs? I've looked before but couldn't find much info.
Well, OK then! :}
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
That's handy! And I wouldn't have known about it otherwise- thanks!
I don't have to prove a goddamned thing. You asked how the shell with sarin fit into the WMD debate, I told you. Now you're changing the question, which ain't gonna work. One shell found a year after Saddam's fall proves dick all. That thing could be 20 years old for all you know.
And I could give a rat fuck if you think I look unhinged, jerkoff. Go back to getting ass fucked by the cock that is the right wing media. Maybe enough propaganda and rhetorical trickery will change the facts, but I'm thinking probably not.
Oh, and by the way: how are Bush's poll numbers looking, babycakes? They hit the 30's yet? And what's the burn rate on those vaunted millions in campaign contributions? And how's that scar on dipshit in chief's chin lookin?
You goddamned egotistical fool. You got ramrodded and aren't man enough to admit it. Pride cometh before the fall, though. I mean, Eye-raq was such a huge fucking threat! Goddamn! At least, that's what IRAN was telling us. Iran told Chalabi, Chalabi told Cheney, and Cheney told Bush. We were a proxy for Iran! Did their work for 'em! Suckers! He was a goddamned spy for Iran, but shit, that's good enough to sit behind Lady Laura at the State of the Union!
Ain't that a total cunt.
Not that you'd know what one of those are.
(signed)
Mr. Proudly Unhinged