Slashdot Mirror
Mac OS X 10.3 Defrags Automatically
EverLurking writes "There is a very interesting discussion over at Ars' Mac Forum about how Mac OS X 10.3 has implemented an on-the-fly defragmentation scheme for files on the hard drive. Apparently it uses a method known as 'Hot-File-Adaptive-Clustering' to consolidate fragmented files that are under 20 MB in size as they are accessed. Source code from the Davwin 7.0 Kernel is cited as proof that this is happening."
Obviously doing this process slows down file access a little. I wonder whether any safeguards are in place, such as turning the system off after a certain I/O load is reached? If not, this may not be such a good idea.
Also, I wonder whether if you were to calculate the extra time (perhaps 500ms) to defragment each fragmented 20MB file against doing a manual defrag every month, and whether it's actually worth it...
Don't some Linux filesystems already do this to some extent? I could be hallucinating again, but I'm sure I read this somewhere.
ahh shucks.
"Slashdot, where telling the truth is overrated but lying is insightful."
If Apple got rid of HFS+ they would need to replace it with something else. No other filesystem supports FileIDs for example.
Time for HFS++
>80 column hard wrapped e-mail is not a sign of intelligent
>life
I think it's a battery-saving feature for the new PowerBooks...
"Flyin' in just a sweet place,
Never been known to fail..."
Surpisingly, yes, it can. If you look at line 94 character 7, you can see the comment where it says it can.
It'll survive, but I'm not sure I will when the next bill comes ;)
GREAT, if one ever had to defrag with HFS+
Me, I'd take the comparatively modern HFS+. I'm still confused as to why metadata isn't being taken seriously by the rest of the computing world.
You are not alone. This is not normal. None of this is normal.
No other filesystem supports FileIDs for example.
How are FileIDs different than inodes then?
-molo
Using your sig line to advertise for friends is lame.
I'm in two minds about this. One, it's a potentially good thing to keep a filesystem constantly running well, but Secondly, I've never had one single process kill more partitions in the last ten years of using computers than when I was defragmenting them. That's my biggest concern.
If I were to get a Mac I'd hope a feature like this could be turned OFF permanently. I'm not one for just using a machine and spontaneously having an unreadable drive.
What about ReiserFS?
In my day, we'd crack open the drive on our Mac SE30s, sharpen a magnet on a whetstone, and defrag that sucker by hand.
Kids these days. It's the MTV, ya know - makes 'em lazy.
MacOS FileID's?
Are they comparible to what Reiser4FS will have? Are they better that XYZ offering in Linux?
I'm seriously interested in what EXACTLY they are. Please spare the fanboy attitude if you do wish to answer..
But instead of defraging FFS just makes better decisions as to where to put files....Seriously, if Apple got rid of HFS, none of this technology would be necessary.
You speak as though HFS+ has trouble with file fragmention. It's easily already one of the best filesystems for avoiding fragmentation - I've worked on Macs that have been run for years without attention and were better than 90% unfragmented. This is considerably better than any of the Microsoft filesystems, for instance. This tweak is an improvement, to get from 90% to 99%.
HFS+ doesn't just put the files down randomly, either, it has some smarts.
This also explains why the hard drive on my iBook seems alot hotter since upgrading.
The only way this feature can do that is if you're writing small files continuously. That's very strange software behavior, and perhaps a worst case for this optimizer. Why would you be doing that?
Don't get me wrong, HFS+ isn't the best filesystem ever created, but it's very featureful (multiple forks, file ID's, case-preserving, case-insensitive-possible, UNICODE, attributes, 64-bit file sizes, POSIX compliance, etc.) and the MacOS relies on it heavily. Anything to replace it would be a superset of HFS+. Fortunately, Apple hired the guy behind the Be Filesystem a few years back. I doubt he's working on iMovie 3.1.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
According to the ARS writeup, this feature is on only when journalling is on. This makes total sense, since journalling prevents an incomplete or unverified write from being used.
Some drink at the fountain of knowledge. Others just gargle.
POSIX compliance
Doesn't being case insensitive violate POSIX? Or has that been fixed?
If you refer to a file by an inode you are basically creating a hard link so if the file is deleted the file still 'exists'.
Also you cannot get a file path from an inode thus if the file path is changed (moving a file for example) the application cannot know what the new path is.
A FileID is really more equivalent to a path, or rather used in place of a path with the advantage that the path can change and the fileID remains the same. Thus referring to a FileID is less fragile.
Also FileIDs are smaller so searching for files using a FolderID or FileID is faster and uses less memory.
They're not equivalent.
>80 column hard wrapped e-mail is not a sign of intelligent
>life
Well that's fine. The real upside of this is for people that have never heard of /. and don't really know what a hard drive is, let alone know how to defrag one.
Previously these people would just go forever without defragging. Now they can still do that, because Apple is doing it for them behind the scenes.
This is yet one more example of Apple's winning philosophy: Keep it simple, make it better.
A Multiplayer Strategy Game for Mac OS X, Windows, and Linux
You can but it's SLOOOOW
>80 column hard wrapped e-mail is not a sign of intelligent
>life
UFS on MacOS X is also very slow and your Aliases will break when the original file is moved or renamed.
>80 column hard wrapped e-mail is not a sign of intelligent
>life
1) Yes it does, break it
2) Fuck POSIX |-)
I'll take usability over some standard which is easy to work-around.
>80 column hard wrapped e-mail is not a sign of intelligent
>life
BFS also lacks many of these things, most notably FileIDs.
I hope he's working on HFS++ instead of some BFS port.
BTW filename extensions still suck...what was Apple thinking?
>80 column hard wrapped e-mail is not a sign of intelligent
>life
> The only way this feature can do that is if you're writing small files continuously. That's very strange software behavior, and perhaps a worst case for this optimizer. Why would you be doing that?
Sounds like compiling to me. Typical usage for a developer.
- Muggins the Mad
Hail Caesar!
2^5
Actually, I usually hear HFS+ described as case insensitive while reading, but case invariant or case preserving while writing. That is, the filesystem will record the file however it was first written, but can permit case insenstive searching.
That way, when working in the BSD shell, everything works the same way it does on any other POSIX shell, with case sensitivity being the norm, but when working in the Finder you can browse & search in a case insensitive way.
There's a pretty good case to be made that this is the right way for a filesystem to go -- it kind of adheres to the rule of thumb that systems should be generous in what they accept (broad definitions of acceptable input, etc) but strict in what they produce (narrow definitions of what will be returned).
DO NOT LEAVE IT IS NOT REAL
Its a just recently added feature!
See the -s option for newfs_hfs:
(from man newfs_hfs)
-s
Creates a case-sensitive HFS Plus filesystem. By default a case-insensitive filesystem is created. Case-sensitive HFS Plus file systems require a Mac OS X version of 10.3 (Darwin 7.0) or later
"I'm a Genius!"*
*Not an actual Genius
That's gonna mess up my UT 2003 ranking! I work hard for those frags! Every one of them!
There exists no way of exchanging information without making judgments. --Bene Gesserit Axiom
> I'm still confused as to why metadata isn't being taken seriously by the
> rest of the computing world.
The rest of the world does take metadata seriously; we just don't store it
the same way as the Mac does. What do you suppose email headers are, if not
metadata? The thing is, different kinds of data have different kinds of
associated metadata, so it's not a foregone conclusion that storing the
metadata at the filesystem level is necessarily the only good way to do
things. Making it a part of the file format in many cases makes a large
amount of sense. OpenOffice files, for example, contain very significant
amounts of metadata -- not just formatting, but things like the author of
the document, when it was last edited, and so forth.
However, if you want to see a filesystem that goes hog-wild with storing
metadata at the filesystem level, you should try out the BeOS, where (for
example) your address book is just a folder containing a bunch of zero-byte
files.
Cut that out, or I will ship you to Norilsk in a box.
> The only way this feature can do that is if you're writing small
> files continuously. That's very strange software behavior
Err, what? Writing small files is more than 99% of the writing that a typical
filesystem does, and many applications cause them to be written fairly well
continuously. Just three examples off the top of my head of applications that
normally result in this behavior include email, usenet, web browsing (disk
cache). Two of these are *extremely* common desktop end-user applications.
Any filesystem that has problems continouously writing small files is junk.
Cut that out, or I will ship you to Norilsk in a box.
Windows XP has a similar feature that waits a until the computer is not in use for a certain amount of time. It would make sense that Apple would give users the same option.
I think I think, therefore I think I am.
Try this...
...and this...
The source code is posted to that thread; the only conditions are (1) 3 minutes after the system time starts (i.e. avoid doing so when booting up), (2) less than 20 MB of size, (3) file isn't already opened.
The only negative consequence is a possible speed hit, though. There's no danger.
I'm pretty impressed by this. Sure, it's been done before. Sure, there are more elaborate methods. But this is just a simple little lump of code that'll defragment the worst files most of the time.
this should defrag all of the 20M or less files on your hard drive.
it locates every file, opens it and reads every bite then closes it.
This should force the defragger to run on all files under 20M. Not that technically the defragger only activates when the file is broken into more that 8 extent regions. So this does not actually defrag everything.
but its also possible that having the file broken into less extents is harmless. first because the the first 8 extents are the fasttes to access in HFS+ and second its theoretically possible that on a multi-headed disk drive that having the file slightly fragmented might be good. Larger numbers of frags than read head would be bad of course
Some drink at the fountain of knowledge. Others just gargle.
Notice that the defragger only activates if the file is fragmented more than 8 ways. thus this mutli-read head issue is already accounted for. if you fragment the file into more extents than you have read heads this would presumably be bad. but perhaps 8 is the optimal number or close to it? its also faster because of the way hfs stores the file table that beyond 8 the lookup time increases.
Some drink at the fountain of knowledge. Others just gargle.
POSIX compliance
Doesn't being case insensitive violate POSIX? Or has that been fixed?
yes
Don't call me back. Give me a call back. Bye. So yeah. But bye our, well, but alright we are on a shirt this chill.
I'm Not sure the windows approach is really better. Notice that the apple approach is more minimalist in moving files.
Some drink at the fountain of knowledge. Others just gargle.
You are not alone. This is not normal. None of this is normal.
NOOOOOooooooooo....
>80 column hard wrapped e-mail is not a sign of intelligent
>life
Nope:
It's always case-insensitive (unless you've constructed a case-sensitive HFS+) - there's no BSD shell vs. Finder difference.
Because the rest of the computing world is more interested in successfully interacting with itself and has realised that filesystem metadata is practically impossible to successfully move between systems using "common" tools.
Apple has finally figured this out, too, which is why they're moving away from it.
Filesystem metadata is another one of those cool ideas that is (or, rather, was) kneecapped because of lowest-common-denominator restrictions.
I stand corrected, and I should have thought of this.
This is exactly the problem when installing the LWP library for Perl -- it offers to install /usr/bin/HEAD for you, as a tool for doing http-head requests on web servers. On most POSIX filesystems this isn't a big deal, but on HFS+ it ends up clobbering /usr/bin/head, the standard tool for retrieving the opening lines from a file or data stream (a/k/a file). The LWP maintainers don't see this as a bug on their end, because they get the behavior they want on every other platform -- hence three years after OSX came out and it's still an issue for some people. (Maybe Apple should just add LWP to their default Perl and the problem would go away for newbies...).
Thanks for the correction. :-)
DO NOT LEAVE IT IS NOT REAL
Interoperability.
shouldn't the opening lines util be called /bin/head, not /usr/bin/head? in which case the worst this does is render HEAD inaccessible
We've secretly replaced Slashdot with new Folgers Crystals - let's see if it notices.
The biggest problem with using UFS is poor compatibility with Classic and multi-forked classic files. Specifically, there is NO UFS support in Classic.
You can use UFS, but you better not run anything but pure OS X.
I'm not feeling witty so bite me
I hate when I can't get head
IMHO because its sort of a 1/2 and 1/2 solution. With a mainframe you get a full database filesystem: "datafiles" don't exist in the Unix sense and data is owned by applications. The effect is very powerful as you intermediate forms of data don't exist and applications can engage in very complex behaviors towards data.
Conversely on Unix you get the "everything is a file" and "a file is a stream of ASCII text" which is really powerful for shell programming but makes end user cross application interaction complicated.
Metadata is sort of 1/2 way in between. Datafiles have other associated data which sort of connects them to applications which may or may not be intererpreted the same way by other applications which may or may not respect the metadata and you can't search or organize based on this metadata.
By the logic you seem to be applying, every time a file is accessed you "risk" corrupting it.
It does not matter if you double check what you wrote, because that only decreases the chance of making an error. it does not eliminate it. you might make the same read error twice in a row.(e.g. to make this plausible imagine say a weak magnetization that flips after a temperature change later that night). or perhaps you may have read the file wrong in the first place. or something might go wrong when you are updating the file allocation table or there might be some bug in the code that makes an error.
Please explain why these events are more likely to occur to a file's new location than the old one.
the point is that if you just left the file alone inthe first place its safer.
Why ? It's equally as vulnerable to all the same one-in-a-billion events.
thus the apple approach of only defragging a file in active use and leaving all the other's alone may be preferred to a blanket de-fraggmenting of a disk.
So, assuming you're correct, the Apple method is preferred because it (minutely) increases the chances of corrupting the files you access, as opposed to some random file ?
Tell me, which file do you think you're more likely to care about - a random file X from the set of all files on the drive or a random file Y from the set of files you deliberately access ?
furthermore, when was the last time you tried de-fraggmenting a 500GB disk? do you have any clue how many days this would take??? The apple approach of doing it just-in-time makes a lot of sense. It follows the same logic as jounaling, which is in part a response to no having to fsck a 500GB disk.
After many years of experimenting with defragging and not defragging, I've come to the conclusion that it makes bugger all difference whether a disk is fragmented or not. If I can't tell the difference, it's not worth doing.
"I've come to the conclusion that...it's not worth doing."
Good answer. Defragging a 500GB disk really isn't worth it, you got that much.
But if the OS is going to do it in the background with practically zero overhead, even without any other benefits I say go for it!
Moof.
Did you just say what I think you said?
You are wondering why an operating system in text mode is faster than the same operating system when it has to render a graphical user interface?
Bleh. This is probably a troll because you misspelled Darwin twice. Or you might just be a bit slow, which would be in line with your comment.
Moof.
It's hardly the 500G aspect. Defragging the 20G hard disk in my laptop shows no discernable benefit, even when going from "99% fragmentation" to "1% fragmentation".
But if the OS is going to do it in the background with practically zero overhead, even without any other benefits I say go for it!
Except it will - by definition - have overhead. The more files a user is manipulating, the more overhead it will have.
The summary appears to not be quite right.
To clarify, there are 2 separate file optimizations going on here.
The first is automatic file defragmentation. When a file is opened, if it is highly fragmented (8+ fragments) and under 20MB in size, it is defragmented. This works by just moving the file to a new, arbitrary, location. This only happens on Journaled HFS+ volumes.
The second is the "Adaptive Hot File Clustering". Over a period of days, the OS keeps track of files that are read frequently - these are files under 10MB, and which are never written to. At the end of each tracking cycle, the "hottest" files (the files that have been read the most times) are moved to a "hotband" on the disk - this is a part of the disk which is particularly fast given the physical disk characteristics (currently sized at 5MB per GB). "Cold" files are evicted to make room. As a side effect of being moved into the hotband, files are defragmented. Currently, AHFC only works on the boot volume, and only for Journaled HFS+ volumes over 10GB.
I'm a bit less sure of this one, but I think the file must be severely fragmented (i.e. into 8 pieces).
Just three examples off the top of my head of applications that normally result in this behavior include email, usenet, web browsing (disk cache).
Perhaps for a server, but not for workstation. If I'm browsing the web, it's going to be at least 10 seconds between cache writes, as the page loads and I read it. Email or usenet would be similar - a bulk write every five minutes or so, maybe an occasional write on message send. That's not 'continuously' for a filesystem or a hard drive - that's light duty in so far as affecting its thermal properties.
If we're talking about a high-volume mail or news server, I can see your point, but we expect server disks to run hot anyhow.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Having metadata in the file system doesn't mean you can't interact with other platforms. It does mean that you lose the metadata when you move a file from one place to another - however:
- The icon used to represent a file does not have to move from one computer to another
- The position of that icon within the window that shows it does not have to move from one computer to another
- The application used to open the file does not have to move from one computer to another
- The file type does need to move from one computer to another - but that is the only metadata that currently is transfered from one machine to another, albeit regularly via file extentions
Metadata is metadata. It's not the data, so it rarely needs to actually move from one system to another. For the specific cases where it does, we usually have mechanisms for transfering that data. There's no reason to forget about the other cases, and the fact we continue to do so means that computers continue to be less forgiving, less intelligent, and less user friendly than they could be.You are not alone. This is not normal. None of this is normal.
Metadata is information about a file. Most file systems include a limited about of it - actually, most restrict it to the filename and maybe some very limited type information (Unix, for instance, stores whether it's a file or directory or pipe, etc.) We often overload the filename with type information to get around the lack of an actual file-type field stored with files.
Metadata isn't a half-and-half solution. You add it to a filesystem and you have something you didn't have to begin with. It adds capabilities that didn't exist and doesn't remove a thing. Microsoft, however, doesn't do metadata, and unfortunately most of the people working on the big Desktop replacement projects for Unix/Linux have got it into their heads that whatever MS does is the way to do it.
Even the frickin' Amiga had metadata and had a solution for storing it on a file system that doesn't support such things. It was ugly (you had a ".info" file for every ordinary file) but it worked. It's a shame that the lack of competition and alternatives has meant that people have forgotten what's necessary.
You are not alone. This is not normal. None of this is normal.
You'd probably have to take that up with Apple, but I suspect that the tool is seen as a userland one rather than a system level one, hence /usr/bin instead of just /bin. I've just checked on two Solaris machines (`uname -r` one gives 5.7, the other gives 5.8) and one Linux machine (RedHat 6.2), and all three of them have their head utility at /usr/bin/head -- so it's not just OSX that puts head in with the userland toolkit in /usr/bin.
I think the fix you're grasping for is to put LWP's HEAD in something like /usr/local/bin or /opt/bin with the rest of the manually installed programs that are not part of the default distribution. Aside from the $PATH issue that you hint at, this should be a portable, non-destructive solution to the problem.
The only complication I can think of is that the local/manual bin/ dir tends to be in different places on different systems -- OSX doesn't ship with an /opt tree, for example, but most Solaris systems do. The Fink project for Unix software on OSX likes to use /sw, but this is non-standard, and manually putting stuff in there could interfere with packages managed by Fink itself. I think that /usr/local is nearly universal across distributions & versions & SysV vs. BSD lineages, but I've heard some people complain about it, so I'm sure that some systems at least won't even have that.
Writing LWP in such a way that it can sniff out & use the correct local installation directory may seem to be more trouble than it's worth to correct a problem that, to date, is only apparent on systems running on top of the HFS+ filesystem. There's a case to be made for "fixing" LWP to be more portable, but the counter-argument to that -- which has been winning since 1999 or so, when people first started trying to install LWP on their OSX Public Beta machines in fairly large numbers -- has been "you Mac guys are the only ones complaining, maybe the problem is on your end." And so we end up with a stalemate.
*shrug*
DO NOT LEAVE IT IS NOT REAL
In large part that's because you aren't using most of those files. Probably 99% of your files can be fragmented (even heavily) and it won't make a difference because they aren't used (often or at all).
"Make the common case fast",
That's what this is all about. Defragment only the files that you actually _use_.
Of course, since there are performance requirements to doing this on the fly, it still has limitations. (i.e. nothing over 20MB, nothing with 8 fragments).
The hotfile clustering has the same idea - put only the files read the most into the hotband. Same sort of limitations apply (1.2MB, read-only files)
>I'm still confused as to why metadata isn't being taken seriously by the rest of the computing world.
It isn't being taken seriously by much of the UNIX world.
Microsoft is actively heading in that direction WinFS is essentially an integrated database of arbitrary indexed attributes related to file data. Even NTFS supports arbitrary numbers of forks in a file, which is a step up from HFS/+ 's 2 fork system.
No need to get defensive. I was just trying to suggest that Apple would have a setting that would wait until the computer was not being used. Apparently you feel threatened by the mere mention of Windows. If you ask me the whole PC/Mac debate is going nowhere. It's just a juvenile pissing contest.
I think I think, therefore I think I am.
I think you're musunderstanding something.
A FileID is a persistent handle that refers to a file. It is only valid for a single disk volume. If a file is moved to a different volume, the ID changes.
In every respect that matters, it's functionally equivalent to an inode number.
The real question here is why nobody has made a UNIX API to open a file using a volume-ID of some kind (which may have to be defined) and the inode number. Using an API like this, UNIX apps could get the same functionality as Mac apps. Moving a filename around the directory tree would not affect a program's ability to open it.
Creating a hard link to the inode is unnecessary for an API like this. After all, the standard open() call already does all the requred work. It already has to convert a pathname into a volume/inode index as a part of its normal behavior.
There are some Unixes that let you open a file give it's i-number (er, and device number, or a path to the devices FS, I forget which). There are also some Unix-ish OSes (Plan9!) that let you get a part for any open file.
If you had those two primitaves would not FileIDs and i-numbers be the same? Or is there more to a FileID?
Maybe. It is also why if I take CoolApp and rename it to CoolApp.old and install the new CoolApp in the same place as the old one all manner of stuff still runs CoolApp.old! I assume it will also sometimes be unable to find a file if I end up moving someting ontop of the old one (unless things that keep FileIDs keep both the FileID and a path to the file in case the FileID goes stale...which may be the case because I don't recall this happining, unlike the CoolApp.old bit)
Microsoft's file systems do as much metadata as UNIX does. On FAT (and I think FAT32) every file a timestamp and a set of attribute bits (directory, read-only, hidden, system and archive). On NTFS, each file gains two more timestamps (so you have create, modify and access) and security attributs.
Of course, neither Windows nor UNIX supports all the metadata that MacOS does - file type, file creator, icon position, user comments, memory partition sizes (for classic apps), etc.
HFS+ actually supports an arbitrary number of forks, even though there are only standard definitions for the resource and data forks.
Not too long ago, it was common to find anti-Mac zealots routinely criticize Apple for having a resource fork. They had a wide variety of disparaging remarks to make against the concept. I wonder if these same people will maintain that attibude when Microsoft starts providing the same thing. Somehow, I don't think they will.
Truth be told, I don't even think HFS+ does enough metadata, but it does some above datestamps and filenames, and that's a big improvement over the others.
BeFS would be the way to go.
You are not alone. This is not normal. None of this is normal.
I don't think so. The source code files are generaly written in their entirety by the editor even if you only make changes starting halfway though a file (or add a few functions on the end). So they will normally only occupy one extent (i.e. be unfragmented), and are stunningly unlikely to have mroe then 7 extents (needed for auto-defrag to kick in). The object files are again only written in their entierty. The .s files (if written) are the same.
I think there is seaking around when you write the executable out, but only to fill in the jump addresses, and they are overwrites not appends. Unlikely to be much fragmentation there.
The systme include files and lib files ought not be fragmented, but if they this would fix that the first time you use each such file, and since they are all unfragmented next time through you won't have a problem there (it might even eventually save you some time).
I don't think a developer will run into this much (if at all!) during compiling.
"Except it will - by definition - have overhead. The more files a user is manipulating, the more overhead it will have."
*sigh* We've already talked about this. Since it only does it with files that really need it (over 8 fragments), it doesn't need to do it every time a file is opened. The speed is saved when the file is defragged and therefore nothing needs to happen. But like I said, for your understanding I was ignoring all the other benefits. You've simply gone and turned one of the benefits into something bad.
My first point was, you think defragging is pointless. This new system doesn't defrag the entire drive, just the files you use, and it does it with practically no loss, so why not?
Moof.
YEEEEEeeeeeeess....
You have the option of formatting your drive as case sensitive, if you really want to.
Personally, I'd recommend against it, unless you have a really good reason to choose otherwise. Most Mac apps assume case insensitivity. For instance Dantz has this knowledgebase article regarding Panther and Retrospect. Note the paragraph which reads:
Interestingly, they say that this feature is only available on the server edition. Can anyone confirm or disprove this?
Alright, there ARE multiple heads, but there is still only 1 head to each platter-surface, and all the heads on all the platters are connected at the base, so they cannot move independently. You ARE better off defragged than not, but even better would be to 'arrange' files based on access paterns.
I tell you this, I notice quite a speedup after I perform a full-backup-and-restore, the only way to defrag most Linux filesystems. I saw loading OpenOffice take 7 seconds before, and 3 after.
When I load Mozilla it has to read in a LOT of other files, a proper filesystem would log that and try to move those files closer to each other. ReiserFS is trying to implement something like this, I believe, they are marking the 'temperature' of inodes based on access patterns.
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
Dear ColMustard,
I was wondering why my Ford Escort doesn't get good pickup when I've got it packed with bricks. It goes fine when it's just me and the seats, but once I pile those bricks in it's so laggy! Do you think I should put brighter headlights in, or should I change the blinker fluid?
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
Actually, if you read some of the articles about Microsoft's preview of Longhorn you'll see that they're developing a new filesystem that not only incorporates metadata but takes it to the level of a relational database. Expect to see many articles on how metadata is the best innovation to come from Microsoft since Windows 95.
-- thinkyhead software and media
It is and has been taken seriously - it's called "the registry" by our pals up in Redmond.
"You might as well get your son a ticket to hell as give him a five string banjo." -unknown minister
I wonder how that interacts with the "secure" delete. Does it seeks out previous copies of the file and securely delete them too? That would be quite a feat.
(Also, has anyone confirmed that the code snippet is actually executed?)
My experience with HFS+ is that it suffers very badly from directory fragmentation. DiskWarrior should be supplied as standard with all macs; on 10.2, directory fragmentation can become a problem in months on a well-used system, and it reduces the reliability of HFS+ to the point that it can cause kernel panics. (Which are random, and make you think you have a RAM or a processor problem). Plus it makes filesystem performance really, really suck - you start to find that some operations can be hundreds of times slower than they should be.
/* IMPLEMENT ME */ comment in the source somewhere near there).
Apple's fsck is useless: it can detect these sort of problems, but can't fix them. Norton Utilities can't either, sometimes (orphan files seem to fox it, in particular). Can't remember the specific message that fsck gives, but you'll know it if you see it (it implies a problem was fixed that wasn't. I suspect there's a
DiskWarrior reported that the disk was about 30% out of order; running it found about 20 files that nothing else could, solved the kernel panic problem, and made a huge difference to speed (things that would take minutes suddenly were instantaneous). It shouldn't have been necessary, though.
You dont' move a file to another volume, you copy a file to another volume. Sheesh
And it is NOT an inode! And before you even say it, it isn't a file descriptor either.
>80 column hard wrapped e-mail is not a sign of intelligent
>life
I understand what metadata is. The purpose of metadata is to attach additional information to application specific data. A database filesystem plus datafiles being application specific goes much further in that direction. That's what I meant by 1/2 and 1/2. You are getting 1/2 of what the mainframe package gets you.
A good example is a versioning system (version 1, version 2...) gets you 1/2 of what CVS gets you (versioning and branching).
> If I'm browsing the web, it's going to be at least 10 seconds between
> cache writes, as the page loads and I read it.
Surf with images disabled, do you?
> Email or usenet would be similar - a bulk write every five minutes or so
You apparently get a lot less email than I do. When I check my mail, it
writes files every second for several minutes.
Cut that out, or I will ship you to Norilsk in a box.
> I'm talking about FILE metadata.
Yes, but my point is, metadata is metadata. The difference between storing it
as part of the file and storing it as part of a thingydoo attached to the file
is a difference of paradigm, not of substance. The BeOS would store info about
when a file was last printed in the filesystem metadata for the file; OO.o
would store it in the metadata XML, not in the content portion of the file,
but it would be part of the file as far as the filesystem is concerned. To
the user, the approaches are equivalent (except that one is independent of the
filesystem and therefore makes sense for a cross-platform application; the
other is so highly platform-specific that no cross-platform app can use it,
which makes it effectively redundant except for single-platform apps, which
are hopefully a dying breed at this point).
There are some things it makes sense to store as metadata at the filesystem
level. Mainly, things that apply to all files on the system, not just certain
types of files. So, content-type, for example. (The BeOS stores the MIME
content-type as filesystem-level metadata; DOS and Windows store a file
extension that is essentially equivalent, albeit somewhat inferior. The Mac
has its type and creator codes, which also are essentially equivalent albeit
somewhat inferior.) Permissions are another example. (FAT filesystems lack
here, but the other major ones are pretty decent. Especially with the
introduction of ACLs.)
But things that only make sense for certain types of files (e.g., icons,
which only make sense for applications, generally, or From headers, which
only makes sense for internet messages, or phone numbers, which only make
sense for address book entries) can be stored in a manner specific to that
type of file, without loss of generality (since there was no generality in
the first place).
Cut that out, or I will ship you to Norilsk in a box.
Code is Speech. No to Censorship.
Thanks. I haven't yet upgraded to Panther, so my information is all second-hand, and some of what I've read has turned out to be wrong.
But now Apple has depricated Resource Forks in favor of bundles. Bundles are essentially just folders and each "fork" is a separate file in the folder. That way it it can reside on pretty much any file system.
See for more info.
I use to be indecisive, but now I'm not so sure.
This also explains why the hard drive on my iBook seems alot hotter since upgrading.
The only way this feature can do that is if you're writing small files continuously. That's very strange software behavior, and perhaps a worst case for this optimizer. Why would you be doing that?
Hmm. OSX itself writes lots of small files continuously in general, in general, don't most *nix's?
OSX itself writes lots of small files continuously in general, in general, don't most *nix's?
What makes you think that? Have you ever had a powerbook put its disk to sleep? The OS is still running (and not writing lots of small files continuously).
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
What makes you think that? Have you ever had a powerbook put its disk to sleep?
Well, I meant when you're actually DOING something. You can put your powerbook to sleep, but can't actually do a whole lot without it spinning up. IE, having users login... logs don't write themselves (/var/log), paging in/out memory... *nix's like fast disks.