Slashdot Mirror
The Mac, Metadata, and the World
Rick Zeman writes: "ArsTechnica has posted yet another compelling article, this time on metadata, its history and the future of metadata storage as seemingly indicated by Apple in OS X. Extensions==Bad!"
← Back to Stories (view on slashdot.org)
And the glaring question was: why is Linux blindly following Windows? Linux's file type handling is still in a somewhat early stage, it wouldn't be inconceivable for the paradigm to change.
Very interesting. I never really thought about metadata before, but it brings up a lot of points about the mistake of using file extensions.
File extensions do serve a convenient purpose with a command line, as you can manipulate them easier without using multiple tools. However, if the metadata was stored outside the filename, we could have (and had) UNIX, GNU, BSD, and DOS/Windows utilities to manage them in the past. If all systems were designed to keep track of the metadata, it would have been a better world.
It is unfortunate that the technical lowest common denominator (DOS and DOS-based OSes) dictate so much of our system. While Windows NT based systems (including Win2K, WinXP, etc.) have made tremendous strides, there is a constant need to maintain compatibility that holds us back.
I think that it makes sense for Apple to adopt the file extensions, as unpleasant as they are, to support a networked world. The author's suggestion of adding them on transport makes sense, but definitely leaves something to be desired. It would be confusing to transfer Word documents around and have the extensions pop on and off depending on the environment. If the Mac leaves them alone, it still leaves something to be desired because the file name changed when it left the Mac it was created on for the file server, and when it comes back it has a different name.
It's a shame that a standard for storing the metadata wasn't created long ago. While the PCs wouldn't use the data, it is a shame to lose it. It is also a shame that we have to work towards the lowest common denominator. It's one thing to support it, it is another to adopt the conventions.
Alex
Someone didn't read the article. Apple stored the file type and creator data in a dedicated metadata storage area (analogous to an inode), *not* in the resource fork. Siracusa stressed this point *several* times.
Ben "You have your mind on computers, it seems."
I think the reason Apple went to a dual file type system (extensions and metadata) is because it's too hard to implement the necessary level of interoperability otherwise. Suppose you want to keep the usual Classic MacOS method of just having file type/creator code metadata. You merrily store your files on your hard drive, with no extensions in sight.
Now you share your drive on a heterogenous network. A Win98 box connects, and looks at your files via NFS. Does the MacOS X-side NFS server automatically translate the filenames and add extensions?
Then another MacOS X user uses ssh to connect to your box. He types 'ls'. Does he see "virtual" extensions or not? What if it's a Windows user telnetting in? How would your box even know what OS the remote user was coming from?
It's just too easy to run into inconsistencies if you stick with the system of mapping file name extensions. Yes, extensions annoy me, especially since older versions of MacOS are stuck with a 31 character limit (yes, it's 31, not 32--Ars is wrong), and I have to keep file names short to be backwards compatible. Unfortunately, it's just another bad MS decision we have to live with.
http://www.beosbible.com/exc_filetype.html
and here:
http://www.beosbible.com/exc_query.html
The BeOS has solved the problem, years ago. The BFS has integrated all these features into the OS itself, so all applications are making use of them. The Byte.com BeOS articles from Scot Hacker are also a must read!
Unix pipes. How else are you going to get file type metadata if it isn't in-band. That is what the magic number is all about. Pipes, stdin, stdout, etc.
I think this is purely an application level problem and and not a filesystem problem.
It still matters in the gui world too. If we ever develop GUI drag and drop style graphics filters and such, say a webcam output into a filter into something else, that info is still in-band.
How would you represent the file type of a named pipe, or a socket?
1. funny.txt.vbs emails where
2. Hidden extensions allow Finder.app and Finder.whatever to appear as "Finder" in the same Folder....
He then goes on to say why Apple would reccomend developers use extensions (which is redundant)...A networked world demands MacOS be a better "citizen". He claims extensions are unneccisary since email apps can append extensions to files when sent...Not to mention his speculation that Apple would drop it's current model from a Windows model...
Problem with his analysis. E-mail isn't the only way to share files in OS X. Currently OS X offers FTP, HTTP, Appletalk, NFS, SSH and X.1 will add CIFS. Appletalk handles Meta information transparently, going from Mac to Mac, no need for extensions. FTP, HTTP, SSH and NFS (NFS will almost always go to a flat filesystem) offer no way to store/send OS X style meta information. Yes OS X treats a NFS drive (and CIFS drives if you use Sharity) as a UFS drive and stores Meta data properly so that the Mac can use the file, but the remote computer has NO idea what kinda of file that is, unless it has an extension. So a Mac user who casually copies a extension-less Word document to a PC zip disk, when they put that disk in a PC, it's useless (unless the user knows of the problem). So it is clear file extensions are needed for a networked enviroment....
But Mac users don't like extensions so Apple will let us hide them.... which creates the problems he described (funny.txt.vbs and duplication file names in the same folder). The first is really a non problem since the Mail application in OS X doesn't hide file extensions even if it's named funny.txt.app and double clicking the in Mail does NOT launch the file. This potential problem can be further alleviated but noting what kind of file it is below it in Mail. The second issue of duplicate file names can be solved easily too...don't allow it. In other words DumbName.jpg and DumbName.txt should not be allowed in the same folder. Then hide all the file extensions and the users would be none the wiser.
Burn Hollywood Burn
The UNIX file-system is brilliant compared to DOS, but ONLY compared to DOS. It is still designed for command-line users convenience. I am NOT criticizing the command line, I use it daily under OpenBSD, Linux, WinNT4, Win2K, and Mac OS X. It is nice to have the control of a CLI, as well as the ability to run scripts.
HOWEVER, the system of making things conveniently obvious for the CLI results in engineering decisions that give the OS less flexibilities. GUIs can provide TREMENDOUS ammounts of information BECAUSE the user decides when to get that information.
For example, the filename and type need easy access for the user. For a GUI user, they need the filename and the type deciding the application binding. For a CLI user, including the type with the filename makes it easier to manipulate.
While you could setup ls (or dir) with many flags to pick and choose the information, you create a minor mess. Additionally, things like changing the type to a list from a database is one thing for a GUI with a dropdown box, it's a nightmare to implement in a CLI. If you designed for the CLI, you made a tradeoff.
Additionally, UNIX was developed in a hardware environment more restricted than the DOS world. Early machines used in development are nothing compared to modern machines.
Take the NTFS file system. If you are on an NT4 machine, or a Win2K machine, (running NTFS of course, not braindead FAT/FAT32) you see filenames as normal. Inside the properties, there are MANY more options. Do it on a Win2K machine, and you see more information than on an NT4 machine if you look closely.
The UNIX approach is old and dated. Microsoft has moved on, it's important for the UNIX community to do so as well. ACLs (implemented on NT) are FAR more flexible than users/groups. Private user groups are an ugly hack to handle the user/group system. The whole UNIX model needs to be modernized. There are ACL UNIX systems, but they aren't the mainstream.
I love the power of UNIX-based server, they give me tremendous capabilities. A proper CLI is awesome. But let's not kid ourselves. Beating Win95/Win98/WinME at ANYTHING was never impressive, they were ugly hacks onto DOS that has its roots in the 8086 processor. Everytime people toute the advantages of Linux, they compare it to Win9x. Beating a legacy desktop OS in terms of uptime, etc., is NOT impressive. Compared to Win2K, Linux's technical advantages are pretty minor. There are some, but not many. Compared to the BSDs or commercial UNIXes... well, Linux doesn't look that impressive. It has advantages and drawbacks, different engineering decisions.
The problem with UNIX is an LCD (lowest common denominator) and designed by committee problem. Having a common API that programmers can target is tremendous, it helps with portability. However, failing to keep moving that API foward is a mistake.
As it stands there are many applications that only work on one variant. Extending the UNIX common API once or twice a year to encompass vendor extensions would be a tremendous boost, and allow UNIX to escape this trap. If Sun has a great idea and incorporates it into Solaris, their ISVs should take advantage of it. The rest of the UNIX world should have it within a year (or two at most) so ISVs can port to other UNIXes. As it stands, you either write to an old standard OR to a particular UNIX. Neither is a good choice.
Alex
In a lot of ways this is a pretty good article, but there are a surprising number of instances when the author seems to have bent himself into thinking about a limited number of filesystems. Barring anything academic, experimental, or "fancy," it's pretty clear he's never tried to think about UNIX linked-list style filesystems: within the framework of his discussion, I would assert that a file's name is not part of its essential metadata in a UNIX-style FS. Why? All of the file information is contained in the file's inode and data blocks (the immediate decomposition into metadata and data being obvious). A name for the file is just an entry in a "directory," which is just another file. A given file might be listed in hundreds of different directories, nevertheless, there's still only one file. It might have a different name in each directory it's in; in which case it hardly makes sense to talk about "the filename," unless one is willing to assert that inode + data blocks don't constitute a file, and that each instance of a reference to a particular inode is to be considered a file.
/boot/vmlinuz > /dev/audio'. The results of such voluntary file polymorphism aren't always useful, but they sometimes are.
Furthermore, the examples of "immutable" metadata (ill-considered vocabulary in the first place, I think) are poorly considered. File size can be altered without altering the underlying data on BSD-style unices that provide truncation and extension system calls. Modification time often gets changed on many systems without any change to the underlying data: many, if not most, kernels will change mtime any time a file is opened for write or append even if no subsequent writes are done to the file. "File type" is essentially a nonsense notion on most UNIX filesystems (and DOS too, given the weak representation), a file's type being an interpretation an imaginary multipurpose file handler is to give the data. In such situations, "file type" is decided either by regexp matching of the file name (which can be anything, remember) and judicious use of magic (man magic if you don't get it). In many cases, this doesn't produce an unambiguous answer: 'file blah' produces 'blah: data' with amazing frequency. Arguably this just means that UNIX filesystems don't have an adequate mechanism to express the idea of "file type," but I would argue that regardless, the notion of file type is at least partially bogus. There's nothing to stop me from interpreting data many differnt ways: an XPM is something I can edit with an ordinary text editor, and hence a file of type "text," but it can also define pixmaps, so depending on what I want to do with it, it might be of at least two file types. Similarly, I can try to view a raw audio file as a compiled pixmap, or, to recapitulate the famous joke, 'cat
There are further aspects of the article which are either incorrect, or at least fail to reflect my personal experience, but for the most part it's simply repetition of previous errors. It seems abundantly clear to me that the author is a thoughtful and well-educated person whose primary computing experience has been with Macs and post-DOS MS machines: and while he may have used UNIX-like operating systems, he doesn't know much about data representation of filesystems on them, and clearly hasn't considered more modern developments like filesystems with journaling or ACLs instead of permission bits.
Perhaps my criticism is a little too sharp, I would like to emphasize that I liked much of the article and I laud the author for thinking about some important concepts in detail, but I feel the viewpoint adopted is one unnecessarily limited by the author's personal experience.
I liked this article, it was thought provoking. It reminded me of the Archimedes, with its 16 bit file types, and the Mac. Oh, the dear Mac. How many times did I scream at it, "yes you will bloody open that file!"... While it sits there all like, "No I bloody will not, it's the wrong type, I'm not even looking at it!"... .c and .h files? What would happen if you didn't have extensions?
Hang on a minute though. It's a bit much to have a go at Microsoft about file extensions. Unix? Written in C?
Anyway. Personally I get all excited by the idea of accessing files more as a database action. I know there are people that hate the idea.
Interestingly, NTFS allows you to hang arbitrary stuff off a file. It's also a good way to hide stuff, because almost no-one knows about it. Oh, well.
A lot of things would be better if the 'lower' OSes would just pay attention to MIME types. But there's one obvious situation where it falls apart.
Joe Mac User makes an HTML document referencing a bunch of JPEG and Flash images. The JPEG and Flash files don't have extensions in their names. He sends his HTML directory to his Windows-loving friend. Assuming that the Windows or Mac apps payed attention to the file types (either Just In Time on the Mac to add extensions, or the Windows app payed attention to MIME), the user's documents would have appropriate extensions added to them. The Windows user's HTML is busted.
While it royally bites that I have to put up with extensions in OS X, I can understand why Apple did this.
You non-tech-savvy computer user (I'd think that's 80% of computer users out there), are damn clueless, and would be completely unable to fix that HTML example.
Moderators should have to take a reading comprehension test.
Linux has traditionally not bothered very much with file type. The user generally knows what to do with the file, and does so. What look like extensions are actually just generically part of the filename; there are conventions for them, but they are no more strict than the conventions for filenames in general (Makefile is probably a makefile, README is plain text, foo.c is C source, etc.).
An important thing to realize is that file type, like, for instance, size, can be determined from looking at the data. In fact, many programs look at data files and determine the file format from the data; "file" does a pretty good job of detecting non-human-readable formats, even without knowing any information at all about the file type.
Where this all breaks down, of course, is when the user wants to omit the program name. On a Mac, you normally double-click on a data file to open it (and hope to get a program that does what you want). On *nix, you traditionally have to specify the program-- and much of the time, you select a different program depending on the desired result: for foo.c, I could use emacs, or gcc, or I might want gcc -M (get dependencies), or even wc (to see how big it is), not to mention less or grep or etags.
I think part of the Mac fascination with file type is due to the monolithic program structure; you find the file, and then you open a single program that does to it anything that you will ever do to it. In this model, there is a right program, and which program is right is based on file type. Windows clearly suffers greatly from having this model but not having a more reliable fashion of determining file type than Linux.
Incidentally, has anyone else noticed that the MacOS scheme is equivalent to having 4 character extensions which aren't displayed, with the corresponding problem of having malicious executables named README.txt (or even README)?
I'm afraid you misunderstood his definition of immutable. In this example, you changed the data, and what was originally a plain text file became an HTML file. His definition of immutable was that if the file data changed, then its type did not change.
Also, it didn't mean that the metadata need be unchangable, since it could be changed to reflect greater precision, or if it was wrong in the first place. For example, an html file is a text file (but more). So it is entirely reasonable to change the type from text to html (provided it actually is).
slashdot.pl and slashdot.txt should NOT collide on my desktop...
I agree that slashdot.pl and slashdot.txt should not collide, but that is just because they are part of the name. They should also not be required to be a given type.
How hard is it to change a bash script to a different shell? Change the first line.
I agree that metadata should be readily accessible. The only reason it is tough on a mac is because it was intended to be difficult, so that new users would have trouble shooting themselves in the foot.
How would you like it if you had to name all your executable perl scripts ending with .pl?
You don't, because the operating system specifies an (optional) header section to every executable file, which allows it to determine which program to run the file with. This is metadata, of the magic number variety. It is data added to the beginning of the file, for the sole purpose of determining its type (ok, in this case it also specifies the path to the perl executable and any flags to be passed it, but ignore that for a moment).
The reason we have such magic numbers (which are also in most other standard file types, ps, gif, jpeg, etc) is because there are no common operating systems which support file types, so applications are on their own, and are forced to include what is properly (in my opinion) metadata in the file data itself. As long as we are going to store this data, why not have it in a standard location where it can be used by the rest of the operating system?
Actually, I lied. I don't hate MacOS; I just wanted to get your attention by yelling about it. Now that you're here, though, I have to say that I LOVE the MacOS, and have ever since I first used it, before it was even called MacOS. I started with System 7, which was so attractive and easy to use that it's still my bar for measuring other interfaces.
.wav opens in a player, while another opens in an editor or burner. Well, I think the solution offered by Windows and by some *nix environments is better, easier, simpler, more elegant. A simple context menu, brought up by right or center-clicking, provides any options you could want. That way to open something in my viewer application, I just double-click--I know on my Windoze box that all image files (except .psd) will automatically open in my viewer, ACDSee (which recently became available for Mac, too)--no surprises, no metadata editors needed. If I want to edit it, I just right-click and choose the command "Edit" from the menu, which is set to open images with Photoshop. Same with .wav and other such--double-clicking opens in WinAMP, right-clicking and choosing "edit" opens in SoundForge. You can create any action, and choose any app to be associated with that action, for each file type--and then a list of all the possible actions for that file type will be displayed when you right-click a given file. But it will open in whatever your set to be your standard viewer, by default, if double-clicked. Much better than relying on hidden metadata. But even better and simpler than having to set up the actions and associations in the Folder Options dialog, is just using the Send To sub-menu that is brought up on right-click--just drop shortcuts to the apps you usually use into the Windows\SendTo folder, and those apps will appear on the Send To submenu when you right-click. That way I can easily open any file with any application, by using only one right-click and one left-click. In terms of launching files, it's like having the flexibility of a CLI, but within the ease-of-use of a GUI. That's one feature the Windows GUI actually got right, and got right very early on. MacOS can keep its metadata, but this is easier, simpler, better. I love the Send To submenu, though it's usually under-utilized by most people.
:-)
But if there is one thing I intensely dislike about MacOS, it's the metadata. I know I'm practically alone in the Mac camp, but I hate metadata. I have always thought it was just a space-hogging pain in my ass.
Now, the space issue is no longer a big concern since we have such big, cheap drives that a little filesystem metadata isn't such a burden on capacity. But back in the days of floppies I was pissed that I could fit so few files on a floppy when my friend with DOS could fit noticeably more. I was especially annoyed that even when I formatted a disk as a PC floppy, the Mac would still waste my space by creating and hiding from me files and folders on the disk to constiture the resource forks. I wanted every kilobyte, which counts when you're cramming a lot of small files onto a lot of small disks.
But of course this is no longer the big issue it used to be. But if I were storing large numbers of files and running out of space on a Mac, I'd still silently curse all that metadata wasting my capacity.
The part that still bothers me, now that capacity is no longer a substantial issue, is that in Windows or *nix I can instantly change file types from the interface, but not with Mac. It comes up a lot--many times a day. Click a filename, change three letters, and a text file is recognized as a script or batch file to be executed rather than opened. A click and three letters, and a file I just downloaded from USENET goes from text to UUencoded so that when I double-click it will be decoded for me. A click and three letters is all it takes to change a file's type and its application association from the GUI, without having to resort to some clunky special editor. And it's even better if I need to change the type/association of a great number of files--just open a CLI and type a quick line, and it's all done. What a pain it would be to have to use a metadata editor instead of just manipulating three letters in filenames. Simple file extensions put more power over the file within easy, simple, even automatable reach.
The advantage of metadata is something many Mac users, and theoretists like this article's author, seem to believe in, but I cannot see it. For instance, it's thought a great advantage that you can set a file to open with any application, despite the filetype. I hate downloading things on a Mac because of this. Some idjit will have a file set to open in an application I don't have, and the computer may be too stupid to know that I always open that file type in Application X. A dialog pops up on any reasonably modern MacOS to help, but it's still a big pain in the ass compared to having a PC automatically know what I open that file type with. Even more annoying is when I really do have the application the file is set to open with installed, but I always want that file type to open in a different app. This most often happens with graphics files--I do not under any circumstances want to have Photoshop or Graphic Converter open a graphics file, just because that's what it was created in. I have a simple image viewer for viewing images. If I want to edit them, *then* I open them in Photoshop. Same for Premiere and others--I do not want a big, slow editor to open my files just because that's what they were created with; we have smaller-footprint and more versatile file viewers for that.
The other part of it is that the "simplistic" (sometimes the most simple designs are the most elegant, while the more complex are just gaudy) file typing systems also solve the problem of opening certain files of a given type in one application but others of the same type in another application. Metadata proponents always point out how "great" it is to have one, for example, JPEG open in JPEGview or whatever, while another JPEG opens in Photoshop; one
I hate to say it, but the metadata folks are IMHO going the wrong way. I want more power and flexibility within my clicks, not less. I hate having to edit metadata when a simple three-letter change is all that would be needed in *nix and 'doze. And as I said, the advantages of metadata in terms of application/file association are entirely negated by the right-click menu and its Send To submenu in Windows, and similar functionality in some *nix GUIs. Metadata may have good uses, but none I can think of that can't be done more simply and elegantly. I also dislike the idea of my filesystem hiding things fom me, which unfortunately is exactly what MacOS does and what the newer NTFS in Win2k and up can do (I believe Ars had an article when Win2k came out about the new NTFS and some of the still-largely-unused metadata fields). Ext2 or FAT32 all the way, baby--and before you poo-poo FAT32, it may have almost no modern features, but it is straightforward, simple, and actually very fast in performance (thanks to the fact that it implements no real modern features); I recall it beating out NTFS in terms of raw speed in an old Ars article. Poor crash recovery is its main weakness.
I like to keep things as easy to manipulate as possible. And contrary to what many make the mistake of thinking, file extensions are not just easy for CLIs--as I said, it makes sense in a GUI too, since it can be directly manipulated from within the GUI's file browser, without having to open the file in a metadata editor. It also makes the type of file crystal-clear--especially important if you don't want to accidentally run an executable that has an icon to make it look like a file. Unless OS X has some way which I haven't noticed to visually set executables apart from other file types, even when they're on the desktop or somewhere else that doesn't show details, I can't wait for someone to create lots of OS X viruses that have common file icons. That's already a case in the Windows world, where you'll find files called Report.doc.exe that have Word icons, but if you notice the trailing extension you won't mistakenly execute them (though the "show extensions for all file types" option isn't the Windows default anyomore, alas). How can you tell by a glance in OS X, or any other place where metadata rules instead of file extensions?
Oh well. Windows may not have a lot right--but it does have its use of simple file extensions and simple context menus right. I always hated editing resource forks. It's just another *unnecessary* layer getting between a man and his hardware. Tell me one very useful thing that can be done with filesystem metadata, that can't be done easier and put more in direct control of the user. And before you say "labeling," like MacOS prior to X used to have--that's what folders/directories are for.
Chasing Amy
(We all chase Amy...)
"The more corrupt the state, the more numerous the laws"-Tacitus
if I understood the article, file types are bad because they get in the way of allowing the user to determine how to open and view files. The only real reason to want file types is closely related to application binding, IMHO - some users want *all* html files to open in Frontpage, others want to pick and choose on a per-file basis, most want something in between.
But then why even *have* file types? You can survive quite nicely without them if you do have application binding metadata. Whenever you use an app to create a file, that file shoudl be bound to that app. If you want to subsequently open that file in a differrent app, then you shoudl let the app try. It's up to the APP, ot the filesystem, if it can open it or not. Why shoudln't you be able to open a JPG in notepad? if notepad has a hex viewing capability, it shoudl open just fine.
a well-designed app shoudl let the user attempt to open any file. It shoudl try and interpret the data correctly, and it should allow the user to bind the file to the app if they so choose.
IMHO the whole notion of file types is a mistake - the Mac approach seems to be, incorporate type as metadata, the windows approach seems to be use an extension. But neither is really necessary.
as a final note - dumping file types avoids the "identical icon" problem that the author demonstrated in the screenshot. Simply use the icon for teh file that corresponds to the *binding* , not the file type.
Don't blame me - I voted for Howard Dean. http://dean2004.blogspot.com
From the article:
Any part of the Mac OS user experience that exactly duplicates the experience on another platform ceases to be a compelling reason to buy a Mac.
I totally disagree. I had absolutely no interest in Macs until OS X, and the reason I switched was because it acts just like a *nix. I can pull up bash, run emacs, grep, sed, awk, etc. Duplicating the unix experience was a very compelling reason for me to buy a Mac. Naturally, little things like Quicktime, games, and DVD support sweetened the deal. :)
As far as metadata is concerned, I think that Mr. Siracusa is right. The current unix way of handling metadata sucks. Unfortunately, the future does not lie with the old "Mac Way", which is arguably a good deal more elegant. Steve Jobs knows this, which is why his new OS is based on unix, despite its' occasional warts (like file extensions). Apple has done what it had to do to survive in this new world. I just hope that a lot of the old Mac partisans will stop trying to cling to the past and join us for the ride.
This
The author mentions that in CoreServices, two different Finders appear. /System/Library/CoreServices with terminal.app, I can see that one is simple called "Finder", the other is called "Finder.app". Changing my Finder view to "table" I can see that one is a "Application"; the other, a "Classic Application." So there are ways to differentiate the files-- though neither is quite elegant. The extensions are probably necessary for Nextstep compatibility.
.ps extension for Distiller is just plain rude), the immutability of the Creator/Type codes, save for ResEdit, is someaht inconvenient. I remember writing Applescript applications to change these codes en mass. Not exactly user friendly.
Checking my
In Windows 95 & and successors, the GUI hides the extensions, and as the author points out, this can cause serious problems with vbs viruses. But what was left unmentioned is that it also is hard on programmers. If you can't tell the difference at a glance between "myclass.h" and "myclass.cpp", it really cramps your coding style...
Microsoft also hides files that end in ".dll"-- which is a pain if you program libaries. This is somewhat more defensible, but not by much.
Truth be told, although certain aspects of the Type/Creator code were far more elegant than enaything Windows 9X ever developed (Note to Adobe-- grabbing the
These did the trick.
On HPFS, they were stored as part of the file in the filesystem. You could copy the file to a FAT formatted floppy, however, and the EA's were stored as a separate file, allowing you to keep all attributes, including the long file name.
Before /. went kablooey earlier today, someone pointed out that BeOS used MIME for identifying file types.
As for the MIME example you give above, it is as much the job of Windows to add (or ignore) the extension to a MIME'd file as it is for Apple to add the proper extension. In other words, I'd say it's the Windows machine's fault for not recognizing the file for what it was, just as much as it was the Apple's fault for not adding the extension. Interoperability requires both sides.
On Mac OS X it's a little different, though. The "Types Change" plugin isn't available (yet?). But the "Open Using" plugin isn't really necessary, since you can force-open any file by dragging to an app in the dock while holding down command and option. Hopefully there will be a way to change the type and creator of a file on X soon, and all will be back to normal.
Free Hans!
What I do have a problem with is the splintered way that MIME is done in practice. Suppose I want my file type "foo" to be associated with a certain mime type and opened with my fooviewer, I would have to register my application/x-foo in:
/usr/share/mimelnk/application/x-foo.kdelnk
/usr/share/applnk/Multimedia/fooviewer.kdelnk
/usr/share/mime-info/fooviewer.keys
/usr/share/mime-info/fooviewer.mime
/etc/mime.types
/etc/mailcap
/usr/local/lib/netscape/mime.types
I can't even figure out what the heck Mozilla uses for local MIME types... It apparently isn't any of these, in the version of Mozilla I have. I see it makes some nice XML files for user defined types, but those don't work with plugins.
Why can't we just standardize on using
I've had enough abrasive sigs. Kittens are cute and fuzzy.
1.) make sure apps hide file extensions, preferring, instead, icons .tar.bz2 to signify bzip2'd tar files, right? Get ready for 8bim.tif. (for anyone curious, 8bim is the creator code for Photoshop docs on a Mac.)
2.) Hell, UNIX people use extensions like
Really, there's the wonderful, superior data you get on a Mac when dealing with a Photoshop TIFF. "8bim" as the creator code (huh?) and the more sensible "tiff".
Sure, sounds great. *rolls eyes*
Sure, feel free to rip me a new one if I didn't use the proper terminology. I mess with ResEdit maybe once a year. :-P
Stating on Slashdot that I like cheese since 1997.
I agree kind of with this. But Mac OS X fixes this. In the file info there is a place for application. Where you can choose a application which opens only that file or opens all files of that type.
Ok, first, the linux system of actually lookin' in a file to see what file type it is seems pretty un-idiotic to me.
But more importantly, I strongly disagree with your point about ACLs. Different priviledge levels might be useful (as opposed to simple user-or-root), but I don't see a good reason to apply this to a filesystem. As it is, it's very easy to see quickly exactly who has what rights to what area -- with complicated ACLs, everything can get confusing and you might not notice a security problem. Sometimes simple is good.
The private groups notion is far from an "ugly hack" -- in fact, there's no "hack" involved at all: it's just *using* the group and umask functionality in a nice elegant way.
Hmm. What type is Courtney?
The point is that you like to see the type as a name next to the file. This is a question of interface, not of metadata. The SAME information for the extension could be shown there.
You could sort by Type instead of "By Type - a hack off the extension".
The point of this was that there is no advantage to storing it in a limited extension as opposed to meta data.
Even more important (that the article went into) is that you can change the extension without changing the file. Think about that, you've changed what the type of the file is without altering the data? That is the point of keeping it as Meta-data, the type should never change without changing the data inside it to another type.
Alex
- John argues that the OS can handle flattening files and creating file extensions when they are written to transports & filesystems that don't support the MacOS metadata properly.
-
John argues that the user should always have control over a file's naming and not the OS, yet acknowledges that renaming-with-extensons will often be required in a networked multi-OS environment.
-
Finally John views the possibility of Apple moving from it's MacOS X HFS+ native filesystem to some other with alarm; I see this as evolution.
Apple no longer lives in it's own comfortable bubble. It's now a peer OS in world increasingly sophisticated and fast-moving. Having grafted MacOS's strengths onto the Next operating system Apple has now entered the rejuvenated unix environment and needs to compete not only on it's own terms but also on those of the other modern operating systems.This relies on the MacOS always having appropriate mappings between filetype/creator codes and those annoying DOS extensions - not something that is always possible. Furthermore in an increasingly networked future it's not always assured that files will pass directly in & out through the OS but rather will likely just as often come & go through alternate transports, all of which would have to all be rewritten to support this. As this enforced-extension functionality is already standard in many applications it seems reasonable to simply codify it there then rewrite everything else, particularly as the creating application will have far more insight into the appropriate extension then the OS could.
Personally I would always prefer any extension-addition be made and clearly communicated when I explicitly create a file and not later when it passes in and out of MacOS-metadata-supporting networks and filesystems. Just as John is appalled at the proposal for hiding these extensions from the user's view I'd be appalled at their being automagically added to my file's names at some later date when they may get moved around or viewed from another OS. At least when I name a file "whiz" and the application insists on creating it as "whiz.bang" I know about it, I don't find out later that my "whiz" is that on some servers and "whiz.bang" on others or it's "whiz" for the other Mac users and "whiz.bang" to the *nix & Wintel folks.
HFS+ is a fine filesystem but it's unique in an increasingly unnecessary way. Other more modern filesystems are being created and if MacOS X is to remain current it needs to keep up and take advantage of these advances. Journaling filesystems are poised to become a standard feature of modern *nix implementations - should Apple lock themselves out of this? Furthermore it's not obvious that new filesystems will necessarily obviate the MacOS-metadata (ReiserFS seems particularly well poised to eventually incorporate much of this) but preparing for all eventualities seems wise.
While it remains important to retain those strengths that have made MacOS such a survivor it's also necessary to not hobble it with dependencies on unnecessary Apple-only limitations. Flexibility is the order of the day and this includes some reasonable level of filesystem versatility. Apple already supports a variety of filesystems now it's come time to allow for the possibility of multiple "native" ones while retaining much of it's vaunted metadata strengths.
I don't read ACs: If a post isn't worth so much as a nom de plume to its author then I wont bother either.
Mac files do take some extra space, but it is the resource fork that causes it, not the meta data. Since the resource fork is implemented as a separate file, a small Mac file uses 2 disk sections, not one. The type/creator meta data is only 8 bytes.
d ata-6.html ("Second, I mention it because...")
The article addresses the common misconception that the resource fork is meta data at some length in http://arstechnica.com/reviews/01q3/metadata/meta
I agree that handling metadata in MacOS should be easier. Just a simple command to view and edit them would solve most problems. But don't confuse that lack of tools with a fundamental problem with the nmeta data concept itself. And as someone else pointed out, there are rpetty good freeware tools available. to fix this.
After using the Mac for 10 years (and the PC alongside... and a lot of Linux as well), and after reading the mentioned article, i must say the Mac way is the best way -- in a closed model. For years, it's been so conveinent: no file extensions, nothing to worry about. Heck, even I add ".txt" to a Photoshop file, it'll still open with Photoshop correctly.
.psd. That's really a Photoshop file!" So, it opens then with Photoshop. Alright sir, 'nuff said.
Add to that Windows.
Now the Mac has to be aware of Windows files. It's a Mac "control panel" called File Exchange. If there's a file without a type/creator metadata (which the Mac depends on, in part), File Exchange says, "Hey, that's a Windows file that ends in
Add to that networking/internet.
Now the Mac not only has to worry about file extensions, but also its forks (data fork / resource fork). So if I send a program over email -- even to another Mac -- the result will be garbled data that won't work. I have to first convert the Mac file to MacBinary -- which squooshes together the forks. On the other side, it can be uncompressed into a two-fork program again and it works perfectly.
Eh, sorta annoying, but I compress things I email anyway because I hate emailing huge files.
Mac OS 9 gives me no problems. Files work right over PC networks, etc. Mac OS X works even better over networks -- in fact, in my work it is much smarter and works more efficiently than Windows NT (or even Linux).
The problem? Now Mac users have to worry about file endings --- in a sense. Applications use the Bundle methodology. Works great!
Files, on the other hand, *sometimes* need extensions.
When don't they?
If the file is opening in a "Classic" application (meaning it is being run through the old mac os 9 codebase... though it's not really "emulated"). Because those "old" files HAVE type/creator codes the Mac understands.
When DO they?
If it's purely a Mac OS X file, for the most part. Now in 10.1, the file endings can be hidden. But that doesn't solve the real problem: the Mac is battling PC/Unix files from the net AND its original OS 9-and-lower files that now have to carry redundent metadata.
Apple really needs to solve this. I know a lot of the OS X programmers, and they're extremely committed and bright, so I'm sure the problem will be fixed. Most importantly: make the Mac work great over the 'net (which is does really so far), AND make the experience very easy. I HATE file extensions. I love the old type/creator method. But I'm sure it could be done even better to satisfy all.
The next comment I write will be ready soon, but subscribers can beat the rush and see it early!
A few articles at The Register have talked about how MS will be "building SQL Server into the OS - effectively making the file system a relational database." This will greatly improve efficiency, and concentrate them into one database format instead of a mixture. And its even probably legal.
This isn't a problem with metadata, just a problem with MacOS' file typing.
BeOS handled all this very well. Double click to open with the default app. Right click to see a list of every program on your hard drive that opens that kind of file or files like it. (IE, a text editor would show up as an option for an HTML file.) Choose another option and open a dialog to set a file-specific preference.
I must have said "BeOS did it better" about six times today. I feel like an Amiga user.
tell application Finder
set creator type of file foo to "8BIM"
set file type of file foo to "EPSF"
end tell
run that in smile and your problems are solved... or, you can just use snitch.
2 1337 4 u!
Similarly, the windows system, with file extension associations that are essentially a total mystery to the average user is also tremendously confusing. You can install some lame-ass scanner software and have it decide that it owns all the image file types you used to have associated with photoshop. Now, how do you get back to normal?
The point that I'm making is that doing almost anything "automagically" has the potential of being a source of confusion. UI designers need to think a little bit more about empowering the user rather than just concealing things from them. Obscurity != ease of use.
(I strongly suspect that hiding file extensions by default was a really bad idea.)
Why does the file name have to be there?
Why are we still using file systems?
Computers are quite capable of managing huge organized trees of data. Why are we still fighting with bitstreams like this?
--TheOrangeSquid Is it any wonder things seem so awry? We swim in a sea of confusion and don't have to think to survive
While AmigaOS used .info files to keep track of things like applications, I did like the way IFF was used as _the_ file format. Sounds, images, they were all was stored in IFF files that kept track of what exactly the file held. Sort of like a bundle.
Then there was Data Types. The theory was that if an App knew about data types, it didn't need to know how to write a particular format as long as Data Types did know. I liked this idea. If a new format came around, I didn't need to update all my apps (as long as they knew how to use Data Types).
Xix.
"Everything is adjustable, provided you have the right tools"
Yes, index.html is an HTML formatted file - yet it's also a TEXT file - and it may contain Java/PHP/etc Scripts embedded in it. Do I want to limit a file to only one "Type" when it can be many, depending upon how I choose to use it at a given moment? Of course not!
It's true the older MacOS file type methodology worked well when the Mac was used as a much more limited system. The fact that MacOS X includes Apache is an example of how much more versatile the Mac is today. In order to be more versatile, you have to reduce some things to the LCD. To do otherwise limits the usage of the system and that's exactly why the Mac has remained the cherished possession of so few. To expand market share, Apple needs to expand the uses of the Mac - that means more flexability, and less stringent ties to those technologies (however useful in old-school aspects) that limit that flexability.
I AM, therefore I THINK!
In the current shipping version of MacOS (read X) you can "Show Info" on a file (same thing as in "Get Info in OS 9) and there is a pull down menu that gives the option application. Its very easily noticed. There you can reset not only the meta data for that file, but also, the default for all files of that type. You no longer need to use file typer. Anyhow in OS 9 this was a non-issue. If you wanted to open something with a different application, all you had to do was drag the file onto the icon for the app you want to launch it. You could even do multiples.
"I hate downloading things on a Mac because of this. Some idjit will have a file set to open in an application I don't have"
This is deceitful. If you download a normal everyday file off the internet, odds are not going to get the meta information of that file, even if it was orginally on the Mac. Almost every single application for downloading on the Mac uses the System level settings for mapping extensions to type/creator, which will ALWAYS be an app you have. The only times you will get meta info with the file is when you 1. download a stuffit archive and 2. use hotline. I personally can't remember the last time I downloaded a file as a stuff it archive, unless it was an installer, which in that case it was irrelevent since it was self contained. Now if you are heavily using Hotline for exchanging files, then odds are you are a pirate and deserve what you get.
Burn Hollywood Burn
One thing that bugs me: when a program (e.g. Nautilus) builds a thumbnail for an image file, the thumbnail isn't attached to the file, it is stashed somewhere (e.g. in a hidden directory called ".thumbnails" or something like that). This is a hack.
It's worse when you have multiple programs that want thumbnails; there isn't a standard yet and you get multiple thumbnails.
What I really want is some metadata attached to the image file, and the thumbnail in there. Then when you copy or move the file, the thumbnail goes along. And of course we need a standard so all the programs that want thumbnails will all do it the same way.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
wrongo. How do you know a text file is a text file?
.txt is pretty unique in that many people can guess that it probably means "text". Most extensions are not as clear, you just have to know them all by heart to be able to use them.
Windows: It says ".txt"
Mac: It has an icon that people who've been using macs for a long time know means "text"
Or to use the same standard for both systems:
Windows: It's name ends in ".txt", which people who've been using PCs for a long time know means "text". Unless they're watching the file name with the extension hidden, which is often (but not always) the default setting
And of course
The main issues regarding file attributes are "how do you find it", and "what do you do with it when you've found it". UNIX users are used to thinking of these as being tied strongly to file names, but that's not fundamental. Consider Microsoft's Fast Find and Active Directory, for example. Or the way the MacOS handled applications; it didn't matter where they were, because there was a database (the "desktop") that automatically tracked them.
Much of the complexity associated with UNIX programs involves finding their various parts. This usually involves some combination of path variables, configuration files, command line options, shell scripts, and fragile directory tree structures. Something better is needed.
A separate issue is whether files should be flat streams of bytes or should have structure. The Mac had both; the "data fork" of a file was a byte stream, and the "resource fork" was a tree of records, rather like the Windows registry. This was a good idea, although it suffered from the fact that the machinery for updating the resource fork was prone to corrupting it, which discouraged its use for dynamic data storage. (Or, as Apple put it, "the Resource Manager is not a database.")
Many of Apple's better ideas suffered from what Mac developers called the Mess Inside. One major effect of this was that things that required keeping complex data structures consistent didn't work too well. Application bugs could corrupt the desktop or resource forks, and the system-level machinery which processed those data structures didn't check them. ("It's more fun to be pirates" - Steve Jobs.) So data tended to turn to mush, which gave resource forks a bad name. Today, most programs store all the important stuff in flat files.
The current problem is that filesystems don't make it easy to store properties of a file. The HFS made a brave attempt by dividing files into content and properties (using data and resource forks), but it still didn't objectify the filesystem. For example, creating children of a particular file necessitated converting the file into a folder and then wondering what the hell you were going to do with the folder properties - were they going to be placed within the folder or within the parent folder.
A solution would be an object-oriented filesystem, that allows every file to have children without nasty conversions, and implements a simple store for properties (a Berkeley DB file would seem a natural solution).
Sometimes I want vim, sometimes I want gnotepad, sometimes I want something else. I never want Word, and I don't want some stupid meta data setter telling me I do. No, thank you, DickBreath.
Friends don't help friends install M$ junk.
Like I said, I never want Word. It would really upset me to Work on a project with some dickbreath who used word, if his modifications would make name.c open up that way from seeing the brilliant created with metadata.
Friends don't help friends install M$ junk.
Linux does have file change notifications, it's just not integrated with the mainline kernel yet. IIRC, it's SGI's Irix implementation which they opened for Linux.
I'm talking about a modern UNIX, meeting the resent specifications (I believe UNIX 98 is the most recent).
The ORIGINAL UNIX was designed well for its time. I would suggest that there is little in common (code wise) between the original UNIX and modern UNIXes. The BSDs and Linux share NO code with the original. While the commericial UNIXes may share some code, overall the have been rewritten.
Now, each vendor makes their own UNIX. The "Linux" distributers make their own system, though the kernel and the C-library appear to be standardized.
The point that I am getting at, a modern UNIX is a well engineered machine. However, when you write portable code, you write either to the specification for all UNIXes (the designed by committee standard that hasn't been updated in 3 years, and most of the standard is MUCH older), or you write to a particular UNIX. Yes you can write a fall-back to the standard and optimize for your platform, but you really haven't helped then, have you? Your platform is optimized, the rest just run.
Take a current issue in the BSD community. The BSD rc system is beautifully simple, and less of a mess to manage than a SVR4 system. However, it does have some problems in that there is no easy way to bring down/up services, etc. We wrote our own service script (inspired by Redhat's, but custom to us) that we edit the configuration for as we add services. This gave us some control of the system.
NetBSD wrote a new rc system that uses shared scripts and small configuration files for each daemon. It seems to be an intelligent system, without the mess I have seen in Linux systems.
FreeBSD appears interested, and will likely adopt it.
For reasons that AREN'T clear if they are technical or political, OpenBSD will not. This means that people writing to the BSDs can choose to support two systems (the NetBSD/FreeBSD AND OpenBSD), or target one platform (either NetBSD/FreeBSD OR OpenBSD).
Now in this case, the rc system, it is less of a concern. Targetting both is trivial. However, as most people use FreeBSD or NetBSD, those of us in the OpenBSD community are part of a smaller niche.
However, we see the potential for a problem. Targetting the BSDs requires targetting BSD 4.4. Any extensions made by the BSDs may or not be ported to the rest. Playing nicely with Darwin/MacOSX is another problem, but Apple seems to be willing to follow FreeBSD's lead on the BSD side, focusing their creative/decision process on the Aqua/Quartz side.
To get the BSDs to move foward, you need a committee that shares improvements (unlikely for political reasons) and gets them standardized when everyone moves foward.
IDEALLY, you can target your platform of choice, but the features you want will be in the other platforms within 1-2 years. That way, you can even have a fallback for lagging systems. However, there should be a process by which useful extensions become part of the base, so the base is slowly evolving.
I understand your preference for a less-is-more approach, which I think makes sense for the system/kernel. That design philosophy encourages better systems. However, when there is a way to do something, there should be an attempt to develop the best way, then everyone should use it instead of reinventing the wheel.
The core UNIX system has two standards, POSIX and X11. Both of those systems were designed separately, and you can target them and it will work everywhere.
If you want a GUI app, targetting X11 is a pain. Targetting Motif is slightly more pleasant. Targetting Qt is easier. However, you can't target ANYTHING but raw X11 until you know that the standard requires something.
That is what I'm getting at. Motif was adopted as the UNIX standard long ago, but it's non-free status has kept it out of Free systems.
Java promised to be that universal platform that we could all target. In an ideal world, there would be no more native code for applications, all new applications would target the JVM or something similar. Java offered the idea that MFC did for Windows programmers, a real environment to build applications in. Unfortunately politics and other limitations got in the way, and we still don't have a universal platform. Targetting all UNIXes is okay with an LCD approach, targetting all UNIXes and WinNT requires a different approach. But nothing out there presents a good way to move the common base foward so I can build an app that is optimized for everything, or at least will be when everyone catches up.
Alex
A comma delimited list of numbers is a comma delimited list of numbers in accordance with some pre-defined structuring rules.
The file type is immutable. How we use the files may differ on context.
> Current shipping version of Windows (ME) does not allow that. You have to right click on the
> file and get the properties dialog, because....the extensions are hidden.
Bah. No one who knows anything about Windows uses ME. It's just Win98SE, but slower and bloated with more useless features. But even if you *are* using it, just click on the "View" pull-down on any open window, select "Folder Options," click on "View" tab, and un-check the box that says "Hide file extensions for known file types." Then the file extensions will always, always, appear for all files.
Before you complain that that's complicated, you only have to do it once--and anyone with even the most basic skills knows that he needs to customze his preferences when setting up a box, whether Windows, Mac, or *nix. And you also should be aware that Apple has plans to do the same little preference about showing/hiding file extensions in the next OS X release. It's a way to make the extensions invisible to really dumb people who wouldn't know what to do with them and shouldn't be allowed to accidentally change them when they change a file's name, but still allow users with a modicum of experience (enough to un-check a box, which isn't much) to do their magic. And BTW, I did point out in my original comment that showing all file extensions is no longer the Windows default and must be changed by the user.
Therefore, that "arcane" dialog box for changing extensions in WinME is unnecessary once you un-check a prefeence box.
> Just not true. Netscape, IE, all every FTP app I can think of all reset the type and creator
> on download
If that's true, it's insane. Do you know what a CRC or CSV or SFV is? It's a small algorithmically-derived digest of a file, or of an entire set of files, which can be downloaded and automatically check the files to make sure none are corrupted. These are very commonly used to ensure file integrity, whenever there is a file or series of files of some importance which one wants to ensure are completely original and intact. Since type/creator is not in the resource fork, but in the file itself, altering them would alter the file in a very small and usually insignificant way, however the CRC/CSV/SFV would be rendered absolutely useless, since a change of even 1 bit would show up in the digest and therefore the file would be reported as corrupted. That is just insane, if true. My downloading software should IN NO WAY modify my files, without my explicit knowledge and permission. It renders file integrity checking utterly useless.
The last version of Netscape I have ever used, on a Mac or otherwise, is 3.04Gold, so I would not know. I do recall that Netscape had a list of "helper applications" for what app should open what file--but that was only used if you selected within Netscape to "Open" the file upon download, not when just "saving" a file. I now use IE on Windows and Mozilla on Linux, though when Mozilla for Windows gets a bit better and faster, I plan to switch to that too. If what you said is true, though, it is absolutely stupid for the reason I mentioned above. You can't have file verifitcation if your OS midifies the file in any way whatsoever. File verification is especially important when downloading software; the software companies, particularly ones who make security software, often offer CRCs to their customers so that the customer can be confident that he has not received a trojaned or otherwise interfered-with software package.
Second, you may not download much in StuffIt archives (who does)? But I download an awful lot in Zip and RAR archives. Doing that on a Mac would not affect type/creator of the files within the archive. In fact, you will find that Zip is the most commonly used method for transferring multiple files at once.
Chasing Amy
(We all chase Amy...)
"The more corrupt the state, the more numerous the laws"-Tacitus
First, you say we don't need no stinkin metadata, then you say we need it with more power and flexibility.
I think we can all agree that application binding is a cool thing, and saves us a lot of work as an automatic shortcut to opening documents.
The problem here is, and NOBODY has gotten this right so far as far as I'm concerned, is having a decent way for power users to edit and manipulate metadata, and configure the OS's treatment of it.
Yes, the high and mighty programmers have access to it. The hackers have access to it. The grannies don't want or need access to it as long as application binding functions in a basic, and intelligent way. The power users have access to it, the same way the hackers do - but it's often a pain in the ass, using tools that weren't really designed to do anything other than mess around. Nothing useful can be done with these tools ON ANY OS, in terms of allowing a power user to quickly and easily manipulate the metadata to set up a custom behavior that suits his or her purposes.
And that's really the whole problem.
That, and of course the fact that filename extensions really have got to go.
You'd think that the OS vendors would think about this, and provide the users with some nice tools. And I'll agree with you, Microsoft's solution is kind of nice. Where the CM for a Batch File in explorer will give you the choice to RUN the batch file, or Edit it. I think power users need that kind of flexibility for html as well, OPEN the file in a browser, or OPEN the file in an editor.
I'm often frustrated with graphics files - sometimes I want to run a quick image viewer to display an image file - sometimes I want to tear it up in Photoshop. Launching Photoshop is a 60-second ordeal on some people's machines, and it's a necessary ordeal if you want to do serious editing.
You see the problem here? Granny needs the file to open in her browser on a double-click. The power-user or content creator needs TWO choices on the execution of an icon. Edit or View. And in the case of executible content, Execute. This needs a user paradigm - probably a lot easier to use than a CM. And it must be MUCH quicker than the stupid "open this unregistered file in one of these " deal, which is annoying and slow on every OS I've seen it.
when I think about how annoying this problem is, and how NOBODY has ever offered a real, workable solution for this. I see - an opportunity. . .
These are my friends, See how they glisten. See this one shine, how he smiles in the light.
So would data about Metadata be... "Gitadata"?
I know, but I just had to.
Kurdt
I'm not anti-social. Just pro-technology.