Slashdot Mirror
The Challenges Of Integrating Unix And Mac OS
Schemer writes: "Wilfredo Sánchez, the lead developer on Darwin has posted his usenix paper, 'The Challenges of Integrating the Unix and Mac OS Environments' on the Web. In it he describes the difficulties and solutions to the problems encountered while trying to adapt BSD Unix for use with MacOS X. It's a very good read, even if you aren't a fan of the Macintosh." The OS X team have been dealing with the serious complications of mixing one established, beloved interface with another -- this is a thoughtful look from the inside at how they've dealt with it, and a good explanation of some underlying assumptions and conventions of each OS.
maybe they should talk to Hemos?
___
Apple might consider spending a little effort on keeping some of its biggest supporters -- developers -- happy.
I use Macs for work, Linux for education, and Windows for cardplaying.
From starting with 'case preserving' but 'case insensitive' filenames ... ??? WTF???
Excuse me for being overly indoctrinated with Li/*nix -- but something seems wrong with this.
If your foundation is not stable, then anything you build will also be unstable. I feel this to be the case with this filesystem.
This post encoded with ROT26. If you can read it, you've violated the DMCA. Handcuffs please, sergeant.
Not a good start to the article. The operating system that Apple developed is poor at best; a single application crash almost always brings down the machine.
The interface, on the other hand, feels solid and reliable; serious thought went into every part of it.
Nice GUI, shame about the OS.
--
"Where, where is the town? Now, it's nothing but flowers!"
My greatest impression from reading the paper was one of a schizophrenic system. The differences between the two systems that form the basis for OSX seemed not to have been resolved, instead they were patched over with an additional layer of complexity, and a great deal of hope applied that the two different OS's at the core wouldn't misbehave and contradict each other.
Usually, this is the sort of thing that makes software developers run screaming down to the pub.
Charles Miler
--
The more I learn about the Internet, the more amazed I am that it works at all.
The interesting question is: should the day come that Linux implements metadata, could/would the Apple team merge the same Unix API into the BSD layer of OS-X?
www.eFax.com are spammers
That actually seems like it'll be their biggest problem. The paper states that MacOS X will translate slashes in Mac filenames into colons when read from BSD. Um, the colon is like the universal token separator under UNIX. (Paths, passwd, etc).
A similar problem occurs where I work. The directory naming convention for packages is packagename,vX.X. Unfortunately, some programs use a comma as the separator, and you can figure out what happens.
Personally, I never get hurt by this, because I use the intersection of available characters from all OSes in my filenames. That means things like colons, slashes, commas, and usually spaces (just for easy shell access) are always out.
TO BUY A NEW CAR WOULD MAKE YOU SEXUALLY ATTRACTIVE.
"More subtle" is perfecly valid English. The construction works better for some words than other; it's fine in this case.
--
"Where, where is the town? Now, it's nothing but flowers!"
I said "filenames", not "pathnames". MacOS will let you have a file called "baz:foo/bar" ("foo/bar" in the "baz" directory). BSD programs will see it as "baz/foo:bar". Now, try accessing that file in a colon-delimited environment, like your password file. Or your path.
TO BUY A NEW CAR WOULD MAKE YOU SEXUALLY ATTRACTIVE.
Since the core of the OS X is unix. Does this mean that quicktime will be ported to other *nix systems?(i.e. linux)
No. QuickTime runs on top of the Carbon API layer on Mac OS X. The Carbon layer is effectively the previous Mac OS 9 API and libraries running on top of the Mac OS X BSD and Quartz layers. Moving QuickTime from Carbon/Quartz to another UNIX is no easier than moving it from Mac OS 9 to another UNIX.
If you ported Carbon and Quartz from Mac OS X to (say) Linux, then you could relatively easily move QuickTime over to Linux, but that would be a massive effort and isn't going to happen any time soon.
Sailing over the event horizon
People keep asking this, and the answer is probably not. The BSD layer is in the middle, underneath all the GUI stuff. Anything graphical is written using Apple's proprietary APIs.
I can imagine, though, that if you're running on PPC, something like Wine might work out. And of course, you can run MacOS under Linux/PPC right now with Mac-on-Linux. (Hey, anyone know if MacOS X will work under that?)
TO BUY A NEW CAR WOULD MAKE YOU SEXUALLY ATTRACTIVE.
Sad to say, I've had similar experiences with Be, Inc.'s treatment of developers. While I feel that the BeOS is a technically superior operating system for desktop machines, I think the only sensible route for a small third-party developer is to develop for an operating system which is GPL'ed.
Mike
-- Could you use my software consulting serv
One of the things that I've seen being discussed in the Kernel development threads is adding metadata support to Linux.
The nicest idea I've seen is Application Logical Bundles of Data (ALBODs), which look like a tarball if you open(), stat(), etc. them, and look like a directory if you opendir(), readdir(), etc with them. Check out Treating Multiple Files as One for Kernel Traffic's summary of the linux-kernel discussion last year on this topic.
This would allow applications to do the kind of substorage that they need to efficiently; a word processor document may have jpegs, spreadsheet tables, etc. embedded into it; things that can be more efficiently stored and manipulated as a directory full of files than as a single file. However, people don't want to manage a directory for each document, they want to manage a file. So, only new applications (or wily users; imagine being able to copy the images out of a document with "cp my.doc/*.jpg ~") that know to expect an albod actually try treating it as a directory; old tools that try to treat it as a file just get a file, so "cp my.doc my2.doc" still works without changing cp.
Unfortunately, this wouldn't let you add forks to normal files, and in fact adding forks to normal files seems impossible without breaking POSIX semantics and rewriting a bunch of old programs. You got it exactly right: "cp", "tar", "ftp", "apache", and a billion other programs act on files by simply doing an open() and read or mmap on the file. If we provide metadata at the end of that reading, then it gets copied correctly, but the program actually using the file probably breaks. If we don't provide metadata that way, then applications work unchanged, but we need to change all the applications that copy files around. And either way, what happens when you use FTP, scp, HTTP, etc to transfer your file around? Do you send the other end a tarball and leave it to the resource fork aware receiver to handle it ok? It'd be a mess.
Mac OS is built like a tank in the areas that really count in the area of average user computing. A program can be installed at any time and that won't kill other, existing programs. Similarly, a program that is already installed will never preclude another program from being installed (like RPM does). And if you delete all the mac OS configuration files, programs can still run. How many times has someone installed one windows program and this has totally killed another working program? Or the registry got corrupted in one particular area and dragged other areas down with it? Or how many times has someone has tried installing a linux application, only to find out they have to screw around with environment variables to even get the program to start up. Mac OS doesn't have these problems, and in the Reality That Is The Average Joe Consumer Desktop(tm), crashing is far less of a concern than something not running at all or something screwing up the computer. A crash in this area of computing is an annoyance, something failing to work at all is completely unacceptable.
/etc, on windows delete the registry, and on MacOS, trash the preferences folder. Reboot and see which computer has the most functionality that an average computer user requires. Guarentee you it won't be the first two. Which is unfortunate since it would be great if linux and windows had the system/application integrity that consumer level usage requires. True, anyone geek enough could fix the first two computers, but keep in mind that most average joe computer users are not like us geeks.
Let's do a test. On a unix system, do rm -rf
Wanna bet Bill Gates is losing sleep over the possibility of an essential merger between the Mac and Linux communities, based on a (presumably) evolving interoperability and transparent portability between the two?
--
Sheesh, evil *and* a jerk. -- Jade
Meta data is a seriously nifty concept if implemented correctly. BeOS has a similar (though much superior, read up on it) system called attributes. Not only does it allow for info such as icon to be stored within an attribute attached to the file, but it allows things like bookmarks to be implmented entirely as attributes. The cool thing about that is that since attributes are indexed by the file system, they are searchable database style! The contacts database is a set of files with attributes, and just by moving a file into a directory you get an instantly searchable database of contacts. Of course, it necessitates some additional stuff. open() can't read attributes (and it probably won't be able to read meta data, an alternate OS call would have to be implemented). Also, you'd have to switch to something like the .zip format because gzip and tar don't preserve attributes or metadata. If it can be done within the Linux architecture, I think it should. ReiserFS is getting database functionality eventually, and if you could connect the two, you could do some nifty tricks. (And isn't that what Linux is all about? :)
A deep unwavering belief is a sure sign you're missing something...
I think the NT guys have the right idea on how to support backwards compatibility with two different OS architectures (don't laugh!) Microkernel OSs (like NT) use servers to provide system services to applications. In NeXT there is a BSD system server running on top of Mach. NT has POSIX, OS/2, DOS, and Win32 servers running on top of the NT kernel. What the MacOS X people should have done is have a BSD/Cocoa server along with a Carbon server running next to it. What they chose to do is layer it instead (Carbon / BSD /Mach). I have a bad feeling that comes more from the macrokernel approach of MacOS than any conscious design decision.
A deep unwavering belief is a sure sign you're missing something...
I'm shocked by you, grammar nazi. Though you usually find a couple of the more egregrious grammar errors in documents and miss the more subtle ones, I find it amazing that you make so many in your posts.
;). Additionally, according to my 20 volume OED (as well as dictionary.com), "Arian" refers to one born under the sign of Aries or relating to the Roman bishop Arius. Perhaps you were looking for "Aryan," which refers to the Nordic Caucasian gentiles of Nazism.
To be fair, my first nit to pick is style and not grammar: "imroper, non-American, and redundant grammar" is indeed redundant from your point of view
I've seen various other grammar errors in your other posts, but have refrained from pointing any of them out to you. I suggest you pick up a copy of Sleeping Dogs Don't Lay : Practical Advice for the Grammatically Challenged or, preferably, Elements of Grammar. In any case, you can always email me for grammar tips; I proofread theses and papers for fun. To the morrow.
Quite possibly...
We now, once again, has a filesystem which is sort of compatible but loses information
You mean like Linux's support of the DOS filesystem? If I copy a Linux file to my DOS partition and back, I lose the permissions. With MacOS X, it's less radical than that; that only happens when someone uses the same physical volume on both MacOS 9 and MacOS X (which for most users will be exceedingly rare).
(compare long file NAM~1s).
That's not a good comparison; the NAM~1s creep up all over the Windows "experience", from FTP transfers and the like. Getting default permissions on a file instead of the real permissions? Bah, most users would hardly notice, especially if sensible defaults are used.
We have "special" directories which the user "should" not look into.
No, we have a UI that hides certain uncommmonly-user-manipulated directories from the user. Much like files beginning with a period in Unix. If you know they're there and want to muck with them, well, go ahead, but be sure you know what you're doing. Besides, the bsd-ish directories aren't hidden from MacOS X users in the shell window.
We even have the very same double-root kludge: the underlying os (DOS/unix) expects one structure, and the user another, so we have a "real" root which users are not supposed to know about, and then the friendly /usr/lib/desktop, no doubt.
MacOS X's structure is fairly elegant in this domain, please see the online docs at apple's website for a clue about how this is done.
And programs are supposed to move to a "package" model for un/installation, like the "add/remove program" item in the Control Panel. I predict it will take five years for Apple, too, to get that trick to work.
Well no, it's the "add/remove program" item and uninstaller type of nonsense that they're trying to get away from. Instead, they have all of an application's resources bundled up together in a single directory. This way, the user can do the intuitive thing: drag the Application from the installer disk onto their own disk, and it's installed. Drag it to the trash, and it's gone. No registry entries to clean up, no preferences folders or extensions or random little files laying around. One nice big handle to pick up a whole app by.
This is not to say that the problem could have been solved much better (although I personally think they should have discarded much more of the Unix side of the system -- they have no obligation to be backwards-compatible there!).
I think it's highly debatable whether Unix is something one is "backwards" compatible to, especially in this audience :) Going to a Unix-based OS is a very bold move, and rather than reinvinting the guts of a high performance scalable OS, they've chosen to adapt to one. With that decision comes the obligation to actually interoperate with said OS.
But the net result is a less elegant system than any of the ingredients (MacOS, Unix, NeXt) that went into the brew. This makes MacOS X a much less interesting OS alternative for me.
From what perspective is this less elegant? (and have you seen linux source code!?) It's far more elegant than letting these issues fall on the floor; instead the average user, and average developer, get a tightly integrated OS with some impressive functionality. Apple's been very good about smoothing technology transitions (e.g., x86->ppc), and this continues to be true.
My $.02
This article goes a long way towards rebuilding my respect for Apple, which began to wither years ago when Copland never appeared.
For a long time I really wanted Apple to come out with a new OS that would kick the pants off of System 7, and I could have cared less about backwards compatibility. I would have bought all new applications, converted my old data, and kissed the old Finder goodbye. I bet some developers at Apple would have liked to have started from scratch too.
I'm glad they didn't, because they would have left a lot of people out in the cold.
The fact that Apple is making HFS work with a unix filesystem shows that they care about thier customers. My grandfather has been a Mac user since he gave up his TRS-80 in 1988. He's published three books and dozens of papers on his two Macs. (his current a 7100 that still runs beautifully) I really think he is a poster Mac user. He is very intelligent, creative, and he has no interest at all in how his computer works - it's just a tool to him.
Tell my grandfather that to use a new operating system he'd have to give up programs he's used for 10 years, and convert all his files for use in a new filesystem, and he'd tell you that his current Mac OS was just fine.
While PC users (myself included) have been sucked into the complete hardware / software upgrade every 2.5 years cycle, the core constituency of Mac users hasn't - and asking them to change everything just wouldn't work.
So please read the complaints about the mechanics of OSX in this discussion keeping in mind the balance between pragmatism and idealism Apple has kept.
Idealism: Make a perfect OS
Pragmatism: Make it backwards-compatible
Idealism: Care about the customer enough to be pragmatic.
:-)
Of all the ways of dealing with file resources, I think only the Windows format has got it right. When I speak of resources here, I will limit my discussion to the icon for an executable file.
/usr/share/icons or somewhere. Personally, I have always hated this. I like programs to have a known icon associated with them. For one, it makes them easier to recognize when using other's systems.
In my opinion, whoever came up with "data" and "resource" forks in Mac OS was an idiot. What you have is a file that is effectively two files. This is fine as long as the files are only delat with on a Mac, but when you get into file servers or trying to FTP these files across the internet, all sorts of complications arise. The two files have to be somehow combined into one. This makes it really difficult to deal with Mac files on other OS'es, as the author explains.
On Unix/Linux X applications there is, as far as I know, no way to imbed an icon in an execuatble file. If one is even included, it must come along as "icon.xbm" and be put into
Windows, or the PE executable format, I believe, has it right. The icon (and other resources) are embedded in the file, and are shown as the default icon, even though they can be changed, if you so desire.
I conclude that Unix/Linux needs either a new executable format that will allow for standardized resource embedding, or an addition to the existing executable format (whose name I forget) to allow this.
I'd like to hear other's opinions on this. I think this is one thing that would help make Linux a better desktop OS.
gratuitous talking heads reference?
---- I made the Kessel Run in under 11 parsecs.
Do you have the faintest clue why it is not feasible to port Mac OS over to x86 hardware? If you can put Unix and user friendly on the same box then you're going to rush boxes out the door. The problem here is that Apple produces their software and hardware and therefore don't need to worry much about third party driver support with their hardware. When you port OS X over to Intel hardware you're got people wanting to stick it on their 400$ chump change PC with parts that have absolutely no drivers for the new OS. If you've ever tried getting X to work on a POS cheap PC you're really out of luck. Since hardware vendors are reluctant to spend the money to produce Linux drivers, why the fuck would they produce OS X drivers? I don't think GNU Beta home brew drivers are going to fly with the ease of use crowd.
I'm a loner Dottie, a Rebel.
The problem with Win95 and 98 was that they run apps in a virtual machine; there is an 8, 16, and 32 bit VM that is launched when an 8, 16, or 32 bit program is launched. DOS apps are each run in their own separate VM which means they won't kill other DOS apps (usually). The main stability problem is in the 16 bit arena. All 16 bit processes are run in one large memory space and cooperativly multi-tasked. This means a fault is one 16 bit app (library, extension, ect) will take out all of the 16 bit apps. Things arent really run in a layered fashion, 16 bit resources are just managed in one big heap on top of the process scheduler which make it seem like you're working through layers. Unix also has a process scheduler which does the same thing as the one in Windows, it is the basis of multi-tasking and premptive multitasking. The ability ro run KDE apps in GNOME has nothing at all to do with the system's scheduling and process handling, it has to do with the libraries that are available. As long as a program can access the libraries it needs it can run since both GNOME and KDE are running on top of X.
I'm a loner Dottie, a Rebel.
Have you heard of the 'file' command? It isn't perfect, but it works most of the time.
WindeX?
U-knows?
I thik maybe I should call it maybe the McGuyver Operating Environment. (MOE.)
I didn't understand why they cannot integrate hard links, the author only seems to explain the consequences, not the reason for it. Anyone care to explain?!
A lot of people, including me, have notebooks as their primary computing platform. I run Linux on an IBM ThinkPad 770Z and it works great as a development platform for my server-based applications.
So I agree with the original complainers - it would be a bummer for the OS not to work on PowerBooks.
D
----
There is no 8 or 32 bit virtual machine. I don't know where you get your info. DOS has *never* been 8 bit. Its main selling point when it was released was that it was 16 bit and CP/M was not.
I've finally had it: until slashdot gets article moderation, I am not coming back.