Slashdot Mirror
Brief Tutorial on Reverse Engineering Mac OS X
rjw57 writes "There is an article on OSNews I wrote about how the guy behind Desktop Manager goes about reverse engineering APIs from Mac OS X with a brand new example not revealed anywhere else. From the article: 'I am often asked in email how I uncovered the API calls I use in Desktop Manager which are, unfortunately, undocumented. This article aims to give a little insight into the techniques I use to reverse engineer Mac OS X in order to provide extra functionality to users and extra information to third-party developers. In this article all the utilities I use are a standard part of Mac OS X's developer tools which are freely available.'"
Apple tries to make an example of our friend?
All credit to the author; it looks like quite a feat of reverse engineering and some genius coding has probably gone into it. Apologies for the subsequent inflammatory opinion.
... try having a real discussion instead).
However, Apple have already come up with a perfect way of handling large groups of windows on one screen; it's called expose. I used to use virtual desktops on Linux, which was adequate, but when I got a Mac I settled in nicely with Expose; OS X has a near perfect user interface designed by actual HID experts. The only reason I can think of for using virtual desktops is if you're some kind of Linux zealot.
(Don't mod me down just because you're an anti-Mac zealot mod
Great example of apples practice of breaking API's leaving developers out of the loop
It's an undocumented, unsupported API; that's what the article is about. What broken APIs were you thinking of? Carbon took over most of the OS 9 ones (apart from some clearly documented exceptions).
If you had read the article you might actually have had something useful to say.
It's an undocumented API.
That's one of the many reasons why some APIs are left undocumented: because they are expected to be unstable.
Can't really blame Apple on this one. They didn't publish the API, and changed it in Tiger to a more flexible three-part solution. Eventually they may decide the design's a good one and publish the API.
Until then, use it at your own risk.
In this example, Apple broke undocumented APIs. Anyone writing or using an application that takes advantage of undocumented APIs should be prepared to discover that they've been changed, moved, or deleted entirely.
The APIs that DesktopManager uses were probably left undocumented precisely because Apple knew they were going to be subject to change.
Apple is good, and we are going to talk to talk about it.
This space intentionally left blank.
I myself have found that by really learning how to manage windows the "apple way" I don't really feel the need to use virtual desktops much (I used to use DesktopManager).
For me, this means using Hide (Command-H), Swich app (Alt-Tab), Focus on window (active) or next window (a custom key binding like Alt-Tab), and Expose.
But that doesn't mean there isn't a place for virtual desktops.
One thing that expose relies on is that the conceptual groupings of "All app windows" and "All of this apps windows" are all you need. The problem is if you have a large number of similar looking windows from different applications it can be difficult to manage even with Expose.
Virtual desktops can give you custom Expose groups - which can narrow the search for a particular window. This can be useful if you are working on several complex tasks that use multiple windows from multiple apps (each task can get its own desktop), and also have a bunch of side apps - like your calendar, email, instant messenger etc.
So Expose solves the window management problem to an extent, but it can be combined with virtual desktops when things become even more complex.
So jobs invented capitalism and anything you have to buy is evil?
Do you live on a commune and make your own clothes too?
Did you buy your car, TV or make those?
The if it isnt free, its root of all evil" crowd is tiresome with the dogma.
I think people mean: this is a great example a post by someone who doesn't read the article. :-)
Personally, I'm all for adding cool new features to Mac OS. They will likely document them at some point.
I don't know, but it works for me.
So now we find out that Apple has used - and is using - undocumented API calls.
Sounds like something Microsoft would pull.
Oh, wait, this is Slashdot. I forgot.
Well, then Apple's just trying to protect its intellectual property. No harm.
Ah - right. So when a company has undocumented APIs then it is because they are not documenting them for altruistic reasons.
Unless the company is Microsoft - then it becomes evil.
The slight problem with this slightly simplistic argument is that the APIs are not undocumented. They are documented, otherwise the Apple developers would not be able to use them (and, after all, the article is about disassembling an Apple application to find out how they worked).
The APIs are just not documented publically - Secret APIs in other words.
Of course this is no doubt still a case of Apple's Secret APIs: Good vs Microsoft's Secret APIs: Evil, but it's late, and I'm not up to the mental gymnastics that will be required to make the leap. No doubt an Appleista will be along in due course to make clear the path to enlightenment.
I don't think there's anything wrong with Microsoft having undocumented (call them secret if you want) APIs. You're making the "all Slashdot posters share the same opinion" mistake.
This space intentionally left blank.
all API's are documented, well, API's that are written in a company the size of apple anyways and in any other software project done by pro's(hopefully anyways).
they're just some that some committee or workgroup decided that they shouldn't be used.. all closed os's have those.. hell, I'm frustrated at the moment at symbian because of precisely this(well, nokia to be more specific but anyways..). it bugs the hell out of me that there's a bunch of things in there that aren't open.. but that are in and usable and you need to use them for certain projects(and the api, including them, changes between versions anyways).
everything is subject to change. just that when it's undocumented you'll never know about it untill it just doesn't work anymore.
world was created 5 seconds before this post as it is.
Well, Apple has a long history of not documenting alot of their APIs. You know the little icon march that appeared when you booted Ye Olde MacOS? Even though it was widely used, it was never documented for 10+ years.
So now we find out that Apple has used - and is using - undocumented API calls.
Um, no Apple has no applications that use the virtual desktop APIs to compete unfairly with third party apps. In fact Apple has no competing application in this area at all, and two of the three applications that DO exist are open source.
On Solaris there is a command truss that is the king of all truss-like commands. Unlike strace, ktrace, and BSD truss this tool can print a trace of all function calls made by an application as it runs (among many other useful things). Does anyone here know of an analogous tool for MacOS X? If not I wonder if an awk/perl script munging the output of nm to generate tracepoints for gdb where each trace point creates a new tracepoint at the instruction where the function call returns, prints out the funtion name and the contents of r2-r10 or so, then continues on or something like this would be something someone has already written.
"He who would learn astronomy, and other recondite arts, let him go elsewhere. " -- John Calvin, commenting on Genesis 1
Actually, these are undocumented SPIs, Systems Programming Interfaces. SPIs exist in many layered systems, and are intended to provide functionality used to implement higher level mechanisms.
In Mac OS X, for example, the CGS SPIs provide functionality used by the Carbon, Cocoa, and Java environments. (You didn't really think Carbon and Cocoa somehow each implemented their own window systems, did you?) The SPIs are private because, simply, the functionality they provide is pretty raw, low level stuff. Using the private SPIs behind the back of an application could readily lead ti inconsistent state in an application, leading to exciting events such as the Spontaneous Program Logic Application Termination (SPLAT).
"Documentation' for some of these really low level SPIs may consist of sophisticated mechanisms along the lines of "Hey! Andrew! How's this supposed to work?" (Yes, everything should be documented, and preferably folded into a CASE system that won't let you compile anything until the documentation is written. After all, that's how the LINUX kernel is managed, right?)
No doubt an Appleista will be along in due course to make clear the path to enlightenment.
You called?
The answer to your difficulty is obvious: follow the money. What strategic advantage does Apple gain by not publicly documenting these APIs? A corner on the windows management market? I'm sure is worth a whole lot because you can see how much Apple charges for it at the Apple Store. Oh, wait, you can't, cause there is no such separate competing product that Apple profits by leveraging their OS.
vs. Windows, where, let's see, they made a substantial amount of their $50 Billion on by selling Office--which required that they kill their competition in Office applications.
"But what of IE?", I hear you plaintively cry. "Doesn't Microsoft give that away for free?" Certainly. But their clear strategy was to use the product to own the web, and IE was the platform to do it.
When Apple sells a virtual desktop management tool, besides the OS, and doesn't document the APIs, you'd have an argument. For example, I imagine QT has access to things that WMP doesn't, but proving that is an exercise for the reader. As it is, you're just trolling. Speaking of simplistic arguments.
--
$tar -xvf
...what I really want is to be able to hit (for example) control-alt-F1 and get a full-screen command line. Every so often I've got a lot of crap open and I just want nothing but a big, empty, command window.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
I think the difference is this: does Apple use undocumented API's to purposely cripple, outright break, or unfairly compete with third party vendors. The list is long when dealing with microsoft, all the way back to dr. dos. if Apple did this, say a photo-manipulation app that competed with say, adobe, then there'd be room to bitch if if they used undocumented and secret API's to speed up rendering. or perhaps they have a secret network protocol that is used to connect os x clients to os x servers only. it sounds like their desktop features are designed to compete with windows, which makes the API's not relevant. and unlike windows, there really is a mac way to do things, which apple really wants to enforce. they don't care if someone is screwing around with xcode and comes up with some neato little $5 shareware app, they are concerned with big commercial apps and they want them to do things a certain way, the mac experience. like it or not, that's what they do. if it's to enforce application uniformity than that's hardly a sin.
My problem? I was perfectly gruntled, until some numbnuts came by and dissed me.
When has anyone said Microsoft's undocumented API's are evil? Are you trolling or just a twat? I guess if you see every Apple article as being a threat to your nerdy little world then making up bullshit arguments and playing the 'Mac Zealot' card is pretty much all you can do I guess. Are you this pathetically defensive in real life?
Shitram Brown, PhD
Professor of Mathematics
The functions in question are not API at all, by definition. Since they are explicitly not supported, they are not part of the Application Program Interface, and not documented (of course).
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
I know that it's fun to pick apart the framework code, make guesses as to what it's doing and how, and write code that exploits it in some way, but don't, don't, don't write a commercial product that depends on what you discover by this kind of spelunking, unless you are fully prepared to deal with the consequences of it breaking at any software update.
If your users call Apple because your program broke, Applecare will tell them to talk to you about it. If you ask Apple for help with an undocumented API, your request will be declined.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
You can sort of hack together that functionality using two windows. ex:
(first window) ktrace -f less.ktrace less foo.txt
(second window) kdump -l -f less.ktrace
Only thing is that you have to run the first one first - if you run kdump first, then start ktracing the comand, ktrace will respect kdump's read lock on the file, and you won't see anything. Which means that there's that little time at the start, during which you're not seeing "live" syscalls, but catching up to the backlog...
I needed more workspace, so I went multihead. Some apps lend themselves well to virts - web browsing, email, a pile of xterms... but when you're running something like photoshop and you need more "room to maneuver". adding extra heads is the way to go.
:-(
:-|
I seem to be the only OS X user that neither uses no likes Expose much. Part of it's the fact that a few apps I use bind to F11-13, though my BIG gripe is that F14-F16 ARE NOT MAPPING OPTIONS. Why can't I put the shortcuts for Expose onto the three keys that I NEVER use for ANYTHING?
That aside, I've noticed that virts are something the "I used to use freenix but the desktop sucks so I switched" crowd complains about, as well as sloppy focus and the fact that portables have one button trackpads (something of an annoyance if you're using X11 applications). As a whole, the freenix imports seem to be so used to doing things Their Way that the mere notion of a UNIX not having $feature makes them positively apoplectic.
It's not like you had to look far to see a post highly critical of MS hidden APIs - the immediately previous post in response to mine does so.
Trolling - nope. But it seems that not only is anything slightly critical of Apple seen as a troll in these here parts, but even a pointing out of the double standards that apply when judging Apple is seen as a troll.
This is slashdot - a feverent member of the open/free software world. If anybody suggested that any other vendor* should be able to keep APIs private and for their own use only the outrage would be immense. We're no even talking about truly internal APIs, but APIs that are used in the vendor's applications.
If the offender was Microsoft then it would be nerd apocolypse.
* The exception is, of course, google - they also get a free pass when indulging in questionable behaviour. It seems that there is a lot of "the enemy of my enemy is my friend" going on, with MS being the enemy. I'm old enough to remember when MS was in google's position, and IBM was the leviathan, crushing all before it. And I'm astute enough to realise the there will be a new enemy along in a minute, and the next generation of geeks will be vilifying google (or whoever - google are my bet) praising a new upstart through rose-tinted glasses, and slightly bemused that MS was once seen as an 800lb gorilla.
I could be wrong - google could become a benevolent mega-corp. But I'm not holding my breath.
Ah, the usual worldly-wise accusation of naivete. "You kids today, you just don't understand ..." Well, as another geezer, I'll weigh in.
* The exception is, of course, google - they also get a free pass when indulging in questionable behaviour. It seems that there is a lot of "the enemy of my enemy is my friend" going on, with MS being the enemy. I'm old enough to remember when MS was in google's position, and IBM was the leviathan, crushing all before it. And I'm astute enough to realise the there will be a new enemy along in a minute, and the next generation of geeks will be vilifying google (or whoever - google are my bet) praising a new upstart through rose-tinted glasses, and slightly bemused that MS was once seen as an 800lb gorilla.
Yes, because this is a practical way to think in the real world. It's not so much "the enemy of my enemy is my friend" as "no permanent allies, only permanent interests." When Apple, and Google, and IBM, act in ways which are in the community's interest, we support them. When they act in ways which are contrary to the community's interest, we oppose them. And because right now Microsoft can be counted on to act against the community interest at every turn, anyone acting against Microsoft is therefore acting for the community. No one thinks this is a permanent state of affairs, but it is indisputably the current situation.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
When I say things like the "community does this" and "we do that" people are swift to leap on the supposed fallacy of "you suppose slashdot posters all share the same opinion". Yet add the magic ingredient of "opposition to Microsoft" and shazam - it's OK. I did mention double standards earlier, didn't I? Well, what you just wrote was a great example.
Being without Desktop Manager on the Tiger development builds was one of my biggest peeves.
The bug I submitted to Apple was marked as "Third Party to Resolve" . . . Thank you Rich for resolving the issue!
Fine, substitute "informed users who pay attention to what's going on in the industry" for "community" above.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
Like many things, it depends on the intention for keeping it secret. Sometimes it's because things aren't stable enough to document. Sometimes it's because your documentation team is understaffed and can't get to it yet. Sometimes it's an illegal abuse of monopoly to give your companion products an unfair advantage.
Except for the complete and total lack of supported multiple desktop software.
70e808a22cb027cde4a6abddf6435d55
Big talk from someone we've never heard of.
The mac development community is small. We tend to notice who the major players are. You sir, are not one of those players.
Please take your, "l337er than thou" attitude back to your high school computer lab. You'll impress your peers with that kind of talk, but to pull it off here you'd better start naming programs that shock and delight.
"maybe your'e just not all that big" indeed. What trash. Sure, lots of cool programs are out that use undocumented APIs. But at the same time, some of the most successful OS X freeware and shareware does not. The single best way to discredit yourself is to talk trash like this.
Undocumented because they are not "public" apis, and not guaranteed to be supported in the next release of OSX. As long as that part is clear, go ahead.
But please, let's not complain about apple not documenting function calls.. apple is very clear about which APIs are official and supported.
Why can't I put the shortcuts for Expose onto the three keys that I NEVER use for ANYTHING? :-(
What I want to know is why Apple hasn't put a general purpose input or hotkey manager in Preferences that would let you map any key combo to any hotkey-using application.
Plus, every second keyboard these days has half a dozen extra "Multimedia" or "Internet" buttons. Why can't I map those to actions?
Why would anyone pay for a newsreader when Google Groups exists?
Um, because Google could arbitrarily change Google Groups at any moment and remove features that you depend on?
Oh, wait, Doctor Evil, that already happened.
(yes, it's their code, their hardware, they have a right to do it, the point is they can and do, and that's one good reason to do Usenet yourself instead of depending on the kindness of strangers)
Dud, if you're all raring to go out and start innovating all over Mac OS X, how about filling in some of the missing tools that can be done using supported APIs?
If it's so perfect, where is focus-follows-pointer? Seriously, the first thing I do at a Win or X11 box is set it for FFP and turn off auto raise. I've only found how to set this for Terminal, and perhaps X11 apps on OS X.
I've also found OS X seems to be behind in right button mouse support. I'm severly annoyed with Safari not having "Back" on the right button menu. I'll probably put Firefox on eventually, but wanted to give the native OS X a try.
This is not to say that I haven't found anything I like about OS X since my recent adoption of a Mini, just wanted to point out that I don't think it rates a "near perfect".
So what you're doing it coming from a Windows perspective and trying to make intelligent and topical assertions about the mac development platform.
This is a mistake. Disabuse yourself of such notions. They will get you nowhere. Apple has a very different policy towards its developers. They give as much power as they can do their developers, and the tools are so good that often applications do not need to go outside the standard APIs.
You might not believe me, because of the perspective you bring to the table. Let me try and illustrate why Apple approaches this differently:
Microsoft views third party developers as competition. Microsoft is a software company, so you compete directly with them when you write a Windows app that fufils a function they want to control. Therefore, they create hidden "advantages" in the OS that only select MS developers can access correctly.
Apple, on the other hand, is a hardware company. While OS X is a serious product for them, ultimately it's leverage to sell more computers. Therefore, they make their development tools very good, and encourage 3rd party developers. If a 3rd party developer makes a killer app, they get a direct benefit from it. This is in sharp contrast to Microsoft, who would actually lose business.
Apple's "hidden" APIs are only hidden because one one of two reasons:
Slashdot. It's Not For Common Sense
He has sucked the cock of Jesus Christ! Talk about Holy Water!