How Encrypted Binaries Work In Mac OS X

An anonymous reader writes "By now we know that OS X uses encrypted binaries for some critical apps like Dock, Finder and LoginWindow. Amit Singh explains the implementation of this protection scheme which makes use of the AES crypto algorithm and a special memory pager in Mach. The so called Do Not Steal Mac OS X (DSMOS) kernel extension helps along the way by decrypting things for the special pager when apps get executed. A funny thing is that if you print the pointer at address 0xFFFF1600 in your own app you get as output Apple's karma poem for crackers! According to the article there are 8 protected binaries in OSX including Rosetta and Spotlight meta data demon. Interestingly Apple's window server is NOT one of those."

Comments

One reason not to encrypt the windowing system

[runlevel 5]

WM's are huge apps and decrypting one before every startup would add a lot of work that has to be done at boot. According to the article, "the SystemUIServer binary within SystemUIServer.app", is encrypted and that is presumably a larege component of the WM. Also, it's virtually useless without the the dock and finder anyway.

Re:One reason not to encrypt the windowing system

[Trillan]

No, SystemUIServer is the process that runs Apple's menu do-dads, like the battery indicator, volume menu, iChat menu, keychain menu, clock, spotlight menu... basically, everything in the top right corner. Except for menus that 3rd party applications add, which are always to the left of the SystemUIServer items.

Originally, developers could inject their own menus into it if they figured out Apple's undocumented API for it. However, Apple shut that down (in 10.2, I think) since an unstable menu would destabilize all of Apple's menus. They're all run in the same address space, presumably to allow Apple to cut some corners in their command-drag reordering system. After 10.2, some developers hacked it to allow them to inject other menus into it. Maybe that's what Apple is trying to stop.

Even so, it's a really odd pick for encryption.

Re:How hard is reverse engineering?

[dwandy]

Just wondering. How easy is it to reverse-engineer a massive closed-source piece of software (like, say, MS Windows)?

Trivial ... just takes time to "re-code" it ... a lot of time ... check out http://www.winehq.com/ who are in fact reverse engineering Windows.

Such a reverse-engineering job would be of obvious commercial interest (especially to parties who work in countries with lax regulatory regimes), so there is an obvious incentive to do it.

Why reverse engineer when you can just print copies? There's very little commercial interest in this...

Re:How hard is reverse engineering?

[s20451]

So you want to RE a proprietary solution specifically to sell it in a region which is known for its "loose" ethics toward piracy?

I'm not interested in re-selling the proprietary solution. I'm interested in selling detailed information about the proprietary solution.

Reverse engineering makes security holes more obvious (does it not? Otherwise, how do hackers find security holes?). This is of obvious interest to "industrial" crackers -- the ones who harness large botnets and sell them to the highest bidder. It's a multi-million dollar business.

Re:Signed binaries = good, encrypted binaries = ba

[theshibboleth]

Actually Apple made it famous. Xerox invented the GUI on the Alto.

Re:That poem is scary..

[bdash]

The US Treasury would disagree with you: http://www.ustreas.gov/education/faq/currency/lega l-tender.shtml#q1. Then again, what do they know?

Re:Signed binaries = good, encrypted binaries = ba

[jcr]

They nicked it from Xerox.

Correction: Apple LICENSED technology from Xerox, and develeoped the GUI far beyond what Xerox had done.

-jcr

Re:Signed binaries = good, encrypted binaries = ba

[fsterman]

GUIs were around in academia long before Xerox. Xerox, not knowing what to do with all this stuff coming from the lab, invested in Apple and let them wander through. None of that made it into the myth, kinda anti-climatic.