Slashdot Mirror

← Back to Stories (view on slashdot.org)

Mac OS X Secretly Cripples Non-Apple Software

Posted by Soulskill on from the hand-in-the-cookie-jar dept.

spikedLemur writes "Vladimir Vukicevic of the Firefox team stumbled upon some questionable practices from Apple while trying to improve the performance of Firefox. Apparently, Apple is using some undocumented APIs that give Safari a significant performance advantage over other browsers. Of course, "undocumented" means that non-Apple developers have to try and reverse-engineer these interfaces to get the same level of performance. You really have to wonder what Apple is thinking, considering the kind of retaliation Microsoft has gotten for similar practices.

32 of 559 comments (clear)

  1. first post! by Anonymous Coward · · Score: 5, Funny

    first post!
    i cheated though, i'm using safari.

    1. Re:first post! by FF0000+Phoenix · · Score: 5, Funny

      It's even more incriminating that all the undocumented APIs are under the "Firefox_Sux" namespace.

    2. Re:first post! by dgatwood · · Score: 5, Insightful

      Or the SPI (System Programming Interface, the private equivalent of an API) takes advantage of inside knowledge of how some data structures are designed but which could change in the future as functionality is added to the class in question. For example, Apple might decide to change CFString to always convert data to BADC-byte-order UTF-32 under the hood for better efficiency on Vax. Not likely, but I never thought I'd see Macs using Intel CPUs, either, so you never know. :-D They could make such a change and still support the public APIs, but if they had an API that allowed you to arbitrarily manipulate the bytes under the hood, they'd be stuck.

      The thing about Safari is that it is effectively insulated from SPI changes because it comes out with OS releases. If Apple needs to change an internal data structure in CF, Foundation, etc. in a way that would break an SPI, all they have to do is rev Safari to not use that SPI. Thus, it is safe for Safari to use any API or SPI. If Apple publishes the SPI as API, FireFox uses it, and Apple changes the data structure, Firefox breaks, and with "luck", so do Photoshop and Word.... :-(

      So you see, Apple has only three choices: A. don't publish that portion of the API, in which case some people complain because they're not able to get that extra 1% from being able to walk inside private data structures of the HFS+ Extents B-tree or whatever, B. publish that portion of the API, in which case they're stuck with that internal architecture and can't ever change it to improve performance, add features, etc., or C. publish the API and break it later, in which case developers scream again. It's a no-win.

      The only thing one could possibly argue is that Safari shouldn't be using the SPI, either, to put them on equal footing. That said, since it's safe for them to do so, where's the harm? There's no monopoly involved, certainly. :-) And as you noted, many of those SPIs that Safari is trying out might become APIs at some point in the future. Having an app like Safari exercise them allows the engineers to figure out what works and what doesn't so that they don't get stuck supporting an API that isn't scalable, is hard to enhance, or isn't easily maintainable. In the long run, everybody benefits.

      I'm certain that Apple doesn't do this to cripple Firefox or to make its own software look better. It's not a vast fruit-wing conspiracy.... Apple limits its public API exposure to ensure that the APIs are sustainable so third-party code doesn't break. If you don't care about that, use the SPI... just don't come crying when your app crashes on launch after a software update or whatever.... :-)

      --

      Check out my sci-fi/humor trilogy at PatriotsBooks.

    3. Re:first post! by dhavleak · · Score: 5, Insightful

      Great post.

      I've been in product cycles where we've gone through exactly the dilemma you pointed out: where making an API public means supporting it until kingdom come, when the scenario is too new for the API to be stable, or you have definite long-term plans that will cause breaking changes in the API and you don't want the burden of having to be backwards compatible with applications designed for the older API.

      The only part I didn't agree with was this:

      The only thing one could possibly argue is that Safari shouldn't be using the SPI, either, to put them on equal footing. That said, since it's safe for them to do so, where's the harm? There's no monopoly involved, certainly. :-) And as you noted, many of those SPIs that Safari is trying out might become APIs at some point in the future.

      I think it should either be ok for all players to have internal APIs or not ok for all players. I mean, if we say that right now it's ok for Apple to do this because they are not a monopoly, what happens if they do become one? At that point do they get penalized for these internal APIs (using which they designed the products that helped them obtain the monopoly)? At what point will Apple cross a threshold at which they need to change this practice and how will they know when they have crossed it? And when this threshold is crossed, is it suddenly ok for MS to start this practice again (of having undocumented APIs).

    4. Re:first post! by darthflo · · Score: 5, Insightful

      What about the other option?

      "The SPI namespace is subject to change with future OS releases, minor as well as major. Changes will be made public on this site at least two weeks prior to shipping an update with any SPI modifications. Enter your e-mail address to be auto-notified about all changes, grouped into two e-mails per week at most."

      Developers have documented access to the additional performance those libraries might give them while knowing they might change. For safety's sake, the versioning could be included into the interface, allowing thoughtful devs to fall back to the (slower) API on a version number update. Also, the documentation could include some sort of discussion plattform allowing qualified devs to propose changes that might, after approved by the Steve and his Turtleneck, improve Apple's SPI (and, trickling down, the "stable" API).

  2. Article is a Troll by Anonymous Coward · · Score: 5, Insightful

    Oh give me a break, if you use an undocumented API for something that does not mean you "cripple" other pieces of software. It's not like OS-X says "oooo Firefox, quick make it run twice as slow". Grow up.

    1. Re:Article is a Troll by Anonymous Coward · · Score: 5, Insightful

      Yes, the article is a troll at face value. Apple has every right to keep its API secret from 2nd and 3rd parties. It is true however that Microsoft has been widely criticized for not opening up its APIs that give Office, IE, etc. and advantage. What the article is doing is predicting that Apple will be given a pass by the development community, thereby allowing the author to scream "hyprocrisy!" Of course none of this has happened yet, except for your comment. So yeah, the article is a troll.

    2. Re:Article is a Troll by truthsearch · · Score: 5, Informative
      Possibly confirming your post is this comment on TFA:

      The programmatic disabling of coalesced updates should not be public API. It's actually a very dangerous thing to do. We aren't really happy with that code in WebKit, but we had to do it to avoid performance regressions in apps that embedded WebKit. Technically it's wrong though, since we turn off the coalesced updates for any app that uses WebKit! This includes drawing they do that doesn't even use WebKit.

      As for the window display throttling, that was a pref designed for Safari (that we don't even use any more). It's not private or magic. It's nothing more than a pref that we can examine from Safari-land, so linking to that is just silly. ;)

      Many of the private methods that WebKit uses are private for a reason. Either they expose internal structures that can't be depended on, or they are part of something inside a framework that may not be fully formed. WebKit subclasses several private NSView methods for example, and it cost us many many man hours to deal with the regressions caused by the internal changes that were made to NSViews in Leopard.

      As you yourself blogged, there was a totally acceptable public way of doing what you needed to do.

      For any private methods we use that we think should be public, we (the WebKit team) file bugs on the appropriate system components. Many of these methods have become public over time (CG stuff in Leopard for example). Be careful when you dig into WebKit code, since we may continue to use the WK method even though it's not public API just because we need to work on Tiger.


      --
      Developers: We can use your help.
    3. Re:Article is a Troll by pavera · · Score: 5, Insightful

      The difference here is that the "problem" that firefox was hitting is a completely documented FEATURE and has been around since 10.4.

      Also, There is a 100% documented, public, and simple way to disable the feature. The Firefox dev found this configuration, added 2 lines of XML to firefox, and bam, done, speedy. So I really don't see the comparison to MS at all.

      Also one of the comments on the blog is from a webkit developer at Apple who says "yeah, these APIs basically suck, and they are here for backwards compatibility with Tiger, and they aren't stable, and cause us hundreds of hours of work dealing with regressions, so don't use them, use the perfectly acceptable and documented configuration setting, if there is anything in these APIs that should be made public, it will be once it is stabilized and reliable" He then gives examples of other APIs that have gone through the same process.

      In the end this is 100% open to the public, any software can use this configuration setting to get around this potential performance bottleneck. The reason FF3 was "suddenly" slower than FF2 is they changed from Carbon to Cocoa (2 totally different frameworks) and the new feature is only applied to Cocoa apps. So in short, FF changed hundreds (probably thousands) of lines of code to use a new framework, and found a performance bottleneck, and then found the documentation about it, and changed configuration to avoid the bottleneck... How this is news at all baffles me, that sounds like a normal day in my life.

  3. The thinking is really quite simple: by Valacosa · · Score: 5, Funny

    You don't really have to wonder what Apple is thinking, considering the kind of marketshare Microsoft has gotten for similar practices.
    Fixed.
    --
    "Live as if you'll die tomorrow." Ridiculous. You could die later today.
  4. Round Number? by cobaltnova · · Score: 5, Funny

    We're capping at 30.77 frames per second here. That's way too round of a number.
    The author must be a mathematician.
  5. Re:the difference by spoco2 · · Score: 5, Insightful

    And is that the answer to everything is it? They can do anything that they feel like purely because they are not the dominant player... so all of Microsoft's underhanded playing early on when they weren't the dominant player is all excusable too is it?

    It's ridiculous to try and use this insane rationale in regards to any company that's not Microsoft. At what point do you then start going 'well, actually I've decided they have enough market share now, NOW they should be ethical'

    Bar and truly humbug

  6. Re:the difference by Valacosa · · Score: 5, Insightful

    Monopoly or no, it's undeniable that using secret APIs to give your own software an edge is anti-competitive. Not having a monopoly on the desktop market might mean that it's not illegal, but the legality has no bearing on the ethics.

    --
    "Live as if you'll die tomorrow." Ridiculous. You could die later today.
  7. Um, is this an emulation thing? by pedropolis · · Score: 5, Insightful

    From tfa: "The reason why Firefox 2 wasn't affected was that Fx2 was not a Cocoa app"

    So writing this from a native perspective introduced new APIs found in tech notes you should have read in the first place before writing and running into performance issues?

    1. Re:Um, is this an emulation thing? by Goaway · · Score: 5, Insightful

      The function to turn this on programmatically is undocumented. You're not supposed to do that, you're supposed to set a flag in the app metadata to turn it on. WebKit does it programmatically because it can be embedded in any app.

  8. From the fucking comments by norkakn · · Score: 5, Informative

    David Hyatt
    Feb 28th, 2008 at 1:24 pm
    The programmatic disabling of coalesced updates should not be public API. It's actually a very dangerous thing to do. We aren't really happy with that code in WebKit, but we had to do it to avoid performance regressions in apps that embedded WebKit. Technically it's wrong though, since we turn off the coalesced updates for any app that uses WebKit! This includes drawing they do that doesn't even use WebKit.

    As for the window display throttling, that was a pref designed for Safari (that we don't even use any more). It's not private or magic. It's nothing more than a pref that we can examine from Safari-land, so linking to that is just silly. ;)

    Many of the private methods that WebKit uses are private for a reason. Either they expose internal structures that can't be depended on, or they are part of something inside a framework that may not be fully formed. WebKit subclasses several private NSView methods for example, and it cost us many many man hours to deal with the regressions caused by the internal changes that were made to NSViews in Leopard.

    As you yourself blogged, there was a totally acceptable public way of doing what you needed to do.

    For any private methods we use that we think should be public, we (the WebKit team) file bugs on the appropriate system components. Many of these methods have become public over time (CG stuff in Leopard for example). Be careful when you dig into WebKit code, since we may continue to use the WK method even though it's not public API just because we need to work on Tiger.

    1. Re:From the fucking comments by norkakn · · Score: 5, Insightful

      There is a public way to do the same thing. They just added a total hack to the API to automatically do something by default when WebKit is embedded, instead of requiring a configuration value to be set. They didn't want it to be publicly available since they want the call to die as soon as they figure out a better way to do it. This isn't MS style stuff. There is no hidden feature. You can run the exact same code in a public way, and the it won't break when your user upgrades WebKit.

      So, no, you aren't getting it right.

    2. Re:From the fucking comments by norkakn · · Score: 5, Insightful

      Yeah, that's pretty much exactly why I posted it. IMHO, Apple has been quite good with private APIs. In a major upgrade, they tend to all either become public (often after changing), or die. MS has had a less open history, and I think there are some very valid complaints there, but some are certainly overstated.

    3. Re:From the fucking comments by Graff · · Score: 5, Insightful

      So WebKit is tightly integrated with the underlying OS service like Internet Explorer is alleged to be with Windows. So, if you use WebKit, you benefit from the private, "better" linkage to the OS service, but if you don't, your performance (or perhaps other qualities) will suffer. Am I getting that right? No, you have it wrong.

      Webkit is a framework that is open for anyone to take and put into their own application. Safari and some components of Mac OS use WebKit for their own rendering of html. There is no private, "better" linkage to WebKit, there is just a hack within the WebKit framework that is there so that other applications using WebKit will not have problems with it. The Apple developers knew the internals of the operating system well enough to do this semi-safely but even they aren't happy with themselves doing it because it can still cause problems.

      There is also a public, safer, more documented way of doing the same thing for applications that don't use WebKit. This public method is not perfect either but it is much safer. The Apple developers are keeping parts of the operating system under wraps which could cause major problems if you don't know EXACTLY what the internals are doing. This is a very common thing for responsible developers to do, you don't want to expose API that could fail catastrophically if something isn't set up just exactly correctly.

      In short, nothing to see here, the public API is the safest bet to use. If you choose to use undocumented methods for a bit more speed then you risk bringing down your application in a hard and messy way. The WebKit developers weighed that in their own minds and decided that the risks were worth it, since they had a hand in developing the undocumented methods and could account for the quirks in a semi-safe manner.

      In the case of Internet Explorer, Microsoft had a separate set of completely safe API that were optimized versions of methods other developers got to use. If you were an internal Microsoft developer you could do more with the internal API than anyone could with the external API. This was done deliberately so that Microsoft products could get preferential treatment on the Windows operating system. Microsoft also made it so that you couldn't easily use Windows without having some part of Internet Explorer as part of the system. Under Mac OS X you can remove every mention of WebKit and all that will happen is a couple of programs won't work until you re-install them with their embedded versions of WebKit.

      --
      Sapere aude!
  9. Indeed, where's the advantage? by Foerstner · · Score: 5, Funny

    In Microsoft's case, the goal of keeping "secret" APIs was pretty clear: whoever controls the Windows browser market, controls the browser market, period.

    I can just see Steve Jobs rubbing his hands and gloating to his minions..."Yes, and with Firefox handicapped, we will have five percent of the browser market all to ourselves! Ours...all ours! Muahahahaha!"

    --
    The US free market: two halves of a government-granted duopoly are free to set the market price.
  10. Re:Really? by Ctrl-Z · · Score: 5, Funny

    If you read the article...

    You lost me there.

    --
    www.timcoleman.com is a total waste of your time. Never go there.
  11. *sniffle* by Serenissima · · Score: 5, Funny

    Awwwwww... our little system is all grown up now! Who's a good system? Who's a good system?

    --
    Give a man a fire and he'll be warm for a day. But light a man on fire and he'll be warm for the rest of his life.
  12. Re:Really? by dal20402 · · Score: 5, Informative

    The Safari performance improvements are coming in Safari 3.1, not yet available to the public. To see them today, you have to be running current WebKit nightlies. The difference between the new WebKit builds and vanilla Safari 3.0.4 is pretty dramatic.

  13. Article points finger in wrong directoin by goombah99 · · Score: 5, Informative

    The Slashdot summary is accusing Apple of reserving the tasty bits for safari, but the article shows that it's webkit not safari that knows the shorcuts. Anyone is free to use webkit. it's apples optimized interface for applications. If Firefox chooses not to use it well I can understand why but they need to accept that their interface may not be as optimized.

    Indeed what apple is doing does not seem that out of whack. They have an interface that is optimized for stability not speed. That's the proper way to do it. and they figure out how one can tweak it for speed. Do you make that the defaults or do you put those in a container like webkit where one can manage the tradeoffs better? duh...

    --
    Some drink at the fountain of knowledge. Others just gargle.
  14. Re:From TFA... by Anonymous Coward · · Score: 5, Insightful

    ACRONYMS! THE GOGGLES, THEY DO NOTHING!
    Seriously though, your post was really hard to read. When you referred to OS X as "X", I was thinking "X Windows". Please, for the sake of everyone here and Slashdot reputation, declining or not, refrain from using such atrocious techniques. Really, who uses "%" instead of typing "percentage"? It's not that hard.

  15. Re:From TFA... by Anonymous Coward · · Score: 5, Insightful

    Yes! There definitely needs to be a -5 learn to fucking type mod.

  16. Re:Display Throttling? by johne_ganz · · Score: 5, Insightful

    You are bang on.

    Display coalescing was introduced way back in 10.4. This was well documented, and as the Firefox developer points out, there's even tools provided to enable and disable this behavior to see if you're being limited by it.

    While I don't know all the details, I would venture to say that far from Apple "Secretly Crippling Non-Apple Software" (which they conveniently provide release notes and debug tools with the stock Xcode tools for this secret behavior), this has exposed a serious performance problem in Firefox.

    To put it another way, display coalescing will effectively penalize apps that are rendering more updates that are physically possible to display. As an extreme example, this roughly equivalent to rendering / writing 60 frames of video to the display in 1/60th of a second, which can only possibly display one of those 60 frames. The rest are just wasting CPU, GPU, and bandwidth resources which could be better spent doing other things. Display coalescing will cause an application to "stall" if it's forcing too many updates, the call to flush the buffer just won't return until the the current frame has fully rendered.

    Mac OS X prior to 10.4 did not enforce this, and so as one of those compatibility trade-offs, display coalescing is turned off by default when certain conditions are detected. I believe anything linked prior to 10.4 will trigger it, and carbon apps. Carbon is essentially for those who are unwilling to (almost literally) join the 21st century.

    Just how secret can it possibly be with publicly available release notes published years ago?

    http://developer.apple.com/technotes/tn2005/tn2133.html
  17. Re:From TFA... by unfunk · · Score: 5, Funny

    (yes, there are women on Slashdot, and yes, there are naked pictures, too),. I demand proof!
  18. Slashdot Pressure by superbrose · · Score: 5, Insightful

    I think one of the main reasons acronyms are frequently used on slashdot boils down to the pressure of posting a slashdot comment as early as possible.

    After all, the longer it takes to post a comment, the more comments it will be competing with and the chances of it being read (modded up) dwindle.

    In fact, even if this was a great comment by now the chances of it reaching +5 Insightful are pretty slim.

    Maybe great late comments do deserve more generous attention though.

  19. Slashdot ... has completely misunderstood... by Futurepower(R) · · Score: 5, Insightful

    Quote from the article:

    "Edit: Slashdot seems to have picked up on this, and in typical style, has completely misunderstood the post. To be clear, I do not think that Apple is in any way trying to purposely "cripple" non-Apple software. I also do not think that undocumented APIs give Safari any kind of "significant performance advantage" (as Firefox 3 should show!). However, as I said, the undocumented functionality could be useful for Firefox and other apps to implement things in an simpler (and potentially more efficient) manner. I don't think this is malicious, it's just an unfortunate cutting of corners that is way too easy for a company that's not fully open to do."

    Slashdot has a reputation: "Slashdot ... in typical style, has completely misunderstood the post."

    It amazes me that, after all these years, Slashdot editors still apparently do not do any research before they post the stories. That has reduced the value of Slashdot as an advertising medium enormously.

  20. Re:From TFA... by Tore+S+B · · Score: 5, Funny

    IATWTC? I Ate the World Trade Center?

    --
    toresbe
  21. Remember the World of Warcraft rant? by Overly+Critical+Guy · · Score: 5, Interesting

    Whenever someone brings up Slashdot's journalistic integrity, I'm reminded of CmdrTaco's World of Warcraft rant from a few years ago, Blizzard Made Me Change My Name. An entire 12-paragraph article because Blizzard made him change his name for using a title in it (Cmdr), which is against the naming rules. That CmdrTaco thought it was news, or that he felt so slighted that he thought he'd "strike back" at Blizzard by posting an article about it on Slashdot because he's CmdrTaco of Slashdot, gosh-darned it, just really showed me how immature and lame this site is.

    The headline "Mac OS X Secretly Cripples Non-Apple Software" will go out on all the feeds. A lot of people won't even read the article. They'll look at it and say "See, typical tyrannical Apple." They may skim the summary and head straight for the comments. And they will be unknowingly misinformed.

    --
    "Sufferin' succotash."