Slashdot Mirror
Apple Open Sources Grand Central Dispatch
bonch writes "Apple has open sourced libdispatch, also known as Grand Central Dispatch, which is technology in Snow Leopard that makes it easier for developers to take advantage of multi-core parallelism. Kernel support is not required, but performance optimizations Apple made for supporting GCD are visible in xnu. Block support in C is required and is currently available in LLVM (note that Apple has submitted their implementation of C blocks for standardization)." Update: 09/11 15:32 GMT by KD : Drew McCormack has a post up speculating on what Apple's move means to Linux and other communities (but probably not Microsoft): "...this is also very interesting for scientific developers. It may be possible to parallelize code in the not too distant future using Grand Central Dispatch, and run that code not only on Macs, but also on clusters and supercomputers."
Everyone who reads slashdot isn't an OSX ween and has no idea what "Grand Central Dispatch" is. Perhaps a sentence or two describing why it is important/useful would save users from following the link which doesn't provide that info either.
I want those 5 seconds back! :)
My understanding of GCD is that it makes parallel programming and, importantly, interacting with the UI, pretty damned easy.
And despite that, if the only thing you can think of to do with Blocks is threading, then you seriously need to get back to learning your CompSci. Resource Allocation Is Invocation becomes a load more practical, and that alone would mark a major shift between C code that's mostly resource and error management overhead and code that actually does something.
Why would Apple suddenly open-source a major feature of their new OS? Did they get busted violating the GPL? It doesn't make sense that the most closed-source draconian control-freak vendor of all would suddenly open-source major parts of their OS without some legal Sword of Damocles hanging over their head. What aren't we being told here? Whose code did they get caught stealing?
Someone somewhere owes the community an explanation!
Having done some multiprogramming, I have to tell that it is really an end-user technology. Less a tool for developer.
One of the biggest problems on multi-CPU/core systems if to split appropriately CPUs between applications. That requires quite amount of testing and benchmarking. Then you simply configure max number of threads each application allowed to use. Obviously changing anything at later time when problem was found requires some effort too.
With GDC that all now is much easier. Available CPU resources are presented to applications in a fashion of real-time batch queue. One doesn't need to configure thread pools per application anymore.
That was never a problem from software development point of view - we just provide the 'max threads' parameter. But that was always problem from user/operator point of view who has to actually fill in the parameter.
All hope abandon ye who enter here.
It's an old tech. But it's different this time around.
Old thread pools are per process. This is a thread pool for the whole system. And that's new.
IOW, with GCD you do not need to configure every application how much threads it should start. Applications do not need to bother with it anymore too: they simply queue batch tasks as they arrive and GCD guarantees that they will be executed. Without overloading system.
Shortly, GCD is a system-wide replacement for old per-application thread pool configuration. Makes applications simpler and also doesn't force end-user to understand all oddities of multi-programming to get most out of their boxes.
All hope abandon ye who enter here.
no need for kernel support
open source implementation
yeah. sounds a helluvalot like vendor lock-in to me
You don't think that libdispatch will be very genial to widespread usage, as it has a lot of OS-specific calls, which is an understandable position to take. But as an alternative you offer something whose "only caveat" is that it needs an entirely different compiler to build. A compiler whose most recent activity dates from two years ago.
... How is that a superior alternative?
You are posting in a thread about the fact that Apple made their implementation open source and you are claiming vendor lock-in?
Are you one of those rabid Apple-haters we see so often around here? Or are you just amazingly stupid?
I like how you refer to a "cult" but lash out with some disproportionate animosity to something, which you fully admit, you don't know anything about.
Sounds like pretty cult-like behavior.
Slashdot still doesnâ(TM)t support Unicode after it was added to the HTML standard in 1997.
What if you're running two applications that both are capable of monopolizing all your cpu time? How will your app know that it's only going to get 50% of the available cpu time form the OS, so it should only start threads for half the cpus?
GCD decides how many threads a collection of tasks should be split across. If an app running on an 8-core machine wants to run 100 tasks, then they could be spread across anywhere from 1 to 8 threads, depending on what else is running. Since it's the OS that knows what else is running, it can make more intelligent decisions about how many threads should be running.
So you're saying gcc is being used for vendor lock in?
"Judging from the examples, GCD looks much cleaner and simpler. But that often comes with a price."
I think you answered your own question. Apple is kind of fanatical for cleaner and simpler and not just in their user interfaces. It's nice. Apple APIs are some of the best I've ever used.
The question is... how is this different to Intel's Thread Building Bocks, or OpenMP, both of which are better supported and more widely available to non-Mac developers.
If I'm not mistaken the technologies are for an application.
GCD can coordinate all applications running on the same system.
I guess having their own implemntation (for the mac) makes sense, as they can integrate it throughout the OS. I don't know anything about GCD either, but could it be used in Linux to make that a more parallel-friendly OS with less developer effort, and more standardisation of parallel execution?
Theoretically yes.
Apple here is in unique position.
Most software developers care solely about their own application. Old example from the desktop. On Windows I have 7-zip archiver installed. I have dual core CPU and this 7-zip is configured to use the 2 cores. From prospective of software developers it's all what they can do: let users tell how much cores/CPUs can be used. But I also have a video encoding application installed - and also configured to use two cores. If I try to run them both in parallel, that would cause erroneous amount of context switching harming performance of both the tasks. In worst case that might make my desktop completely unresponsive. As user I'm also lazy to reconfigure every time applications how many CPUs they should use.
Apple itself now produces number of applications which can utilize multiple CPUs (iTunes audio conversion, iMovie/QuickTime/FC video conversion, etc) and obviously they run into the problem that when applications left on their own to decide how much CPU resources they should use, system would overload leading to all the effects. Requiring user to reconfigure all the applications all the time is also kind of stupid.
Since Apple is in control of the OS and applications - and their own software might suffer from the problem, they went out and implemented the solution: system-wide batch queue with a thread pool. They are still threads - local to the process - but they are scheduled on system-wide basis. You do not need to configure applications how many CPUs they should use - nor applications have to think about: they simply put tasks (to be threads) of the queue of GCD.
I'm using the 'batch queue' term because this is the closest what exists now. Though classical UNIX batch queues are different in nature: those are processes and they are executed at some unknown point of time. GCD is real-time in its nature and its threads run immediately, unlike traditional batch queues which wait for system to be idle.
All hope abandon ye who enter here.
it does is not help you in determining *what* should be done in parallel.
I'd disagree slightly on that point; agreed, the technology doesn't actually make certain code blocks suddenly go "me! me! me!" but it does significantly ease the burden on the developer in making those decisions. I can throw something into the parallelization engine with a couple lines of code to see what difference that makes.
The key example would be something like this one here, which Slashdot's filter doesn't like (it says please don't use so many junk characters, bah).
That's a nice and simple change to make, and is easy to drop into any code which previously required synchronous execution. I can just wrap the existing synchronous code in two calls to dispatch_async(), one to throw stuff onto the background thread, and another to throw the last step back onto the foreground thread. If it doesn't help the program I can remove it. If it does help, I can leave it in.
For the usual reason that OSs handle resources. Because the OS is in a position to share out resources between applications in a reasonable way. If each app does it's own thing, then it's either a free for all, and pretty much guaranteed to be inefficient.
It also of course has the other usual benefit of libraries. It means that programs don't have to reinvent the wheel every time they write an app. GCD makes writing multi-threaded code using thread pools vastly simpler than before.
Good support for OpenMP or any of the existing shared memory parallel programming libraries would have been much cleaner and portable.
This seems to be one of the common complaints in this discussion ("why not contribute to an existing project instead?") and others involving Apple's open-source participation, and certainly it has merit. But open-source developers write their own more-or-less redundant tools all the time with no better justification than "I didn't like how tool X handled the specific case Y, so I wrote my own tool from scratch" - SourceForge is littered with them. So (I'm responding generally, not to the parent) I don't see why Apple seems to be given an inordinate amount of grief over this - they didn't like how the existing stuff worked, so they wrote their own. Same thing they've done with launchd for that matter. Or moving away (somewhat) from gcc.
Also, having written their own tool, they released it under a license they preferred. To borrow a phrase frequently used in defense of the GPL - if you don't like the license, you're free to not use the tool and/or go write your own.
#DeleteChrome
That's great and all, but systems have been doing this for years. When I launch a thread on Linux I don't care where it ends up. The scheduler takes care of it. Same with Perl, pthreads, OpenMP and pretty much every other threading technology I've ever used.
What's new here?
I'm honestly surprised how ignorant and lazy the regular slashdotter has become with the years.
Any self-respected geek should be already keeping up to date with Apple advancements which are and will be impacting techology in the years to come.
If you people haven't noticed already, Apple has been consistently releasing libraries and server software as open source projects for the rest to pick up , use and modify, with liberal licenses.
A friend of mine used to say (can't remember exactly... paraphrasing:)
* Microsoft wants all software to be theirs
* GNU wants all software to be free
* BSD wants all software to be better
And releasing GCD, gentlemen, is another master stroke by Apple, just like WebKit, Bonjour, LLVM, the list goes on, to share knowledge and advance technology by merit, not by forcing it down your throat thanks to the monopoly you have been handed.
The term "block" is familiar to Ruby programmers. It's an old concept which Ruby has made easy to use and hence popular and actually useful.
And here's another lesson which OpenBSD, Apple and Ruby have been putting to work without you noticing guys: any technology that is difficult to use, no matter how good it is, will not be used if gets in your way; the technology must be easy to deploy/use and unobstrusive to be actually used and useful.
Just remember SELinux and how many people just disable it, no matter how good it is (which I don't think it is, but that's for another rant). Then compare it with the technology that OpenBSD has been implementing for memory protection which is unobstrusive and ready to use with no extra configuration. Same with Ruby blocks, which more programmers are using and a lot of software is benefitting from it now, even though higher order functions and closures have been around for ages.
Having Ruby-like blocks in C and Objective-C is so COOL, you must appreciate that if you think you're serious at programming. Apple has already submitted it to be a standard. I believe MacRuby will benefit from this too, which is Ruby written in Objective-C, which implements Ruby classes as Objective-C classes, achieving incredible speed, taking advantage of Objective-C and LLVM technologies.
Now, I want my late '90s Slashdot back please, where you could more easily find insightful and informative comments. There's a lot of garbage and Microsoft apologists nowadays.
The best way to predict the future is to invent it
That's exactly my point. With cooperative multitasking, if you know everything that's going on, it can be more efficient than preemptive multitasking. Just like manually managing your threads can be more efficient than something like GCD.
At the most fundamental level, cooperative is more efficient (if well programmed), just as manually managing threads (if well programmed) is more efficient.
It's that "if well programmed" that's the killer. If your environment is completely known, *and* you are skilled, you have the potential to do a better job by hand. But in the real world, were your software is going to run on all sorts of computers with all sorts of different software and processing capabilities, you can't fine tune your program, so letting the system handle it works better.
Looking at it differently, with cooperative multitasking or manually controlling threads, *if* you have complete knowledge of the system *and* you are sufficiently skilled, you can approach 100% efficiency. With cooperative multitasking, or thread management similar to GCD, you might reach only 90% efficiency. But if you have to build a cooperative multitasking or manually manages threads in a program, that is going to run on all sorts of computers, you may only manage perhaps an overall average 70% efficiency.
And the fact that you can get that 90% efficiency with much less effort than you have to put in to get maybe around 70% (or whatever) efficiency, the benefits here are obvious.