Slashdot Mirror
Apple Open Sources Grand Central Dispatch
bonch writes "Apple has open sourced libdispatch, also known as Grand Central Dispatch, which is technology in Snow Leopard that makes it easier for developers to take advantage of multi-core parallelism. Kernel support is not required, but performance optimizations Apple made for supporting GCD are visible in xnu. Block support in C is required and is currently available in LLVM (note that Apple has submitted their implementation of C blocks for standardization)." Update: 09/11 15:32 GMT by KD : Drew McCormack has a post up speculating on what Apple's move means to Linux and other communities (but probably not Microsoft): "...this is also very interesting for scientific developers. It may be possible to parallelize code in the not too distant future using Grand Central Dispatch, and run that code not only on Macs, but also on clusters and supercomputers."
I'm not too well versed in Cocoa development. I pushed some code that should have been in a separate thread into GCD, which requires you to use a block. All in all, I had to add an include, 1 line of code and a closing bracket.
Apple has made some seriously cool stuff here.
It's a library for task parallelism using a thread pool, introduced in Mac OS X 10.6 (Snow Leopard). Wikipedia tells all.
"We recognize that libdispatch is a new technology and you likely have many questions. Here are some documentation resources for getting started:
Introducing Blocks and Grand Central Dispatch
Concurrency Programming Guide
Grand Central Dispatch (GCD) Reference"
From the first paragraph:
Grand Central Dispatch (GCD) is a revolutionary approach to multicore computing that is woven throughout the fabric of Mac OS X version 10.6 Snow Leopard. GCD combines an easy-to-use programming model with highly-efficient system services to radically simplify the code needed to make best use of multiple processors. The technologies in GCD improve the performance, efficiency, and responsiveness of Snow Leopard out of the box, and will deliver even greater benefits as more developers adopt them.
libdispatch is the open source implementation of GCD.
Blocks are sections of program that can be passed around between functions as arguments. They basically allow 'functional' programming in C.
'Sensible' is a curse word.
Blocks:
In Snow Leopard, Apple has introduced a C language extension called "blocks." Blocks add closures and anonymous functions to C and the C-derived languages C++, Objective-C, and Objective C++.
Perhaps the simplest way to explain blocks is that they make functions another form of data. C-derived languages already have function pointers, which can be passed around like data, but these can only point to functions created at compile time. The only way to influence the behavior of such a function is by passing different arguments to the function or by setting global variables which are then accessed from within the function. Both of these approaches have big disadvantages
Full Read: http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/10
Directly in line with blocks is Grand Central Dispatch (and this is, where blocks become really usefull):
GDC is a a technology to resolve the concurrency conundrum by giving programmers a very easy way to split tasks into multiple sub-tasks which can then be loaded onto different threads/cpu. All this also works with normal threading, but GDC makes the process far easier, with the intention to prepare OSX for future multicore machines:
http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/12
It does so by using blocks as separate tasks:
http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/13
"When I first heard about Grand Central Dispatch, I was extremely skeptical. The greatest minds in computer science have been working for decades on the problem of how best to extract parallelism from computing workloads. Now here was Apple apparently promising to solve this problem. Ridiculous.
But Grand Central Dispatch doesn't actually address this issue at all. It offers no help whatsoever in deciding how to split your work up into independently executable tasksâ"that is, deciding what pieces can or should be executed asynchronously or in parallel. That's still entirely up to the developer (and still a tough problem). What GCD does instead is much more pragmatic. Once a developer has identified something that can be split off into a separate task, GCD makes it as easy and non-invasive as possible to actually do so.
The use of FIFO queues, and especially the existence of serialized queues, seems counter to the spirit of ubiquitous concurrency. But we've seen where the Platonic ideal of multithreading leads, and it's not a pleasant place for developers.
One of Apple's slogans for Grand Central Dispatch is "islands of serialization in a sea of concurrency." That does a great job of capturing the practical reality of adding more concurrency to run-of-the-mill desktop applications. Those islands are what isolate developers from the thorny problems of simultaneous data access, deadlock, and other pitfalls of multithreading. Developers are encouraged to identify functions of their applications that would be better executed off the main thread, even if they're made up of several sequential or otherwise partially interdependent tasks. GCD makes it easy to break off the entire unit of work while maintaining the existing order and dependencies between subtasks." (source = above url)
I'd recommend reading the relevant section of the ars technica review of mac os x 10.6
-- Sorry, I can't think of anything funny to say here.
ArsTechnica always does a pretty thorough and reasonably technical review of each OSX release, and the latest one gives a pretty good explanation of GCD as well as Blocks.
http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars
The GCD stuff in particular starts on page 12, but the previous couple pages give a little bit of useful background on why it's important.
One time I threw a brick at a duck.
It's OpenCL that opens up the GPUs to general processing on 10.6. Although GCD certainly plays a role by dispatching threads to those resources.
You should check out this astounding OpenCL demo here: http://www.macresearch.org/opencl_episode1
"Apple has open sourced libdispatch, also known as Grand Central Dispatch, which is technology in Snow Leopard that makes it easier for developers to take advantage of multi-core parallelism."
First line of THE SUMMARY.
I know it's not hip to RTFA, but it's at least a minimum requirement to read the very next line after the title, even while scrolling down eagerly to make a comment.
Well, I can see the logic of your concerns but Apple do actually seem to be fairly good at open sourcing infrastructure-related things. Sure they maintain a tight control on the user-facing stuff that makes Apple products distinctive - on the iPhone they even maintain a tight control on applications. But bear in mind they're working on an open source kernel, employ developers to work on the LLVM compiler (open source), open sourced an init-ish daemon (launchd) they developed, etc etc. On stuff that's "for geeks" they seem fairly enlightened wrt open source.
It's quite surprising from a company like Apple but the fact that they manage to make surprising decisions like that looks like a strong technical management team at work, to me.
Everyone who reads slashdot isn't an OSX ween and has no idea what "Grand Central Dispatch" is. Perhaps a sentence or two describing why it is important/useful would save users from following the link which doesn't provide that info either.
Look I'm as annoyed by poor summaries as anyone, but it seems almost reflexive to complain about them these days. The summary clearly said it, "makes it easier for developers to take advantage of multi-core parallelism". I don't care if you've never heard of OS X and no nothing about it. That sentence right there tells you what effect it has and why it's useful. After that it's just details as to where and if it is applicable to some other project. I guess what I'm saying is, I thought this was a pretty decent summary, enough to know if you should read about it.
Apple has a long history of open source contributions since OS X. Apple has released parts of OS X as Darwin under a BSD license since they first released OS X. OS X was developed from OPENSTEP which came from NextStep which itself was based on BSD. The kernel itself is derived from XNU which was based on the Mach kernel. All of these components are covered by BSD licenses. From what I understand since Apple uses a lot of open source Unix programs like CUPS, etc, they do contribute to fixes and patches on a regular basis.
Well, there's spam egg sausage and spam, that's not got much spam in it.
also known as Grand Central Dispatch, which is technology in Snow Leopard that makes it easier for developers to take advantage of multi-core parallelism.
They kinda' snuck that one in.
I drank what? -- Socrates
You don't think that libdispatch will be very genial to widespread usage, as it has a lot of OS-specific calls, which is an understandable position to take. But as an alternative you offer something whose "only caveat" is that it needs an entirely different compiler to build. A compiler whose most recent activity dates from two years ago.
... How is that a superior alternative?
GCD is designed for small, short-running blocks of code. Read the Ars article on Snow Leopard for examples. Naturally, it will also handle longer-running threads gracefully.
Whoever modded you informative is as ignorant as you.
Shoot, I already noticed the difference on my 2.5 yr old Mac Pro (1.1). First boot on 10.6 and I was like "wow, feels like a new machine again". All of the bundled apps have been recompiled (64 bit) and cleaned up (and apparently take advantage of GCD everywhere possible). I really didn't think I would see that much of a difference with 10.6 and really only upgraded because I could for $29 (I mean at that price, why not right?) I am very happy with my $29 purchase thus far. I've only had to work through a couple app incompatibilities (and as I have been able to work around them just fine, I am happy.) This is of course just my experience thus far with 10.6. I have no hard benchmark numbers for you. But I noticed right away the smoothness it brought to my older Mac Pro. And it was an easier upgrade than going from 10.4 --> 10.5.
You are posting in a thread about the fact that Apple made their implementation open source and you are claiming vendor lock-in?
Are you one of those rabid Apple-haters we see so often around here? Or are you just amazingly stupid?
No, GDC is purely a "C with block extensions" API. The blocks are essentially anonymous methods you can pass directly in to functions. It's integrated at a much lower level than Objective-C, which is only used for the higher-level application framework in MacOS.
Apple open-sourced the GDC API with this announcement, block extensions to C with LLVM implementation last week, and the OS support necessary as part of the xnu kernel Darwin release for 10.6.
E pluribus unum
GCD and OpenMP have very little in common. OpenMP is a language extension. It requires the programmer to understand what environment their program is going to run in, what variables can be shared and how, etc. GCD merely asks you to identify blocks of code that are independent and it handles parsing them out to threads, variable replication, etc. It's the difference between providing detailed blueprints of a car (the OpenMP way) and just saying "I want a car" (the GCD way). You can *almost* think of GCD as a user-friendly frontend for OpenMP.
What if you're running two applications that both are capable of monopolizing all your cpu time? How will your app know that it's only going to get 50% of the available cpu time form the OS, so it should only start threads for half the cpus?
GCD decides how many threads a collection of tasks should be split across. If an app running on an 8-core machine wants to run 100 tasks, then they could be spread across anywhere from 1 to 8 threads, depending on what else is running. Since it's the OS that knows what else is running, it can make more intelligent decisions about how many threads should be running.
Grand central dispatch has many innovations, but the key feature it provides is that thread pooling is now handled by the OS not the program. This means that in a dynamic environment you don't have each application stepping on each other when they ask for too many threads --all total-- than the multi-core system can optimally handle. So if Mail asks for fifty threads and Firefox asks for fifty threads and CPU you are running on can realistically only handle 10 threads then GCD figures out how to manage things so you don't get a spinning beachball.
It turns out a lot of tricks were required to do this including a lot of things like just in time compiling LVM and this C-Blocks stuff, but that's way over my head.
Some drink at the fountain of knowledge. Others just gargle.
I've come to really like GCD; I haven't played with it much in Cocoa (Obj-C) but I've been moving some of the stuff I wrote a long time ago in C to use it and I think I can say that what it does is *really* *really* awesome. It helps when writing code to be run in parallel; it does is not help you in determining *what* should be done in parallel. By putting your work into queues, by way of closures (yeah, blocks, whatever...I'm sticking with the closure name), it's up to the underlying OS to determine what thread gets what work, and on what processor. Having worked with multithreaded stuff on Windows, and calling GetThreadAffinityMask or whatever it was, and being told that it's just a *hint* to the OS, which is free to ignore you, which it always did, GCD really does spread out the work evenly among my 16-proc MacPro, and then turns around and does it just as well on the dual-core mini.
I've wanted something like this for years; a really decent OS thread scheduler that divides up the work on the other processors in a sensible fashion. I was even looking into how much effort it would take to write something like this from scratch for Linux, and now I don't even have to. Sweet!
Caveats: This is in OS X only, so no iPhone GCD (at least, not yet...not really necessary until we have multi-core iPhones), and while I've lived with additions to C++ through the years (templates mostly), the idea of adding, well, anything to C seems strange, let alone something as run-time dependent as closures.
I'm honestly surprised how ignorant and lazy the regular slashdotter has become with the years.
Any self-respected geek should be already keeping up to date with Apple advancements which are and will be impacting techology in the years to come.
If you people haven't noticed already, Apple has been consistently releasing libraries and server software as open source projects for the rest to pick up , use and modify, with liberal licenses.
A friend of mine used to say (can't remember exactly... paraphrasing:)
* Microsoft wants all software to be theirs
* GNU wants all software to be free
* BSD wants all software to be better
And releasing GCD, gentlemen, is another master stroke by Apple, just like WebKit, Bonjour, LLVM, the list goes on, to share knowledge and advance technology by merit, not by forcing it down your throat thanks to the monopoly you have been handed.
The term "block" is familiar to Ruby programmers. It's an old concept which Ruby has made easy to use and hence popular and actually useful.
And here's another lesson which OpenBSD, Apple and Ruby have been putting to work without you noticing guys: any technology that is difficult to use, no matter how good it is, will not be used if gets in your way; the technology must be easy to deploy/use and unobstrusive to be actually used and useful.
Just remember SELinux and how many people just disable it, no matter how good it is (which I don't think it is, but that's for another rant). Then compare it with the technology that OpenBSD has been implementing for memory protection which is unobstrusive and ready to use with no extra configuration. Same with Ruby blocks, which more programmers are using and a lot of software is benefitting from it now, even though higher order functions and closures have been around for ages.
Having Ruby-like blocks in C and Objective-C is so COOL, you must appreciate that if you think you're serious at programming. Apple has already submitted it to be a standard. I believe MacRuby will benefit from this too, which is Ruby written in Objective-C, which implements Ruby classes as Objective-C classes, achieving incredible speed, taking advantage of Objective-C and LLVM technologies.
Now, I want my late '90s Slashdot back please, where you could more easily find insightful and informative comments. There's a lot of garbage and Microsoft apologists nowadays.
The best way to predict the future is to invent it